0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

Resource(CPU & Memory) monitoring/

Posted at
### Get the pid of the relevant batch.

$ ps -aux  | grep java
top command to monitor CPU and memory information (the target in the example is pid: 3067)
$top -p 3067

image.png

For All Process
① 3872540 total > All physical memory(約3.8GB)
② 4190204 total > Memory in the swap space(約4.1GB)
③ 1562088 free > empty space (約0.15GB)
④ 2406996 used > Memory used by all (batch) processes on the server.(約2.4GB)
⑤ 1308640 > Buffer/cache area memory allocated by the OS(約1.3GB)

The formula is

TOTAL(①) = free(③) + used(④) + buff/cache(⑤)
For Each pid Process
Column for each PID: Memory allocated for each PID
VIRT: amount of virtual memory: ex) 3136304  (約3.1GB) 
RES : physical memory ex) 316760  (約0.31GB) 
%CPU:  used CPU(%) (約0.0%)

The formula is

%MEM(約8.2%) =  Amount of physical memory allocated to pid: 23819 (約0.31GB)  / All physical memory(約3.8GB)
Check the actual used memory by JSTAT command.
$ jstat -gc 46702
Automatically execute commands one second at a time with the watch command.
$ watch -n 1 jstat -gc 46702
Check the used space.

(709[S0U] + 0.0[S1U] + 11459.1[EU] +41758.9[OU]) / 1024  = 52.66MB ...*1
*1: 52.66MB is much lower than allocated to this pid (約0.31GB). This is thought to be Java application is allocating the enough memory as a buffer space

Ref: https://qiita.com/kaikusakari/items/9b7c3d1fb524eb6aa348

Result of Migration test on batch

While file(about 10000 lines at each file) is continuously sent to server and batch is processing...

%CPU is roughly 50%. 
If large amounts of data are linked in other batches at the same time, delays may occur, but if large amounts of data are linked in a single batch, there is no problem.

%VIRTUAL MEMORY is same as normal(8.2%)

Memory while file processing do not exceed the amount of allocated memory by java application. 
Real used memory is about 100 MB while batch is processing. 
0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?