Re: [platform-releng-dev] Garbage collection in meter's methods.

[Ikuo Yamasaki <yamasaki.ikuo@xxxxxxxxxxxxx>]

Hi,

Will you have any plan to implement what I requested ?

Since I have several tests where I would like to evaluate not only CPU
time but also (stationary) memory consumption, I need this function.
--------------------------------------
In meter.start(),
  1. do gc.
  2. record current Used Java Heap.
  3. record current CPU time.

In meter.stop(),
  1. record current CPU time
  2. do gc, (if the option is set)
  3. record current Used Java Heap.
--------------------------------------

It is acceptable for me that it is not included in AutomatedTest (if
included, it might be preferred in future) but possible in command-line
based test.

In addition, please give developpers of test cases (like me) a way to
record results(e.g. CPU time, Used Heap Size) into the csv file
specified by the developper for each test case.

# e.g. Prepare meter.setLogCsv(File file).

On Thu, 15 Jun 2006 21:44:55 -0400
Jeff McAffer <Jeff_McAffer@xxxxxxxxxx> wrote:

Jeff> Doing it in setup() is ok but how to do you do it at the end?  teardown() 
Jeff> is too late.  As Ikuo pointed out, you need to do it after the time 
Jeff> measurements but before the memory measurements.
Jeff> 
Jeff> Jeff
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> John Arthorne/Ottawa/IBM@IBMCA 
Jeff> Sent by: platform-releng-dev-bounces@xxxxxxxxxxx
Jeff> 06/15/2006 06:12 PM
Jeff> Please respond to
Jeff> "Eclipse platform release engineering list." 
Jeff> <platform-releng-dev@xxxxxxxxxxx>
Jeff> 
Jeff> 
Jeff> To
Jeff> "Eclipse platform release engineering list." 
Jeff> <platform-releng-dev@xxxxxxxxxxx>
Jeff> cc
Jeff> 
Jeff> Subject
Jeff> Re: [platform-releng-dev] Garbage collection in meter's methods.
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> It definitely makes sense to perform a gc before starting the measurement 
Jeff> if you are testing for heap usage.  I suspect the current performance 
Jeff> meter doesn't do this because most of our current tests are interested 
Jeff> only in elapsed time. Note that even with explicit gc, the heap 
Jeff> measurements are often not reliable due to various caches and weak/soft 
Jeff> references used in the platform.  It seems to be very difficult to 
Jeff> reliably measure memory/heap usage in automated tests.  The setUp() method 
Jeff> sounds like a good place to do a gc. 
Jeff> 
Jeff> John 
Jeff> 
Jeff> 
Jeff> 
Jeff> Ikuo Yamasaki <yamasaki.ikuo@xxxxxxxxxxxxx> 
Jeff> Sent by: platform-releng-dev-bounces@xxxxxxxxxxx 
Jeff> 15/06/2006 12:23 PM 
Jeff> 
Jeff> Please respond to
Jeff> "Eclipse platform release engineering list." 
Jeff> <platform-releng-dev@xxxxxxxxxxx>
Jeff> 
Jeff> 
Jeff> To
Jeff> platform-releng-dev@xxxxxxxxxxx 
Jeff> cc
Jeff> 
Jeff> Subject
Jeff> [platform-releng-dev] Garbage collection in meter's methods.
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> 
Jeff> I'm now doing performance test regarding Equinox service registry by
Jeff> extending org.eclipse.core.tests.harness.PerformanceTestRunner. I would
Jeff> like to measure both of heap memory consumption and CPU time for my own 
Jeff> test()
Jeff> method.
Jeff> 
Jeff> As for the current implementation of PerformanceMeters, System.gc is not
Jeff> called in meter.start()/stop(). Therefore, the results of "Used Java
Jeff> Heap" might include memory allocation for objects to be garbage
Jeff> collected in future. Although I understand there might be a case that it
Jeff> must include memory allocation for those (kind of instantaneous), it's
Jeff> not what I want (kind of constantly).
Jeff> 
Jeff> What I want the methods to do is : 
Jeff> 
Jeff> In meter.start(),
Jeff>                 1. do gc.
Jeff>                 2. record current Used Java Heap.
Jeff>                 3. record current time.
Jeff> 
Jeff> In meter.stop(),
Jeff>                 1. record current time
Jeff>                 2. do gc,
Jeff>                 3. record current Used Java Heap.
Jeff> 
Jeff> # GC must not affect the result of CPU time.
Jeff> 
Jeff> Now, could anyone tell me what I should do ?
Jeff> or do you have any plan to change the implementation of 
Jeff> PerformanceTestRunner.
Jeff> 
Jeff> P.S. Currently, Only thing I can do is calling System.gc() in my setUp()
Jeff>                 method.
Jeff> 
Jeff> ---------------------
Jeff> NTT Cyber Solutions Laboratories
Jeff> 
Jeff>     Ikuo YAMASAKI
Jeff>        E-mail: yamasaki.ikuo@xxxxxxxxxxxxx
Jeff> 
Jeff> 
Jeff> _______________________________________________
Jeff> platform-releng-dev mailing list
Jeff> platform-releng-dev@xxxxxxxxxxx
Jeff> https://dev.eclipse.org/mailman/listinfo/platform-releng-dev
Jeff> _______________________________________________
Jeff> platform-releng-dev mailing list
Jeff> platform-releng-dev@xxxxxxxxxxx
Jeff> https://dev.eclipse.org/mailman/listinfo/platform-releng-dev
Jeff> 

---------------------
NTT Cyber Solutions Laboratories

     Ikuo YAMASAKI
        E-mail: yamasaki.ikuo@xxxxxxxxxxxxx