Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [rmf-dev] performance testing

Dear RMF developers,

I just created a bug entry to follow up this discussion: https://bugs.eclipse.org/bugs/show_bug.cgi?id=414183

@ Michael: Can you create the milestone 0.9.0 in the bugzilla management, please?

Best regards,
Lukas

On 08/01/2013 10:09 AM, Lukas Ladenberger wrote:
Hi Michael and François,

first of all: I think benchmarks are very useful. Said did already some stuff on this topic with respect to our content provider.  The JUnitBenchmark library looks very interesting at a first glance. However, in my opinion we should delay this patch to RMF 0.9.0, because:

(1) the patch using JUnitBenchmarks requires at least junit 4.9. However, the current RMF target platform includes version 4.8.2 (which is part of the Eclipse JDT v.3.7.2). Plese note: Our current build is still based on eclipse 3.7 (indigo). Nevertheless, the orbit update site includes junit 4.10, which could be used as well.

@ François: Do you use the RMF indigo target development definition?

(2) Our plan is to build RMF 0.9.0 on top of Kepler. The Kepler update site includes a newer version of JDT as well as a newer version of Junit.

(3) I guess, that the JUnitBenchmark library is not part of the Eclipse Orbit Project. As a consequence, we need a CQ for the library, especially for the customized version for working with OSGI. I think this should be no problem, since the library is only for testing/benchmark purposes.

I will create a new bug entry covering this discussion.

Best regards,
Lukas

On 07/31/2013 09:54 PM, Michael Jastram wrote:
Hi Lukas,

Have you seen this?

The second one depends on the first and adds benchmarking using JUnitBenchmarks. If you find this useful, you may want to merge it, but I we may want to hold it.
This is because JUnitBenchmarks is still in development, and also because I had to patch it to make it work with OSGi, as I explained in the gerrit message:

I committed the first - what do you think of the second, with respect to our CI  infrastructure?

Cheers,

- Michael

On 30.07.2013 16:46, François Rey wrote:
I've pushed two new change request:
ReqIF10Util: added tests for setting values, factored out common code.
ReqIF10UtilTest: added benchmarking using JUnitBenchmarks.
The first one is just about additional tests to ReqIF10UtilTest. This should be trivial to approve and merge.
The second one depends on the first and adds benchmarking using JUnitBenchmarks. If you find this useful, you may want to merge it, but I we may want to hold it.
This is because JUnitBenchmarks is still in development, and also because I had to patch it to make it work with OSGi, as I explained in the gerrit message:
ReqIF10Util has recently been changed to use EMF reflective api. To get
an idea of how this affects performance, the ReqIF10UtilTest is now
benchmarked using the JUnitBenchmarks utility library residing at:
https://github.com/carrotsearch/junit-benchmarks
However due to class loading issues when running within OSGi, a fix is
required to JUnitBenchmarks so it can properly display timing results.
The fix can be found at:
https://github.com/fmjrey/junit-benchmarks/tree/osgi
The runtime jar included in this plugin contains the fix. While it makes
basic usage possible, BenchmarkOptions annotations are still not
working, therefore the following JVM arguments may be useful for those
wanting to specify global values:
-Djub.rounds.benchmark=500 -Djub.rounds.warmup=5

In the end, using such setup, I was able to see no major impact on performance with the recent changes I've made to ReqIF10Util.
Now off to presentation stuff...
Cheers,
François


_______________________________________________
rmf-dev mailing list
rmf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/rmf-dev


--
Dr. Michael Jastram +49 (162) 274 83 94 http://jastram.de
Geschäftsführer Formal Mind GmbH http://formalmind.com
Vorsitzender rheinjug e.V. http://rheinjug.de
Project Lead Eclipse Requirements Modeling Framework http://eclipse.org/rmf



Back to the top