Re: [platform-releng-dev] performance tests launching a 2nd instance of

[Andre Weinand <eclipse@xxxxxxxxxxx>]

Hi Rafael,

In a second iteration I've added more API to 
org.eclipse.performance so that my (probably 
naïve) StartupTest reduces to:


import org.eclipse.test.performance.*;

public class StartupTest extends PerformanceTestCase {

	public void testStartup() throws Exception {
	    stopMeasuring();
	    commitMeasurements();

assertPerformanceInRelativeBand(Dimension.ELAPSED_PROCESS, 
-50, +5);
	}
}


The new method assertPerformanceInRelativeBand 
(both on PerformanceTestCase as well as the 
Performance façade) lets you specify the 
dimension and the lower and upper bound of the 
band as a percentage.

So in the example from above, the test will fail 
if the startup time takes 5% longer or becomes 
50% faster than some reference value (the 50% 
lower bound was chosen because such a dramatic 
performance improvement most likely is an 
indication for some other problem and not a real 
improvement; however if such an improvement is in 
fact possible, just specify "-100, +5"  :-)

Let me know whether this works for you.
Thanks,
--andre




At 18:20 Uhr -0400 31.8.2004, Rafael Chaves wrote:
> Platform/Core is working on startup performance tests. This requires being
> able to launch a second instance of Eclipse whose startup time will be
> measured (session test).
>
> We could imagine two different approaches for that: the test case launches
> a second instance, some code running in this second instance does the
> measurements, and somehow communicates the results back to the test case,
> running in the first instance. The test case would then make assertions on
> the resulting data. Another would be to do all the measuring and assertions
> in this 2nd instance, the test case in the first instance being only
> responsible for spawning the second instance.
>
> What is the vision for session tests?  I know the performance test
> framework is an ongoing work, but how could one achieve that now (using
> internal API if needed)?
>
> Thanks,
>
> Rafael
>
> _______________________________________________
> platform-releng-dev mailing list
> platform-releng-dev@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/platform-releng-dev