[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [platform-releng-dev] again on session tests
|
The following statement confuses me:
>The main issue here (as I see it) is that all tests inherently run in a
session. If we want to have control over these we need to be able to
>control how they are run (e.g., the command line used to launch Eclipse).
You already have this control in the test.xml file. You can define
different tasks for different test sessions.
For example, here is the test.xml from the jdt.text.tests. The performance
tests include "cold" tests to
open the java editor. These tests are very similar to the start-up tests
you are discussing and
they must be run in a fresh VM.
<target name="performance-suite">
<property name="jdt-text-performance-folder"
value="${eclipse-home}/jdt_text_performance_folder"/>
***one session ***
<ant target="ui-test" antfile="${library-file}"
dir="${eclipse-home}">
<property name="data-dir"
value="${jdt-text-performance-folder}"/>
<property name="plugin-name" value="${plugin-name}"/>
<property name="classname"
value="org.eclipse.jdt.text.tests.performance.PerformanceTestSuite"/>
</ant>
*** another session ****
<ant target="ui-test" antfile="${library-file}"
dir="${eclipse-home}">
<property name="data-dir"
value="${jdt-text-performance-folder}"/>
<property name="plugin-name" value="${plugin-name}"/>
<property name="classname"
value="org.eclipse.jdt.text.tests.performance.OpenJavaEditorTest"/>
</ant>
</target>
Also to do this there is no dependeny on pde or the pde.junit.runtime. The
applications started by these tasks
are included in the org.eclipse.test plug-in.
What am I missing?
--erich
Jeff McAffer
<Jeff_McAffer@ca.
ibm.com> To
Sent by: platform-releng-dev@xxxxxxxxxxx
platform-releng-d cc
ev-admin@eclipse.
orgwever, Subject
Re: [platform-releng-dev] again on
session tests
09/09/2004 05:23
AM
Please respond to
platform-releng-d
ev@xxxxxxxxxxx
Yes, we have the work item to do startup tests. They should be possible in
the very near future. (e.g., tomorrow). The main issue here (as I see it)
is that all tests inherently run in a session. If we want to have control
over these we need to be able to control how they are run (e.g., the
command line used to launch Eclipse).
We have seen elsewhere the notion of "scenario" and "variations" where the
scenario was the basic function operations to do and the variations could
be anything from what VM to use to different command line (e.g., VM) args
to what workspace to use etc. The discussion below has bearing because it
implies that scenario = method on TestCase since a new VM is launched for
each test* method. See my other note for comments on whether or not this
is an optimal mapping.
An alternative is to launch a VM for every TestSuite. This allows
composition of Tests to build a scenario. That may or may not be
desirable/natural.
Jeff
Nick Edgar/Ottawa/IBM@IBMCA
Sent by:
platform-releng-dev-admin@eclipse To
.org platform-releng-dev@eclipse
.org
cc
09/08/2004 04:25 PM
Subject
Re: [platform-releng-dev]
Please respond to again on session tests
platform-releng-dev
The UI team wants to have similar session tests, e.g. time to bring up a
simple RCP app, or time to bring up the IDE.
We have held of investigating our own solution for this in the hopes that
you would provide an answer <g>.
Nick
Rafael Chaves/Ottawa/IBM@IBMCA
Sent by: platform-releng-dev-admin@xxxxxxxxxxx
09/08/2004 11:33 AM
Please respond to
platform-releng-dev
To
platform-releng-dev@xxxxxxxxxxx
cc
Subject
[platform-releng-dev] again on session tests
Platform/Core has been doing some work on running JUnit tests that end up
spawning a second instance of Eclipse, or "session tests". There is a lot
of scenarios that can only be tested with session tests, as startup
performance tests.
We have been doing some work in that area, and were curious if others had
the same need, and what approach was (or is planned to be) used to support
that.
Here is the approach we are taking:
- session tests use a special SessionTestSuite class to build their test
suites
- session tests are actually just proxies for the actual tests that know
which plug-in/class/method implements the test case
- when a session test is run, it launches a second instance of Eclipse to
run a application that is/has a test runner. This test runner will run
that
single test case and report back the test result to the original instance
using a socket (right now we use org.eclipse.pde.junit.runtime test
applications and protocol for doing that, but it causes an undesirable
dependency on a non-platform plug-in). Once the test case has finished,
this second Eclipse instance terminates
- back to the original instance, we check whether the test failed/passed.
If it failed, we fake a corresponding error/failure to occur at the proxy
test, so it is reported as a regular test error/failure to the JUnit
framework
This way, session tests don't need any special support other than using
the
SessionTestSuite class when building their test suites. Also, the current
support for running automated tests as part of the RelEng test framework
or
by using the PDE/JDT support in the SDK still works.
Any feedback is welcome.
Rafael
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/platform-releng-dev
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/platform-releng-dev
