Re: [platform-releng-dev] again on session tests

[Nick Edgar <Nick_Edgar@xxxxxxxxxx>]

Note that the Workbench has a very simple handshaking protocol for running 
the tests once the workbench is ready.  There is currently no support for 
inter-process communication though.
See ITestHarness and TestableObject.  The design goals behind this (small) 
API were to provide maximum decoupling between the test harness, the 
workbench, and the RCP app.  For example, the RCP app doesn't need to be 
aware of the test harness at all, and the workbench does not need to know 
about the PDE test runner.  Ideally the PDE test runner wouldn't have to 
know about the workbench either, but it currently needs to obtain the 
TestableObject from PlatformUI, so there's a dependency there.
Although these types currently exist at workbench level, they have no 
dependencies on other workbench types.  This was done in the hopes that 
they could eventually be generalized and pushed down.   Maybe now is the 
time.

The control flow is:
- eclipse runs the test harness app (e.g. the PDE test runner), telling it 
which main app to run (e.g. the IDE)
- the test harness sets itself as the ITestHarness implementation on the 
workbench's TestableObject: 
PlatformUI.getTestableObject().setTestHarness(this)
- the test harness runs the main app (an RCP app, of course)
- the RCP app creates and runs the workbench as usual 
(PlatformUI.createAndRunWorkbench(Display))
- the workbench initializes itself, restores any previously saved state, 
etc.
- just before the workbench is about to run the event loop, it notifies 
the test harness that it can run the tests (ITestHarness.runTests())
  - this is actually done in a forked thread
- the test harness tells the TestableObject that it is about to start 
testing (TestableObject.testingStarting())
  - this allows the workbench to put itself into test mode, e.g. make sure 
error dialogs don't pop up and hang the tests
- the test harness tells the TestableObject to run the tests 
(TestableObject.runTest(Runnable))
  - currently, due to difficulty in breaking up the flow of control of 
JUnit tests organized in a suite, all the tests are run in the same 
runnable
  - the workbench's TestableObject runs the tests in an asyncExec, since 
the UI tests expect to be run in the UI thread
  - since runTests() was called from a forked thread, the workbench fell 
through to processing UI events, so it will process the asyncExec when 
there are no more OS events
- the test harness tells the TestableObject that it is done testing 
(TestableObject.testingFinished())
  - the workbench reverts back to non-test mode

Does this in any way map onto the kind of inter-process communication 
you're doing now?
Thoughts?

Nick




Rafael Chaves/Ottawa/IBM@IBMCA 
Sent by: platform-releng-dev-admin@xxxxxxxxxxx
09/08/2004 11:33 AM
Please respond to
platform-releng-dev


To
platform-releng-dev@xxxxxxxxxxx
cc

Subject
[platform-releng-dev] again on session tests










Platform/Core has been doing some work on running JUnit tests that end up
spawning a second instance of Eclipse, or "session tests". There is a lot
of scenarios that can only be tested with session tests, as startup
performance tests.

We have been doing some work in that area, and were curious if others had
the same need, and what approach was (or is planned to be) used to support
that.

Here is the approach we are taking:
- session tests use a special SessionTestSuite class to build their test
suites
- session tests are actually just proxies for the actual tests that know
which plug-in/class/method implements the test case
- when a session test is run, it launches a second instance of Eclipse to
run a application that is/has a test runner. This test runner will run 
that
single test case and report back the test result to the original instance
using a socket (right now we use org.eclipse.pde.junit.runtime test
applications and protocol for doing that, but it causes an undesirable
dependency on a non-platform plug-in). Once the test case has finished,
this second Eclipse instance terminates
- back to the original instance, we check whether the test failed/passed.
If it failed, we fake a corresponding error/failure to occur at the proxy
test, so it is reported as a regular test error/failure to the JUnit
framework

This way, session tests don't need any special support other than using 
the
SessionTestSuite class when building their test suites. Also, the current
support for running automated tests as part of the RelEng test framework 
or
by using the PDE/JDT support in the SDK still works.

Any feedback is welcome.

Rafael

_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/platform-releng-dev