[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [platform-releng-dev] again on session tests
|
>+ *may* be natural in the true Junit philosophy (I confess to not fully
getting it)
The JUnit philosophy is that tests should leave the world exactly how they
found it.
This is support is built-in into JUnit and is explained (better than I
could
ever do it) in:
http://martinfowler.com/bliki/JunitNewInstance.html
Being that pure is not always practical. In particular when you
have to start a workbench for a test. So what we do is to run more than one
test in a single workspace. However, in this case it is your responsibility
to clean-up properly. One way to do it is explained in chapter 12 of
contributing
to eclipse:
http://today.java.net/today/2004/02/02/ch12Eclipse.pdf
>Don't we also have something like this to run the test suites in releng
mode?
org.eclipse.test releng includes corresponding applications which use an
Ant aware
TestRunner (EclipseTestRunner).
--erich
Jeff McAffer
<Jeff_McAffer@ca.
ibm.com> To
Sent by: platform-releng-dev@xxxxxxxxxxx
platform-releng-d cc
ev-admin@eclipse.
org Subject
Re: [platform-releng-dev] again on
session tests
09/09/2004 04:11
AM
Please respond to
platform-releng-d
ev@xxxxxxxxxxx
Mechanically this seems fine. It is basically using the mechanisms that we
have already. There is a question as to how people want to organize tests.
Using the support outlined here, each method in a TestCase class will spawn
a different VM. This has good (+) and bad (-) points.
+ simple
+ *may* be natural in the true Junit philosophy (I confess to not fully
getting it)
- makes writing/composing complex test "scenarios" harder as you do not get
to use the TestSuite and TestCase grouping/ordering mechanisms (e.g., if
you write a startup test, a CVS check out test and a full build test, you
cannot somehow then group these into a suite without forcing multiple VMs).
- may make it hard to talk about "scenarios" as a scenario would then just
map onto a particuarl method on a TestCase class.
As for the mechanics, depending on PDE is a bummer for this. We should
seek to push that function down or duplicate. Don't we also have something
like this to run the test suites in releng mode?
Jeff
Rafael Chaves/Ottawa/IBM@IBMCA
Sent by:
platform-releng-dev-admin@eclipse.o To
rg platform-releng-dev@ecli
pse.org
cc
09/08/2004 11:33 AM
Subject
[platform-releng-dev]
Please respond to again on session tests
platform-releng-dev
Platform/Core has been doing some work on running JUnit tests that end up
spawning a second instance of Eclipse, or "session tests". There is a lot
of scenarios that can only be tested with session tests, as startup
performance tests.
We have been doing some work in that area, and were curious if others had
the same need, and what approach was (or is planned to be) used to support
that.
Here is the approach we are taking:
- session tests use a special SessionTestSuite class to build their test
suites
- session tests are actually just proxies for the actual tests that know
which plug-in/class/method implements the test case
- when a session test is run, it launches a second instance of Eclipse to
run a application that is/has a test runner. This test runner will run that
single test case and report back the test result to the original instance
using a socket (right now we use org.eclipse.pde.junit.runtime test
applications and protocol for doing that, but it causes an undesirable
dependency on a non-platform plug-in). Once the test case has finished,
this second Eclipse instance terminates
- back to the original instance, we check whether the test failed/passed.
If it failed, we fake a corresponding error/failure to occur at the proxy
test, so it is reported as a regular test error/failure to the JUnit
framework
This way, session tests don't need any special support other than using the
SessionTestSuite class when building their test suites. Also, the current
support for running automated tests as part of the RelEng test framework or
by using the PDE/JDT support in the SDK still works.
Any feedback is welcome.
Rafael
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/platform-releng-dev
