tycho-surefire:test

Full name:

org.eclipse.tycho:tycho-surefire-plugin:0.22.0:test

Description:

Executes tests in an OSGi runtime.

The goal launches an OSGi runtime and executes the project's tests in that runtime. The "test runtime" consists of the bundle built in this project and its transitive dependencies, plus some Equinox and test harness bundles. The bundles are resolved from the target platform of the project. Note that the test runtime does typically not contain the entire target platform. If there are implicitly required bundles (e.g. org.eclipse.equinox.ds to make declarative services work), they need to be added manually through an extraRequirements configuration on the target-platform-configuration plugin.

Attributes:

  • Requires a Maven project to be executed.
  • Requires dependency resolution of artifacts in scope: runtime.
  • Binds by default to the lifecycle phase: integration-test.

Optional Parameters

Name Type Since Description
appArgLine String - Arbitrary applications arguments to set on the command line.
application String - Eclipse application to be run. If not specified, default application org.eclipse.ui.ide.workbench will be used. Application runnable will be invoked from test harness, not directly from Eclipse. Note that you need to ensure that the bundle which defines the configured application is included in the test runtime.
argLine String - Arbitrary JVM options to set on the command line.
User property is: tycho.testArgLine.
bundleStartLevel BundleStartLevel[] - Bundle start level and auto start configuration used by the test runtime. Ignored if testRuntime is p2Installed.
debugPort int - Set this parameter to suspend the test JVM waiting for a client to open a remote debug session on the specified port.
User property is: debugPort.
deleteOsgiDataDirectory boolean - Whether to recursively delete the directory osgiDataDirectory before running the tests.
Default value is: true.
dependencies Dependency[] - Additional dependencies to be added to the test runtime. Ignored if testRuntime is p2Installed. Note: This parameter has only limited support for dependencies to artifacts within the reactor. Therefore it is recommended to specify extraRequirements on the target-platform-configuration plugin instead. Example:
<plugin>
   <groupId>org.eclipse.tycho</groupId>
   <artifactId>target-platform-configuration</artifactId>
   <version>${tycho-version}</version>
   <configuration>
      <dependency-resolution>
         <extraRequirements>
            <requirement>
               <type>eclipse-feature</type>
               <id>example.project.feature</id>
               <versionRange>0.0.0</versionRange>
            </requirement>
         </extraRequirements>
      </dependency-resolution>
   </configuration>
</plugin>
The dependencies specified as extraRequirements are - together with the dependencies specified in the MANIFEST.MF of the project - transitively resolved against the target platform. The resulting set of bundles is included in the test runtime.
environmentVariables Map - Additional environments to set for the forked test JVM.
excludes List - List of patterns (separated by commas) used to specify the tests that should be excluded in testing. When not specified and when the test parameter is not specified, the default excludes will be **/*$* (which excludes all inner classes).
explodedBundles String[] - List of bundles that must be expanded in order to execute the tests. Ignored if testRuntime is p2Installed.
failIfNoTests boolean - If set to "false" the test execution will not fail in case there are no tests found.
Default value is: true.
User property is: failIfNoTests.
forkedProcessTimeoutInSeconds int - Kill the forked test process after a certain number of seconds. If set to 0, wait forever for the process, never timing out.
User property is: surefire.timeout.
frameworkExtensions Dependency[] - List of framework extension bundles to add. Note: The goal does not automatically detect which bundles in the test runtime are framework extensions, but they have to be explicitly specified using this parameter. Ignored if testRuntime is p2Installed.
includes List - List of patterns (separated by commas) used to specify the tests that should be included in testing. When not specified and whent the test parameter is not specified, the default includes will be **/Test*.java **/*Test.java **/*TestCase.java
osgiDataDirectory File - OSGi data directory (osgi.instance.area, aka the workspace) of the Equinox runtime used to execute tests.
Default value is: ${project.build.directory}/work/data/.
parallel ParallelMode 0.16.0 (JUnit 4.7 provider) Supports values "classes"/"methods"/"both" to run in separate threads, as controlled by threadCount.
User property is: parallel.
perCoreThreadCount boolean 0.16.0 (JUnit 4.7 provider) Indicates that threadCount is per cpu core.
Default value is: true.
User property is: perCoreThreadCount.
pluginArtifacts List - (no description)
User property is: plugin.artifacts.
product String - Eclipse product to be run, i.e. -product parameter passed to test Eclipse runtime.
profileName String 0.19.0 p2 profile name of the installation under test. Only relevant if testRuntime is p2Installed. If tests are installed on top of an already existing installation in work, this must match the name of the existing profile.
Default value is: DefaultProfile.
providerHint String 0.16.0 Normally tycho will automatically determine the test framework provider based on the test project's classpath. Use this to force using a test framework provider implementation with the given role hint. Tycho comes with providers "junit3","junit4","junit47". Note that when specifying a providerHint, you have to make sure the provider is actually available in the dependencies of tycho-surefire-plugin.
providerProperties Properties 0.16.0 Use this to specify surefire provider-specific properties.
redirectTestOutputToFile boolean - Set this to "true" to redirect the unit test standard output to a file (found in reportsDirectory/testName-output.txt).
Default value is: false.
User property is: maven.test.redirectTestOutputToFile.
reportsDirectory File - Base directory where all reports are written to.
Default value is: ${project.build.directory}/surefire-reports.
runOrder String 0.19.0 Defines the order the tests will be run in. Supported values are "alphabetical", "reversealphabetical", "random", "hourly" (alphabetical on even hours, reverse alphabetical on odd hours) and "filesystem".
Default value is: filesystem.
showEclipseLog boolean - Enables -debug -consolelog for the test OSGi runtime
Default value is: false.
User property is: tycho.showEclipseLog.
skip Boolean - Same as skipTests
User property is: maven.test.skip.
skipExec boolean - Deprecated. Use skipTests instead.
User property is: maven.test.skipExec.
skipTests Boolean - Set this to "true" to skip running tests, but still compile them. Its use is NOT RECOMMENDED, but quite convenient on occasion. Default: false
User property is: skipTests.
surefireProperties File - (no description)
Default value is: ${project.build.directory}/surefire.properties.
systemProperties Map - Additional system properties to set for the forked test JVM.
test String - Specify this parameter if you want to use the test pattern matching notation, Ant pattern matching, to select tests to run. The Ant pattern will be used to create an include pattern formatted like **/${test}.java When used, the includes and excludes patterns parameters are ignored
User property is: test.
testClass String - See testSuite
User property is: testClass.
testClassesDirectory File - The directory containing generated test classes of the project being tested.
User property is: project.build.outputDirectory.
testFailureIgnore boolean - Set this to true to ignore a failure during testing. Its use is NOT RECOMMENDED, but quite convenient on occasion.
Default value is: false.
User property is: maven.test.failure.ignore.
testRuntime String 0.19.0 How to create the OSGi test runtime. Allowed values are default and p2Installed. Mode p2Installed is EXPERIMENTAL - only works when installing products under test (see below).
  • In default mode, all necessary files to define the test runtime like config.ini are generated by tycho. This installation mode has the advantage that the test runtime is minimal (defined by the transitive dependencies of the test bundle plus and the test harness) and existing bundle jars are referenced rather than copied for the installation
  • In p2Installed mode, use p2 director to install test bundle, test harness bundles and respective dependencies. This installation mode can be used for integration tests that require a fully p2-provisioned installation. To install a product IU, add it as extra requirement to the test bundle (see example below). Note that this installation mode comes with a certain performance overhead for executing the provisioning operations otherwise not required.
Example configuration which will install product IU under test "example.product.id" using p2:
<plugin>
   <groupId>org.eclipse.tycho</groupId>
   <artifactId>tycho-surefire-plugin</artifactId>
   <version>${tycho-version}</version>
   <configuration>
      <testRuntime>p2Installed</testRuntime>
   </configuration>
</plugin>
<plugin>
   <groupId>org.eclipse.tycho</groupId>
   <artifactId>target-platform-configuration</artifactId>
   <version>${tycho-version}</version>
   <configuration>
      <dependency-resolution>
         <extraRequirements>
            <!-- product IU under test -->
            <requirement>
               <type>p2-installable-unit</type>
               <id>example.product.id</id>
               <versionRange>0.0.0</versionRange>
            </requirement>
         </extraRequirements>
      </dependency-resolution>
   </configuration>
</plugin>

Default value is: default.
testSuite String - Bundle-SymbolicName of the test suite, a special bundle that knows how to locate and execute all relevant tests. testSuite and testClass identify single test class to run. All other tests will be ignored if both testSuite and testClass are provided. It is an error if provide one of the two parameters but not the other.
User property is: testSuite.
threadCount int 0.16.0 (JUnit 4.7 provider) The attribute thread-count allows you to specify how many threads should be allocated for this execution. Only makes sense to use in conjunction with the parallel parameter.
User property is: threadCount.
useJDK ToolchainProvider$JDKUsage - Which JDK to use for executing tests. Possible values are: SYSTEM, BREE .
  • SYSTEM: Use the currently running JVM (or from toolchain if configured in pom.xml)
  • BREE: use MANIFEST header Bundle-RequiredExecutionEnvironment to lookup the JDK from toolchains.xml. The value of BREE will be matched against the id of the toolchain elements in toolchains.xml.
Example for BREE:
In META-INF/MANIFEST.MF:
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
In toolchains.xml:
<toolchains>
   <toolchain>
      <type>jdk</type>
      <provides>
         <id>JavaSE-1.7</id>
      </provides>
      <configuration>
         <jdkHome>/path/to/jdk/1.7</jdkHome>
      </configuration>
   </toolchain>
</toolchains>

Default value is: SYSTEM.
useUIHarness boolean - Run tests using UI (true) or headless (false) test harness.
Default value is: false.
useUIThread boolean - Run tests in UI (true) or background (false) thread. Only applies to UI test harness.
Default value is: true.
useUnlimitedThreads boolean 0.16.0 (JUnit 4.7 provider) Indicates that the thread pool will be unlimited. The parallel parameter and the actual number of classes/methods will decide. Setting this to "true" effectively disables perCoreThreadCount and threadCount.
Default value is: false.
User property is: useUnlimitedThreads.
work File - Root directory (osgi.install.area) of the Equinox runtime used to execute tests.
Default value is: ${project.build.directory}/work.

Parameter Details

appArgLine:

Arbitrary applications arguments to set on the command line.
  • Type: java.lang.String
  • Required: No

application:

Eclipse application to be run. If not specified, default application org.eclipse.ui.ide.workbench will be used. Application runnable will be invoked from test harness, not directly from Eclipse. Note that you need to ensure that the bundle which defines the configured application is included in the test runtime.
  • Type: java.lang.String
  • Required: No

argLine:

Arbitrary JVM options to set on the command line.
  • Type: java.lang.String
  • Required: No
  • User Property: tycho.testArgLine

bundleStartLevel:

Bundle start level and auto start configuration used by the test runtime. Ignored if testRuntime is p2Installed.
  • Type: org.eclipse.sisu.equinox.launching.BundleStartLevel[]
  • Required: No

debugPort:

Set this parameter to suspend the test JVM waiting for a client to open a remote debug session on the specified port.
  • Type: int
  • Required: No
  • User Property: debugPort

deleteOsgiDataDirectory:

Whether to recursively delete the directory osgiDataDirectory before running the tests.
  • Type: boolean
  • Required: No
  • Default: true

dependencies:

Additional dependencies to be added to the test runtime. Ignored if testRuntime is p2Installed. Note: This parameter has only limited support for dependencies to artifacts within the reactor. Therefore it is recommended to specify extraRequirements on the target-platform-configuration plugin instead. Example:
<plugin>
   <groupId>org.eclipse.tycho</groupId>
   <artifactId>target-platform-configuration</artifactId>
   <version>${tycho-version}</version>
   <configuration>
      <dependency-resolution>
         <extraRequirements>
            <requirement>
               <type>eclipse-feature</type>
               <id>example.project.feature</id>
               <versionRange>0.0.0</versionRange>
            </requirement>
         </extraRequirements>
      </dependency-resolution>
   </configuration>
</plugin>
The dependencies specified as extraRequirements are - together with the dependencies specified in the MANIFEST.MF of the project - transitively resolved against the target platform. The resulting set of bundles is included in the test runtime.
  • Type: org.apache.maven.model.Dependency[]
  • Required: No

environmentVariables:

Additional environments to set for the forked test JVM.
  • Type: java.util.Map
  • Required: No

excludes:

List of patterns (separated by commas) used to specify the tests that should be excluded in testing. When not specified and when the test parameter is not specified, the default excludes will be **/*$* (which excludes all inner classes).
  • Type: java.util.List
  • Required: No

explodedBundles:

List of bundles that must be expanded in order to execute the tests. Ignored if testRuntime is p2Installed.
  • Type: java.lang.String[]
  • Required: No

failIfNoTests:

If set to "false" the test execution will not fail in case there are no tests found.
  • Type: boolean
  • Required: No
  • User Property: failIfNoTests
  • Default: true

forkedProcessTimeoutInSeconds:

Kill the forked test process after a certain number of seconds. If set to 0, wait forever for the process, never timing out.
  • Type: int
  • Required: No
  • User Property: surefire.timeout

frameworkExtensions:

List of framework extension bundles to add. Note: The goal does not automatically detect which bundles in the test runtime are framework extensions, but they have to be explicitly specified using this parameter. Ignored if testRuntime is p2Installed.
  • Type: org.apache.maven.model.Dependency[]
  • Required: No

includes:

List of patterns (separated by commas) used to specify the tests that should be included in testing. When not specified and whent the test parameter is not specified, the default includes will be **/Test*.java **/*Test.java **/*TestCase.java
  • Type: java.util.List
  • Required: No

osgiDataDirectory:

OSGi data directory (osgi.instance.area, aka the workspace) of the Equinox runtime used to execute tests.
  • Type: java.io.File
  • Required: No
  • Default: ${project.build.directory}/work/data/

parallel:

(JUnit 4.7 provider) Supports values "classes"/"methods"/"both" to run in separate threads, as controlled by threadCount.
  • Type: org.eclipse.tycho.surefire.ParallelMode
  • Since: 0.16.0
  • Required: No
  • User Property: parallel

perCoreThreadCount:

(JUnit 4.7 provider) Indicates that threadCount is per cpu core.
  • Type: boolean
  • Since: 0.16.0
  • Required: No
  • User Property: perCoreThreadCount
  • Default: true

pluginArtifacts:

(no description)
  • Type: java.util.List
  • Required: No
  • User Property: plugin.artifacts

product:

Eclipse product to be run, i.e. -product parameter passed to test Eclipse runtime.
  • Type: java.lang.String
  • Required: No

profileName:

p2 profile name of the installation under test. Only relevant if testRuntime is p2Installed. If tests are installed on top of an already existing installation in work, this must match the name of the existing profile.
  • Type: java.lang.String
  • Since: 0.19.0
  • Required: No
  • Default: DefaultProfile

providerHint:

Normally tycho will automatically determine the test framework provider based on the test project's classpath. Use this to force using a test framework provider implementation with the given role hint. Tycho comes with providers "junit3","junit4","junit47". Note that when specifying a providerHint, you have to make sure the provider is actually available in the dependencies of tycho-surefire-plugin.
  • Type: java.lang.String
  • Since: 0.16.0
  • Required: No

providerProperties:

Use this to specify surefire provider-specific properties.
  • Type: java.util.Properties
  • Since: 0.16.0
  • Required: No

redirectTestOutputToFile:

Set this to "true" to redirect the unit test standard output to a file (found in reportsDirectory/testName-output.txt).
  • Type: boolean
  • Required: No
  • User Property: maven.test.redirectTestOutputToFile
  • Default: false

reportsDirectory:

Base directory where all reports are written to.
  • Type: java.io.File
  • Required: No
  • Default: ${project.build.directory}/surefire-reports

runOrder:

Defines the order the tests will be run in. Supported values are "alphabetical", "reversealphabetical", "random", "hourly" (alphabetical on even hours, reverse alphabetical on odd hours) and "filesystem".
  • Type: java.lang.String
  • Since: 0.19.0
  • Required: No
  • Default: filesystem

showEclipseLog:

Enables -debug -consolelog for the test OSGi runtime
  • Type: boolean
  • Required: No
  • User Property: tycho.showEclipseLog
  • Default: false

skip:

Same as skipTests
  • Type: java.lang.Boolean
  • Required: No
  • User Property: maven.test.skip

skipExec:

Deprecated. Use skipTests instead.
(no description)
  • Type: boolean
  • Required: No
  • User Property: maven.test.skipExec

skipTests:

Set this to "true" to skip running tests, but still compile them. Its use is NOT RECOMMENDED, but quite convenient on occasion. Default: false
  • Type: java.lang.Boolean
  • Required: No
  • User Property: skipTests

surefireProperties:

(no description)
  • Type: java.io.File
  • Required: No
  • Default: ${project.build.directory}/surefire.properties

systemProperties:

Additional system properties to set for the forked test JVM.
  • Type: java.util.Map
  • Required: No

test:

Specify this parameter if you want to use the test pattern matching notation, Ant pattern matching, to select tests to run. The Ant pattern will be used to create an include pattern formatted like **/${test}.java When used, the includes and excludes patterns parameters are ignored
  • Type: java.lang.String
  • Required: No
  • User Property: test

testClass:

See testSuite
  • Type: java.lang.String
  • Required: No
  • User Property: testClass

testClassesDirectory:

The directory containing generated test classes of the project being tested.
  • Type: java.io.File
  • Required: No
  • User Property: project.build.outputDirectory

testFailureIgnore:

Set this to true to ignore a failure during testing. Its use is NOT RECOMMENDED, but quite convenient on occasion.
  • Type: boolean
  • Required: No
  • User Property: maven.test.failure.ignore
  • Default: false

testRuntime:

How to create the OSGi test runtime. Allowed values are default and p2Installed. Mode p2Installed is EXPERIMENTAL - only works when installing products under test (see below).
  • In default mode, all necessary files to define the test runtime like config.ini are generated by tycho. This installation mode has the advantage that the test runtime is minimal (defined by the transitive dependencies of the test bundle plus and the test harness) and existing bundle jars are referenced rather than copied for the installation
  • In p2Installed mode, use p2 director to install test bundle, test harness bundles and respective dependencies. This installation mode can be used for integration tests that require a fully p2-provisioned installation. To install a product IU, add it as extra requirement to the test bundle (see example below). Note that this installation mode comes with a certain performance overhead for executing the provisioning operations otherwise not required.
Example configuration which will install product IU under test "example.product.id" using p2:
<plugin>
   <groupId>org.eclipse.tycho</groupId>
   <artifactId>tycho-surefire-plugin</artifactId>
   <version>${tycho-version}</version>
   <configuration>
      <testRuntime>p2Installed</testRuntime>
   </configuration>
</plugin>
<plugin>
   <groupId>org.eclipse.tycho</groupId>
   <artifactId>target-platform-configuration</artifactId>
   <version>${tycho-version}</version>
   <configuration>
      <dependency-resolution>
         <extraRequirements>
            <!-- product IU under test -->
            <requirement>
               <type>p2-installable-unit</type>
               <id>example.product.id</id>
               <versionRange>0.0.0</versionRange>
            </requirement>
         </extraRequirements>
      </dependency-resolution>
   </configuration>
</plugin>
  • Type: java.lang.String
  • Since: 0.19.0
  • Required: No
  • Default: default

testSuite:

Bundle-SymbolicName of the test suite, a special bundle that knows how to locate and execute all relevant tests. testSuite and testClass identify single test class to run. All other tests will be ignored if both testSuite and testClass are provided. It is an error if provide one of the two parameters but not the other.
  • Type: java.lang.String
  • Required: No
  • User Property: testSuite

threadCount:

(JUnit 4.7 provider) The attribute thread-count allows you to specify how many threads should be allocated for this execution. Only makes sense to use in conjunction with the parallel parameter.
  • Type: int
  • Since: 0.16.0
  • Required: No
  • User Property: threadCount

useJDK:

Which JDK to use for executing tests. Possible values are: SYSTEM, BREE .
  • SYSTEM: Use the currently running JVM (or from toolchain if configured in pom.xml)
  • BREE: use MANIFEST header Bundle-RequiredExecutionEnvironment to lookup the JDK from toolchains.xml. The value of BREE will be matched against the id of the toolchain elements in toolchains.xml.
Example for BREE:
In META-INF/MANIFEST.MF:
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
In toolchains.xml:
<toolchains>
   <toolchain>
      <type>jdk</type>
      <provides>
         <id>JavaSE-1.7</id>
      </provides>
      <configuration>
         <jdkHome>/path/to/jdk/1.7</jdkHome>
      </configuration>
   </toolchain>
</toolchains>
  • Type: org.eclipse.tycho.core.maven.ToolchainProvider$JDKUsage
  • Required: No
  • Default: SYSTEM

useUIHarness:

Run tests using UI (true) or headless (false) test harness.
  • Type: boolean
  • Required: No
  • Default: false

useUIThread:

Run tests in UI (true) or background (false) thread. Only applies to UI test harness.
  • Type: boolean
  • Required: No
  • Default: true

useUnlimitedThreads:

(JUnit 4.7 provider) Indicates that the thread pool will be unlimited. The parallel parameter and the actual number of classes/methods will decide. Setting this to "true" effectively disables perCoreThreadCount and threadCount.
  • Type: boolean
  • Since: 0.16.0
  • Required: No
  • User Property: useUnlimitedThreads
  • Default: false

work:

Root directory (osgi.install.area) of the Equinox runtime used to execute tests.
  • Type: java.io.File
  • Required: No
  • Default: ${project.build.directory}/work