This task runs tests from the JUnit testing framework in versions 4.10 and above. It to some extent
mimics the style of the default ANT's junit task, but adds the following features:
Note: The latest version of the JUnit framework can be found at
http://www.junit.org.
This task has been tested with JUnit 4.10 and later.
Appropriate junit*.jar must be available in the task's classpath and in the
tested code's classpath. See the examples below.
External references: More information about randomized testing.
junit4 attempts to be a drop-in replacement for ANT's default junit task, but so attributes
that make little sense have been deprecated and are ignored. Attributes specific to junit4 are highlighted in
light green.
| Attribute | Description | Required |
| printsummary | Prints the summary of all executed, ignored and failed tests after all suites have been executed. To get per-suite summaries, configure a console report properly (see below). | No; default is on. |
| fork | Not used. ANT-junit drop-in compatibility only. Tests are always forked. | -- |
| forkmode | Not used. ANT-junit drop-in compatibility only. Tests are always forked into one more more forked JVMs. Each suite is executed entirely on a single JVM. | -- |
| haltonerror | Not used. ANT-junit drop-in compatibility only. Use haltonfailure which handles both errors and failures. |
-- |
| errorproperty | Not used. ANT-junit drop-in compatibility only. Use failureproperty which handles both errors and failures. |
-- |
| haltonfailure | Stop the build process if a test fails (errors are considered failures as well). | No; default is on (ANT's junit has the default set to 'off'). |
| failureproperty | The name of a property to set in the event of a failure (errors are considered failures as well). | No. |
| filtertrace | Not used. ANT-junit drop-in compatibility only. Reports may provide their own filtering capabilities. | -- |
| timeout | Not used. ANT-junit drop-in compatibility only. Test runners provide their own timeout facilities. | -- |
| maxmemory | Maximum amount of memory to allocate to each forked VM (be careful if using lots of forks).
Note:
If you get java.lang.OutOfMemoryError: Java heap space
in some of your tests then you need to raise the size like
maxmemory="128m" |
No |
| jvm | The command used to invoke the forked java virtual machines, the default is 'java'. The command is resolved by
java.lang.Runtime.exec(). |
No; default is java. |
| dir | The directory in which to invoke the forked JVMs in. This is by default the project's basedir.
Because multiple JVMs may clash if they are generating files in the current working directory, one can use per-JVM directory
by setting isolateWorkingDirectories attribute to true; in such case each JVM gets a sub-directory
under whatever dir is defined.
|
No |
| isolateWorkingDirectories | If true current working directories are isolated for each forked JVM. See dir attribute too. |
No; default is true. |
| newenvironment | Do not propagate the old environment when new environment variables are specified. | No; default is false. |
| includeantruntime | Not used. ANT-junit drop-in compatibility only. | -- |
| showoutput | Not used. ANT-junit drop-in compatibility only. Configure console reports appropriately. | -- |
| outputtoformatters | Not used. ANT-junit drop-in compatibility only. Configure console reports appropriately. | -- |
| tempdir | Specify where to store temporary files. These temporary files include a list of suites passed
to each slave JVM so it is usually wise to just leave this attribute unset - the default is to take the value of the
dir attribute or the project's basedir.
|
No |
| reloading | Not used. ANT-junit drop-in compatibility only. | -- |
| clonevm | Not used. ANT-junit drop-in compatibility only. | -- |
| logfailedtests | Not used. ANT-junit drop-in compatibility only. Configure console reports appropriately to get just the failing tests, their output etc. | -- |
| enableTestListenerEvents | Not used. ANT-junit drop-in compatibility only. | -- |
| parallelism |
The number of parallel slaves. Can be set to a constant max for the
maximum number of cores returned from Runtime.availableProcessors or
auto for sensible defaults depending on the number of cores.
Note: this setting forks physical JVM processes so it multiplies the requirements for heap memory, IO, etc. |
No; Can be set to any integer or: 'max', 'auto'. The default is '1' (sequential execution in a forked JVM). |
| dynamicAssignmentRatio |
Specifies the ratio of suites moved to dynamic assignment list (job-stealing). A dynamic
assignment list dispatches suites to the first idle slave JVM. Theoretically
this is an optimal strategy, but it is usually better to have some static assignments
to avoid communication costs.
A ratio of 0 means only static assignments are used. A ratio of 1 means only dynamic assignments are used. The list of dynamic assignments is sorted by decreasing cost (always) and
is inherently prone to race conditions in distributing suites. Should there
be an error based on suite-dependency it will not be directly repeatable. In such
case use the per-slave-jvm list of suites file dumped to disk for each slave JVM.
See |
No; default is '0.25' (25% of all suites are assigned dynamically). |
| seed |
Specify random seed for anything that is randomized in junit4. The order of suites
execution and suite-JVM assignments are a result of this seed for example.
The master seed is also passed to slave JVMs as a system property (to bootstrap randomized runner). |
No; default is a randomly generated seed. |
| prefix | Initializes custom prefix for all randomized runner properties. This must be consistent across all junit4 invocations if done from the same classpath. Use only when REALLY needed. | No; default is randomized runner's prefix. |
| shuffleOnSlave | Predictably shuffle tests order after balancing. This will help in spreading lighter and heavier tests over a single slave's execution timeline while still keeping the same tests order depending on the seed. See nested elements for configuring load balancers. | No; the default is 'true'. |
| leaveTemporary |
Leave temporary junit4 files after the task quits. This can be used to trace the exact order
of suites executed on each forked JVM for example.
|
No; default is 'false'. |
| jvmOutputAction |
What should be done on unexpected JVM output? JVM may write directly to the
original descriptors, bypassing redirections of System.out and System.err. Typically,
these messages will be important and should fail the build (permgen space exceeded,
compiler errors, crash dumps). However, certain legitimate logs (gc activity, class loading
logs) are also printed to these streams so sometimes the output can be ignored.
Allowed values (any comma-delimited combination of): ignore, pipe, fail, warn. |
No; default is 'pipe,warn'. |
| sysouts |
If set to true, any sysout and syserr calls will be written to original
output and error streams (and in effect will appear as "jvm output". By default
sysout and syserrs are captured and proxied to the event stream to be synchronized
with other test events (and properly handled by reports) but occasionally
one may want to synchronize them with direct JVM output (to synchronize
with compiler output or GC output for example).
See examples below on examples of capturing full JVM output. |
No; default is 'false'. |
| heartbeat | A duration (in seconds) before an event is dispatched about forked JVM inactivity. This can be useful to report hung or very long tests. Heartbeat information will include an approximate location inside the non-responding JVM (suite or test case). | No; default is '0'. |
| uniqueSuiteNames | Allow or disallow duplicate suite names in resource collections. By default this option is true because certain ANT-compatible report types (like XML reports) will have a problem with duplicate suite names (will overwrite files). | No; default is 'true'. |
| ifNoTests | Defines the behavior when no tests were executed successfully or failed (either there were no tests at all or all of them were filtered out). One of the following enum constants: 'ignore', 'warn' or 'fail'. | No; default is 'ignore'. |
The <junit4> task supports a nested elements. Some of them are compatible with
ANT's default <junit> task while others are unique to <junit4>.
<classpath> element represents a PATH like structure to be used for each
forked JVM. Note that classpath should include suiteable junit*.jar, otherwise your tests
wont run.
<junit> task, <junit4> defines suite classes as
resources to .class files. These resources can originate from any resource collection and
there can be any number of resource collections with such resources. For example this definition
includes all suite classes from a ZIP file (JAR file), excluding nested classes:
Additional parameters may be passed to the new JVM via nested <jvmarg> elements. For example:
would run the JVM with a serial garbage collector on HotSpot JVM.
Use nested <sysproperty> elements to specify system
properties required by the tests. These properties will be made available
to the forked JVMs during their execution. The attributes for this element are the same
as for environment variables.
An extended attribute <ignoreEmpty> is supported which, if defined,
causes the property NOT to be defined in the forked JVM (as opposed to passing an empty property
value).
would run the test in ANT's VM and make the project's basedir property
available to the test (note the current working directory of a forked JVM will not be the
basedir typically).
You can specify a set of properties to be passed to the target JVM as system properties
with syspropertysets.
This target behaves much like the default in ANT but allows a dynamic set of (remapped)
properties to be resolved at runtime. Similar to sysproperty, this
tag also supports ignoreEmpty attribute to work around
bugs
with local properties in ANT.
It is possible to specify environment variables to pass to the
forked VM via nested <env> elements. For a description
of the <env> element's attributes, see the
description in the exec task.
The location of bootstrap class files can be specified using this PATH like structure.
You can control enablement of Java 1.4 assertions with an <assertions> subelement.
There is no notion of "reports" in junit4. Reports are just listeners
that attach to the even stream and can produce something (to the console, file or otherwise). You
can also attach your own listeners if you like (although you'd have to peek at the code of existing
listeners to get the feeling how this is done -- communication is via events not interfaces).
There are a few predefined listeners to choose from. By far the most common will be report-text
which produces console output (or plain text file). A plain text report configuration may look like this:
This listener supports stack trace filtering to reduce verbosity (console output, for example). For production or integration builds, it is recommended to keep the full stacks. For developer runs, the default set of filters (enabled by default) can declutter the output a bit. An example stack trace filter, with default options and two custom filters, is shown below.
The level of verbosity of the output can be fine-tuned to one's needs by disabling or enabling test statuses and suite summaries or each test's output. Experimenting highly recommended.
Another listener is report-ant-xml which can produce XML files suitable
for junitreport task (although
we highly recommend using the built-in JSON report.
Yet another built-in listener is producing a modern JSON output with test data and an accompanying HTML5 file for visualizing it:
An example of a HTML5 report produced by the above can be seen here, for example.
Suites can be scattered across forked JVMs randomly or using a greedy load-balancing algorithm (and followed by job-stealing if needed). For load balancing, previous execution statistics will be needed. These statistics should ideally come from a single machine and multiple executions so that they average the influence of the environment etc.
A dedicated listener is used to collect and update statistics. The configuration below shows both the listener and a balancer configuration. The balancer uses multiple sources for collecting statistics (for example precomputed statistics and previous local runs):
JUnit4 has been designed to run reports and aggregation of test events on the master JVM, not on the forked JVMs. This slightly complicates things when diagnostic JVM messages are used because these messages are printed directly to original stream descriptors by the JVM, bypassing System.out or System.err substitutions.
In those rare cases when the original JVM output streams are needed (possibly mixed with what the tests have printed), junit4 can be configured to leave the original streams on disk or pipe them to ANT's output. An example is shown below:
Should one need to preserve the original output files (in case they are large, for example), the configuration would then look as follows:
This would result in the following message being printed to ANT (note the output file name contains an execution-unique element to avoid clashing with other forked JVMs or builds).
[junit4:junit4] JVM J0: stdout was not empty, see: /home/[...]/junit4-J0-1058309761ea5fafd.sysout [junit4:junit4] JVM J0: stderr was not empty, see: /home/[...]/junit4-J0-1058309761ea5fafd.syserr