Recording test failures under Gradle Test *¶
See Recording test failures for an overview of LR4J test integration.
The following instructions apply to Gradle.
1. Gradle: record all tests and generate one recording file per failing test class¶
This is the least intrusive method of integration since it doesn’t require build changes. Add the following rule to each of your test classes:
@Rule
public final TestRule watchman =
new TestWatcher() {
protected void failed(Throwable thr, Description description) {
// Tell LiveRecorder to name the recording file after this
// failing method & class
System.setProperty("io.undo.output", description.toString());
// Tell LiveRecorder that the test failed
System.setProperty("io.undo.failed", "true");
}
};
Add the following to the the test section of your build.gradle file:
// pass to test JVM to enable recording
jvmArgs '-XX:-Inline', '-XX:TieredStopAtLevel=1', '-XX:UseAVX=2', '-agentpath:/path/to/lr4j-record-1.0.so=save_on=failure'
// stop at first failure and record just it (set to false to record all failures)
failFast = true
And ensure that the maxParallelForks parameter is either unset or
set to 1.
The failFast parameter when set to true causes the test run to stop at
the first method in a class that fails so that each recording file contains
exactly one test failure, and that failure is located at the end of the
recording (though the recording file may contain some earlier test successes
too). When failFast is set to false, the recording may contain multiple
test successes and failures and will be named after the last failing method.
2. Gradle: record all tests and generate one recording file per failing test method¶
Add the following rule to each of your test classes:
@Rule
public final TestRule wrapper =
new TestRule() {
@Override
public Statement apply(final Statement base, final Description description) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
UndoLR.setEventLogSize(1000 * 1000 * 1000);
UndoLR.start();
try {
base.evaluate();
} catch (Throwable e) {
UndoLR.save(description + ".undo");
} finally {
UndoLR.stop();
}
}
};
}
};
Create a libs folder under your project root and add the LiveRecorder API to it.
mkdir {project_root}/libs
cp /path/to/lr4j/lr4j_api-1.1.jar {project_root}/libs
Include the LiveRecorder API as a build and runtime dependency in your build.gradle file:
repositories {
...
flatDir {
dir 'libs'
}
}
dependencies {
...
testImplementation 'io.undo.jdwp:lr4j_api:1.1'
}
Add the following to the test section of your build.gradle file:
// pass to test JVM to enable recording
jvmArgs '-XX:-Inline', '-XX:TieredStopAtLevel=1', '-XX:UseAVX=2', '-agentpath:/path/to/lr4j-record-1.0.so=save_on=failure'
// stop at first failure and record just it (set to false to record all failures)
failFast = true
And ensure that the maxParallelForks parameter is either unset or set to 1.
3. Gradle: run all tests, and re-run failing test methods under recording¶
The following instructions are for JUnit 4. For JUnit 5 see the rerunner-jupiter extension.
Note
Gradle does not permit re-use of test JVMs so we are obliged to work around this by using the file-system to track progress between test runs. You need to make sure “testProgressFile” is removed after the run.
Add the following rule to each of your test classes:
static Path testProgressFile = Paths.get(System.getProperty("testProgressFile"));
static Set<String> hadFailure = new HashSet<String>();
static {
try (Stream<String> stream = Files.lines(testProgressFile);
stream.forEach(s -> hadFailure.add(s));
} catch (IOException e) {
e.printStackTrace();
}
}
@Rule
public final TestRule wrapper =
new TestRule() {
@Override
public Statement apply(final Statement base, final Description description) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
if (hadFailure.contains(description.toString())) {
// had a failure last time so record
UndoLR.setEventLogSize(1000 * 1000 * 1000);
UndoLR.start();
try {
base.evaluate();
} catch (Throwable e) {
UndoLR.save(description + ".undo");
throw e;
} finally {
UndoLR.stop();
}
} else {
try {
base.evaluate();
} catch (Throwable e) {
// record next time
Files.write(testProgressFile, Arrays.asList(description.toString()), StandardOpenOption.CREATE, StandardOpenOption.APPEND);
throw e;
}
}
}
};
}
};
Create a libs folder under your project root and add the LiveRecorder API to it.
mkdir {project_root}/libs
cp /path/to/lr4j/lr4j_api-1.1.jar {project_root}/libs
Include the LiveRecorder API as a build and runtime dependency in your build.gradle file:
repositories {
...
flatDir {
dir 'libs'
}
}
dependencies {
...
testImplementation 'io.undo.jdwp:lr4j_api:1.1'
}
Add the following to the test section of your build.gradle file:
// pass to test JVM to enable recording
jvmArgs '-XX:-Inline', '-XX:TieredStopAtLevel=1', '-XX:UseAVX=2', '-agentpath:/path/to/lr4j-record-1.0.so=save_on=failure'
// continue after failures and record record all failed test methods
failFast = false
retry {
failOnPassedAfterRetry = true
maxFailures = 3
maxRetries = 3
}
systemProperty "buildTestProgressFile", "${buildDir}/test.progress"
afterTest { desc, result ->
File f = new File(buildDir, "test.progress")
if(!f.exists())
f.createNewFile()
if(result.getFailedTestCount() > 0)
f.append "\n" + desc.getName()
}
And ensure that the maxParallelForks parameter is either unset or set to 1.
- *
Gradle and the Gradlephant logo are registered trademark of Gradle, Inc. and its subsidiaries in the United States and/or other countries. No endorsement by Gadle Inc is implied by the use of these marks.
