Using JUnit Categories to group tests

JUnit allows to organize tests in groups using Categories. A well-organised​ suit of tests is really important, annotating​ our tests with categories could help to differentiate fast from slow tests or stable from unstable/maturing tests.

Defining Categories
We define our categories creating interfaces

public interface CriticalTests {}
public interface SanityTests {}
public interface FastTests {}
public interface SlowTests {}
public interface StableTests {}
public interface UnstableTests {}

And annotating our test class or individual tests with the Category annotation

public class MyFeatureTest {
	@Test @Category(FastTests.class)
	public void testFast() {
		System.out.println("fast");
	}
	@Test @Category(SlowTests.class)
	public void testSlow() {
		System.out.println("slow");
	}
	@Test @Category({SanityTests.class, SlowTests.class})
	public void testSanity() {
		System.out.println("sanity");
	}
}

Notice, that we can annotate a test with multiple categories.

@Test @Category({SanityTests.class, SlowTests.class})

Run Tests by Category
Configuring the POM
We can use categories from maven configuring properly the version of maven-surefire-plugin and selecting the junit47 provider.

			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.17</version>
				<dependencies>
	                <dependency>
	                    <groupId>org.apache.maven.surefire</groupId>
	                    <artifactId>surefire-junit47</artifactId>
	                    <version>2.17</version>
	                </dependency>
	            </dependencies>
			</plugin>

Run tests specifying categories
We can run all the test under a category using the -Dgroups maven attribute. For example run all the fast and stable tests:

mvn test -Dgroups="com.test.groups.FastTests, com.test.groups.StableTests"

That argument could be used in combination with -Dtest to run only a especific test in a test class.

mvn test -Dtest=AppTest.java -Dgroups="com.test.groups.SanityTests"


Run tests with a category profile
The second option is to define maven profiles per category or group of categories. This allows to run maven using a profile

mvn test -PsanityTests


Another advantage of profiles is that allows to combine group selection with file inclusion and exclusion.
Example of profile configurations for multiple categories:

...
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.11</version>
				<dependencies>
	                <dependency>
	                    <groupId>org.apache.maven.surefire</groupId>
	                    <artifactId>surefire-junit47</artifactId>
	                    <version>2.12.2</version>
	                </dependency>
	            </dependencies>
				<configuration>
					<groups>${testcase.groups}</groups>
					<excludes>
                    	<exclude>${exclude.tests}</exclude>
	                </excludes>
    	            <includes>
						<include>${include.tests}</include>
                    </includes>
				</configuration>
			</plugin>
		</plugins>
	</build>
 
	<profiles>
	    <profile>
	        <id>sanityTests</id>
	        <properties>
	            <testcase.groups>com.emc.gs.atlas.ui.categories.CriticalTests</testcase.groups>
				<exclude.tests>**/x/**/*.java</exclude.tests>
	            <include.tests>**/y/**/*.java</include.tests>
	        </properties>
	    </profile>
	    <profile>
	        <id>fastTests</id>
	        <properties>
	            <testcase.groups>com.test.groups.SanityTests</testcase.groups>
	        </properties>
	    </profile>
	    <profile>
	        <id>fastTests</id>
	        <properties>
	            <testcase.groups>com.test.groups.FastTests</testcase.groups>
	        </properties>
	    </profile>
	    <profile>
	        <id>slowTests</id>
	        <properties>
	            <testcase.groups>com.test.groups.SlowTests</testcase.groups>
	        </properties>
	    </profile>
	    <profile>
	        <id>stableTests</id>
	        <properties>
	            <testcase.groups>com.test.groups.StableTests</testcase.groups>
	        </properties>
	    </profile>
	    <profile>
	        <id>unstableTests</id>
	        <properties>
	            <testcase.groups>com.test.groups.UnstableTests</testcase.groups>
	        </properties>
	    </profile>
	</profiles>

Documentation:
Oficial documentation:
https://github.com/junit-team/junit/wiki/Categories
https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
Other links:
http://java.dzone.com/articles/unit-and-integration-tests
http://www.togsblom.com/2010/04/categorizing-junit-tests.html
https://weblogs.java.net/blog/johnsmart/archive/2010/04/25/grouping-tests-using-junit-categories-0
http://technicaltesting.wordpress.com/2012/09/24/using-junit-category-and-maven-profiles-to-get-stable-test-suites/
http://stackoverflow.com/questions/15648557/maven-unit-tests
http://stackoverflow.com/questions/3100924/how-to-run-junit-tests-by-category-in-maven

Advertisements