Combining SuiteBuilder and ClasspathSuite May 13, 2010

In a recent commit to JUnit Kent Beck and David Saff have added an “alpha-ready implementation of SuiteBuilder”. As Kent Beck previously described in a blog post, the idea behind the SuiteBuilder runner is to use annotations on fields instead of annotations on classes.

Limitations of regular test suites

While an annotation can take parameters the arguments must be literals, e.g. constant String values or class literals. For example, the classic Suite runner is configured using the @SuiteClasses annotation that takes an array of class literals, i.e. the test classes to be run:

@RunWith(Suite.class)
@SuiteClasses({
    SomeTest.class,
    YetAnotherTest.class
})
public class AllTests {}

Literals have a severe limitation: they must be know at compile-time! Thus, when using the Suite runner, there was no way of determining the classes to run by any other means such as scanning the current classpath.

ClasspathSuite to the rescue

For this original purpose, Johannes Link created the ClasspathSuite runner. Its basic usage is very simple: just specify it using the @RunWith annotation. In addition, you can also include test classes in JAR files, filter by class names or types, and so on:

@RunWith(ClasspathSuite.class)
@IncludeJars(true)
@ClassnameFilters({".*Test", "!.*AllTests"})
@BaseTypeFilter(MyBaseTest.class)
public class AllTests {}

However, the ClasspathSuite does not support JUnit’s categories as mentioned in an earlier blog post. While it could certainly be extended to support the Category-related annotations @IncludeCategory and @ExcludeCategory, the SuiteBuilder offers a more flexible alternative.

Introducing SuiteBuilder

The SuiteBuilder runner is similar to the Suite runner, but reads the test classes it is supposed to run from a field of the suite class annotated with @Classes. The field can be freely initialized to hold an implementation of the SuiteBuilder.Classes.Value interface which simply wraps a collection of classes. E.g., the first example can be rewritten using the SuiteBuilder:

@RunWith(SuiteBuilder.class)
public class AllTests {
    @Classes
    public Listed classes =
        new Listed(SomeTest.class, YetAnotherTest.class);
}

In addition, you can filter the resulting test runners by annotating a field of type SuiteBuilder.RunnerFilter.Value with @RunnerFilter. For example, the latest commit included a CategoryFilter that filters tests by category:

@RunWith(SuiteBuilder.class)
public class OnlyYes {
    @Classes
    public Listed classes =
        new Listed(SomeTest.class, YetAnotherTest.class);

    @RunnerFilter
    public CategoryFilter filter = CategoryFilter.include(Yes.class);
}

Putting the pieces together

So what? Well, instead of specifying the classes explicitly you could employ the capabilities of the ClasspathSuite to determine the test classes dynamically. For this purpose, I have written a small wrapper around Johannes Links’ ClasspathSuite. The above example can thus be rewritten without explicitly specifying the test classes:

@RunWith(SuiteBuilder.class)
public class OnlyYes {
    @Classes
    public InClasspath classes = new InClasspath();

    @RunnerFilter
    public CategoryFilter filter = CategoryFilter.include(Yes.class);
}

The wrapper offers the same flexibility as the ClasspathSuite, e.g.:

@RunWith(SuiteBuilder.class)
public class OnlyYes {
    @Classes
    public InClasspath classes = new InClasspath().includingJars()
            .filteredBy(".*Test", "!.*AllTests")
            .includingOnlySubclassesOf(MyBaseTest.class);

    @RunnerFilter
    public CategoryFilter filter = CategoryFilter.include(Yes.class);
}

While I will look into how this can be integrated into JUnit or ClasspathSuite feel free to contact me if you are interested in the source code of the InClasspath class.

Update

I am currently working on integrating ClasspathSuite and InClasspath into core JUnit… In the meantime, you can take a look at the code on GitHub.