Thursday, February 27, 2020

Parameterized Tests: An Underused Technique

Over the past several years I've come to really value Parameterized Tests. I think they are an underused technique that should be considered more often. They make it easier to cover more of the test space and reduce cognitive load by being succinct.

So what are Parameterized Tests? Well they allow developers to run the same test multiple times over a set of different values. Here is a simple Java example using Java's Stream class with AssertJ:

Stream.of(null, "", " ", "false", "FALSE")
        .forEach(value -> assertThat(Boolean.valueOf(value)).isFalse());

Cucumber Inspiration


I first discovered this technique with Cucumber, which has the ability to run a scenario multiple times over a set of different values using a Scenario Outline. Prior to this I was in the habit of copying and pasting a Scenario and tweaking the Given and Then steps; or I wouldn't test all the sad path combinations in an effort to reduce test maintenance. Learning about the Scenario Outline changed my world. I was able to combine multiple scenarios into a single scenario and the result was more comprehendible and easier to maintain. For example, this is a common Scenario Outline we might use to test an API endpoint to ensure it validates the input. Previously, this would have been spread across multiple scenarios or not tested at all.

Scenario Outline: Should return a 400 when signing up with an invalid email address
  When I attempt to sign up with email "<email>"
  Then I should be returned a "400 Bad Request" status code

  Examples:
    | Email                       |
    | ${absent}                   |
    | ${blank}                    |
    | ${whitespace}               |
    | mailto:john.doe@example.com |
    | john.doe@example            |
    | john.doe@example.           |
    | john.doe@example.com.       |
    | john.doe@@example.com       |
    | john.doe@example..com       |
    | john doe@example.com        |

Recognizing Pattern


Once I discovered this concept and got comfortable with it, I explored ways to introduce it in lower level tests like unit and integration tests. But the trick was recognizing when to apply it. Basically, any time you find yourself copying and pasting a test and tweaking the arrange and assert statements you've probably got a good candidate for a Parameterized Test.

For example, let's say you have a class that determines if a number is a prime number or not. Before Parameterized Tests you might have been tempted to write something like the following:

public class PrimeNumberCheckerTestOldSchool {
    @Test
    public void shouldReturnTrueForPrimeNumber() {
        assertThat(PrimeNumberChecker.check(2)).isTrue();
    }

    @Test
    public void shouldReturnFalseForNonPrimeNumber() {
        assertThat(PrimeNumberChecker.check(6)).isFalse();
    }

    @Test
    public void shouldReturnFalseForAnotherNonPrimeNumber() {
        assertThat(PrimeNumberChecker.check(9)).isFalse();
    }

    @Test
    public void shouldReturnTrueForAnotherPrimeNumber() {
        assertThat(PrimeNumberChecker.check(17)).isTrue();
    }
}

Here you can see we are just repeating the test with different inputs and a different result. And since it's spread out across multiple tests it's hard to comprehend and this solution doesn't scale well if we want to add additional tests.

The ideal solution would be to define a sort of truth table, like Cucumber's Example table, that includes a combination of the inputs and expected result in a single test. This is where Parameterized Tests comes in.


JUnit 5


My preferred way to write Parameterized Tests is with JUnit 5 (see Parameterized Tests with JUnit 5). It's a big improvement over the JUnit 4 way of doing Parameterized Tests. We can rewrite the earlier example using the @ParameterizedTest and @CsvSource annotations (see other source annotations):

public class PrimeNumberCheckerTestJUnit5 {
    @ParameterizedTest
    @CsvSource({
            "2,  true",
            "6,  false",
            "9,  false",
            "17, true"
    })
    public void shouldCheckPrimality(final int number, final boolean expected) {
        assertThat(PrimeNumberChecker.check(number)).isEqualTo(expected);
    }
}

Here are some of the benefits of this approach:
  • Easier to comprehend since it's not spread out across 4 test methods.
  • Easier to add/remove additional scenarios.
  • Easier to delete if the Method Under Test (MUT) is removed; all we will have to do is delete one test method!

To write Parameterized Tests with JUnit 5 you need to include the following dependencies. Note the JUnit 5 documentation states this is an "experimental" feature.

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.5.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.5.2</version>
    <scope>test</scope>
</dependency>


JUnit 4


A quick note about JUnit 4. If your team/project is still using JUnit 4, you can use both JUnit 4 and JUnit 5 simultaneously. JUnit 5 provides a gentle migration path. Just include the following dependency. I also recommend reading the Migration Tips.

<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.5.2</version>
    <scope>test</scope>
</dependency>

Now you are safe to write JUnit 5 tests without having to migrate all your existing JUnit 4 tests and be able to take advantage of the new @ParameterizedTest annotation.


Conclusion


Well hopefully I've convinced you of the power of Parameterized Tests and you'll look for the right opportunity to try them out. And keep in mind while these examples use Java, this technique should also be applicable to other languages. So do some research and see if your language provides them and if not maybe create your own. That's what we did before JUnit 5 was out and we weren't happy with the JUnit 4 way of doing Parameterized Tests.

16 comments:

Anonymous said...

That's why we choose https://testng.org/ for our tests years ago ;-)

Graham said...

I love using Spock for unit tests, it has a very nice data table feature, just like cucumber: http://spockframework.org/spock/docs/1.3/data_driven_testing.html

360DigiTMGNoida said...

The information you have posted is important. The objections you have insinuated was worthy. Thankful for sharing.
iot training in noida

360DigiTMG said...

Extraordinary post. I simply found your blog and expected to express that I have genuinely refreshing scrutinizing your blog sections.Huge thankful for the significant data.
what is hrdf

360digitmg said...

Thankyou for this wondrous post, I am cheerful I watched this site on yahoo.

360digitmg

hamda said...

Many people assume that the seo of a website starts when your website is developed and has content in it. But I would say, you start thinking about SEO right from the point of Purchasing a domain name.

refrigerator service in hyderabad said...

Nice Post!! The way you written is very good. it gives very valuable information. Thank You So much .
Best Refrigerator Repair Service in Hyderabad

Jessica Lucas said...

The new report by Expert Market Research titled, ‘Global Smart Display Market Report and Forecast 2022-2027’, gives an in-depth analysis of the global Smart Display Market, assessing the market based on its segments like type, display size, resolution, growth end use, and major regions. The report tracks the latest trends in the industry and studies their impact on the overall market. Smart display technology has been gaining traction over the last few years. The growing usage of smart displays in residential and public facilities such as retail shops and the increasing number of infrastructural improvements such as smart homes and increasing demand for smart vehicles are contributing to the market growth. The major players in the market are Alpine Electronics, Inc., Japan Display Inc., Leyard Optoelectronic Co., Ltd., Alphabet Inc., and LG Electronics Inc., among others. The report covers the market shares, capacities, plant turnarounds, investments, expansions, and mergers and acquisitions, among other latest developments of these market players.

Besthyiptemplate said...

Thank you for this article because it’s really informative, I love reading your article and I hope that I will read some more about this stuff, it’s really informative and very entertaining. Thanks a lot and have a great day. buy coinpayments verified account

John Hardy said...

I am very thankful for this excellent information. I hope you will share more updates like this. Now it's time to avail HALFCASTE CREAM SET for more information.

John Hardy said...

I think this is an extraordinary article. You make this information interesting and secures. Thanks for sharing this information. Now it's time to avail Locksmith Leeds for more information.

David said...

Great information in this article. Its very useful and helpful for developer. Thanks for sharing all examples with us. I appreciate you technique and writing skills. Now it's time to get interior demolition for more information.

justin said...

Thanks for writing this quality information in this article. I appreciate your all explanation to really helpful for all developers. Keep sharing more article with us. Now it's time to get warwick taxi for more information.

Anonymous said...

Hi.
I stumbled upon your blog recently, and I must say it's truly extraordinary. Your blog posts are highly enjoyable, and I genuinely appreciate the valuable information you share. I am sincerely grateful for the knowledge I've gained from reading your content. Keep up the fantastic work!Here is sharing some Data Modeling Training information may be its helpful to you.
Data Modeling Training

Ruby Rose said...

Thanks for writing this informative article.I appreciate your great effort and explanation. It is really helpful for the developers.Keep sharing more artcles with us. Now its time to get
medical aesthetics canada for more information.

Anonymous said...

I genuinely appreciate your dedication and clear explanations, which are immensely beneficial for developers. Please continue sharing more articles with us in the future.You can visit yoga classes in mirdif for more information.