Scenario Outline

Sometimes you have several scenarios that follow exactly the same pattern of steps, just with different input values or expected outcomes. For example, suppose we’re testing each of the fixed amount withdrawal buttons on the ATM:

Once again, all the repetition in this feature makes it boring to read. It’s hard to see the essence of each scenario, which is the amount of money involved in each transaction. We can use a scenario outline to specify the steps once and then play multiple sets of values through them. Here’s that scenario again, refactored to use a scenario outline:

We indicate placeholders within the scenario outline using angle brackets (<..>) where we want real values to be substituted. The scenario outline itself is useless without a table of Examples, which lists rows of values to be substituted for each placeholder.

You can have any number of Scenario Outline elements in a feature and any number of Examples tables under each scenario outline. Behind the scenes, Cucumber converts each row in the Examples table into a scenario before executing it. You can prove this to yourself by using the --expand option, which will print each example in a scenario outline as though it were a scenario:

One of the advantages of using a scenario outline is that you can clearly see gaps in your examples. In our example, we haven’t tested any edge cases, such as when you try to withdraw more money than you have available. This becomes much more obvious when you can see all the values lined up together in a table.

Remember that although the syntax for writing them in Gherkin is the same, these tables are totally different from the data tables we described earlier in this chapter. Data tables just describe a lump of data to attach to a single step of a single scenario. In a scenario outline, each row of an Examples table represents a whole scenario to be executed by Cucumber. In fact, you might want to use the keyword Scenarios (note the extra s) in place of Examples if you find that more readable.

Bigger Placeholders

It’s easy to imagine that you can use scenario outline placeholders only where there’s a piece of data in the step. In fact, when Cucumber compiles a scenario outline’s table of examples down into scenarios ready to execute, it doesn’t care where the placeholders are. So, you can substitute as much or as little as you like from any step’s text.

Let’s illustrate this by testing the edge case where we try to withdraw more money than we have. What should we do in this case? Give the user as much as we can, given their remaining balance, or just show them an error message? We ask our stakeholders for clarification, and they’re happy for us to show them an error message. Here’s how we write the scenario first:

There’s still a good deal of duplication here with the flow of the scenarios above it, but because the Then step is so different, we can’t put this one into the scenario outline. Or can we?

Let’s change the scenario outline, replacing the <Received> placeholder with a more abstract <Outcome>:

Now we can simply add our failure case to the bottom of that table:

We can use a placeholder to replace any of the text we like in a step. Notice that it doesn’t matter what order the placeholders appear in the table: what counts is that the column header matches the text in the placeholder in the scenario outline.

Joe asks:

How Many Examples Should I Use?

Once you have a scenario outline with a few examples, it’s very easy to think of more examples, and even easier to add them. Before you know it, you have a huge, very comprehensive table of examples—and a problem.

Why?

On a system of any serious complexity, you can quite quickly start to experience what mathematicians call combinatorial explosion, where the number of different combinations of inputs and expected outputs becomes unmanageable. In trying to cover every possible eventuality, you end up with rows and rows of example data for Cucumber to execute. Remember that each of those little rows represents a whole scenario that might take several seconds to execute, and that can quickly start to add up. When your tests take longer to run, you slow down your feedback loop, making the whole team less productive as a result.

A really long table is also very hard to read. It’s better to aim to make your examples illustrative or representative than exhaustive. Try to stick to what Gojko Adzic calls the key examples.^[17] If you study the code you’re testing, you’ll often find that some rows of your examples table cover the same logic as another row in the table. You might also find that the test cases in your table are already covered by unit tests of the underlying code. If they’re not, consider whether they should be.

Remember that readability is what’s most important. If your stakeholders feel comforted by exhaustive tests, perhaps because your software operates in a safety-critical environment, then by all means put them in. Just remember that you’ll never be able to prove there are no bugs. As logicians say, absence of proof is not proof of absence.

If you’re struggling to pick a representative set from an exhaustive list of examples, take a look at Joseph Wilk’s pairwise^[18] Ruby gem.

Although this is a useful technique, be careful that your programmer’s instinct to reduce duplication at all costs doesn’t take over here.^[19] If you move too much of the text of a step into the examples table, it can be very hard to read the flow of the scenario. Remember your goal is readability, so don’t take this too far, and always test your features by getting other people to regularly read them and give you feedback.

Multiple Tables of Examples

Cucumber will happily handle any number of Examples elements beneath a Scenario Outline, meaning you can group different kinds of examples together, if you want. For example:

As usual, you have the option of a name and description for each Examples table. When you have a large set of examples, splitting it into multiple tables can make it easier for a reader to understand.

Explain Yourself

You’ll normally be using a scenario outline and examples table to help specify the implementation of a business rule. Remember to include a plain-language description of the underlying rule that the examples are supposed to illustrate! It’s amazing how often people forget to do this.

For example, look at this feature:

If you have to implement the code to make this feature pass, can you tell what the underlying rule is?

Not very easily. So, let’s modify the feature to make it more self-explanatory, like this:

By separating the examples into two sets and giving each one a name and description, we’ve explained the rule and given examples of the rule at the same time.