Writing better unit tests faster: Test matrices

Software development is all about problem-solving. But, unfortunately, one of the many things that I have learned over the past several years is that using the computer to solve business domain problems is very inefficient. More often than not, I use the computer to realize my solutions rather than solve problems related to my business domain. Of course, I sometimes use the computer/IDE to explore a problem space, but that's a discussion for another time.

Solving a problem requires us to do at least two things,

1. Understand the problem at hand.
2. Organize our thoughts.

We can achieve both of these using what I call a test matrix (do you folks call it something else?).

A test matrix is a table with possible inputs in its rows and possible outputs/outcomes in columns.

Examples: Gutter icons in an IntelliJ Plugin

Here are a couple of examples from a feature I recently built for an IntelliJ plugin for ApprovalTests. There were two cases I had to tackle.

1. What icon to show next to each approval test?
2. Which pop-up menu items to enable for each of those icons?

People writing assertions in unit tests are familiar with the notion of actual and expected values. The parallels for approval tests are received and approved files. One or both of these files can exist, depending on the result of your test run.

For instance, if your received (actual) result matches the approved (expected) content, then the received file will not be present. However, both files will be present if there is a mismatch, thus enabling you to run a diff between the received and the approved files to understand the failure.

Therefore, which icon to show in the gutter depends on what files are present and their contents. For this problem statement, the inputs are the files and their contents. The output is the corresponding gutter icon to show.

The following diagram shows a test matrix with all possible inputs (the files' existence and what kind of content) and the corresponding outputs (gutter icon).

Now with this matrix at hand, it becomes easier for us to write the appropriate tests.

The next problem statement is to figure out what actions to enable for each of these gutter icons.

The input for this is the state of the test result (represented using the icons), and the output is a set of actions to enable. Thus, the diagram has only one input - the scenario. But, has multiple outputs - the set of actions to enable.

Benefits

• Low effort but effective technique to understand the problem statement.
• Quicker and easier to iterate, use paper & pen or a tablet.
• Easier for peer reviews.
• Offload thoughts to a different medium and free cognitive resources.
• It helps writing tests and production code faster by looking at the matrix.

What techniques do you use to write better tests? Reach out to me @ragunathjawahar and let me know.