Testing
Given the two contracts, most of what a user interface test suite traditionally checks is already decided in one place and checked centrally. What is left is a much smaller and much more valuable question: can a person still get through?
Journey-first makes that argument in full, including the four cases where it is false. This section is the practical half: which level answers what, and how to divide a finite budget between them.
The five levels
Section titled “The five levels”Four are automated. The fifth is a person, and it is the one the other four exist to make affordable.
| Level | Scope | Answers | Fails with |
|---|---|---|---|
| Interaction tests | One component, one state | Does it behave as its contract says? | A component name and a state |
| Accessibility tests | A story, and a journey | Are the mechanical guarantees intact? | A rule id and an element |
| Visual regression | A story, rarely a page | Did something change that nobody meant to? | A picture, and a question |
| Journey tests | A whole path, in a browser | Can a person get from intent to outcome? | A step of a journey |
| Usability testing | A whole path, with a person | Does a real person actually get through? | An observation, and often a redesign |
Read the last column as the real cost model. A failure that names a component and a state is cheap to act on. A failure that hands you two pictures and asks you to decide is expensive, and it is expensive every time, including the ninety percent of times the change was intended.
The budget
Section titled “The budget”Rough shape, for a product where the interface presents behaviour rather than being it:
- A journey test per named journey. Five to fifteen for most products. These are the tests that would have caught the outage nobody caught.
- Interaction tests for every component with behaviour. Cheap, fast, precise. There is no reason to be sparing here.
- An accessibility scan on every story and every journey step. Effectively free once wired, and it catches a class of regression humans do not.
- Visual snapshots on components, deliberately, not on pages. A small set that is actually triaged beats a large set that is bulk-approved.
- Five participants on the journey, several times a year. A round of usability testing fits in a day and finds the class of problem no assertion is ever written for. This is a budget line, not a research project.
The proportions matter less than one property: every level’s failures must be cheap enough to act on that nobody learns to ignore them. A suite people have learned to ignore has negative value — it costs runtime and provides false assurance.
What no level can tell you
Section titled “What no level can tell you”Worth being explicit, because a green suite invites the wrong conclusion.
- Whether the journey is the right journey. That is journey mapping and observation in production. Usability testing tells you whether people can follow the journey you chose, not whether you chose the right one.
- Whether the design is good. That is critique, and a person looking at the real thing.
- Whether it is usable by someone with a disability. Automated scanning catches perhaps a third of real barriers; the rest need a keyboard, a screen reader, and ideally a person who uses one — which is usability testing with the participants teams are most likely to skip.
- Whether it is fast enough to be pleasant. A separate instrument, on real devices and real networks.
- Whether anyone wants it. Nothing in this section answers that. Five people who agreed to attempt a task you gave them are not demand.
The order to build them in
Section titled “The order to build them in”If a team has none of this, the order is not the order of the table above.
- One round of usability testing, on whatever exists today. It costs a day, needs no infrastructure, and will reorder everything else on this list.
- One journey test, for the journey the product exists for. It will find something on the first run.
- The accessibility scan, wired into whatever already runs. Nearly free, immediately useful.
- Interaction tests as components enter the library. Do not backfill; add them with the contract.
- Visual snapshots last, and small. They are the level whose value most depends on the previous four already being in place.
Usability testing is first on that list rather than last for a reason: it is the only level that can tell you the other four are being pointed at the wrong journey.
Starting from the other end — a large screenshot suite over an interface with no contracts — is the most common way to spend a quarter and end up with a suite that is bulk-approved by month four.