Anatomy of an A/B Test

Every A/B test has five parts: a hypothesis, a control, one or more variants, a primary metric, and a duration. Skip any of them and the result will not survive scrutiny.

The hypothesis is the single most undervalued artefact in CRO. Write it in the form: Because we observed X, we believe that changing Y will cause Z, measured by M. If you cannot complete the sentence honestly, you are not ready to ship.

Variants must change one thing at a time unless you are running a multivariate test with the traffic to support it. The primary metric is decided before the test launches — never after. Duration is governed by sample size, not by impatience.

When the test ends, write the result down: win, loss, flat. Add a one-paragraph lesson. The archive of past tests is the most valuable asset an experimentation programme owns.

const hypothesis = {
  because: 'visitors hesitate at the CTA',
  weBelieve: 'a clearer verb will help',
  willCause: 'a lift in click-through',
  measuredBy: 'CTR on [data-test=hero-cta]',
};