What to Automate, What to Skip: A Testing Strategy for Fast-Moving Teams

Ask five engineering teams what their testing strategy is, and you'll get five versions of the same non-answer: "we try to test the important stuff." Nobody disagrees with that in principle. The problem is that "important" is never defined, so it defaults to whatever the last incident was, and the test suite becomes a scrapbook of past fires instead of a deliberate defense against future ones.

A real testing strategy answers a narrower question: for this specific piece of behavior, does automated coverage pay for itself, or does it cost more to write and maintain than the bugs it would ever catch?

The Real Cost of a Test Isn't Writing It

Everyone budgets the time to write a test. Almost nobody budgets what happens after: every test in your suite is a piece of code that has to be maintained, that will occasionally fail for reasons unrelated to a real bug, and that someone has to triage every time CI goes red. A test suite that takes 45 minutes to run and requires someone to "just rerun the flaky ones" isn't providing confidence, it's providing friction with a green checkmark attached.

The real question for any test isn't "could this catch a bug." Almost any test could, in theory. The question is whether the maintenance cost over its lifetime is lower than the cost of the bugs it actually catches, and for a large fraction of tests teams write, the honest answer is no.

What Actually Earns Automated Coverage

Business logic with real branching

Pricing calculations, permission checks, state machines, anything where a wrong branch produces a wrong answer that a human reviewer might not notice. This is the highest-value category for unit tests, because the bugs here are silent: the code runs, it just runs wrong, and nothing crashes to tell you.

Integration points between systems you don't fully control

The boundary where your code calls a payment processor, an auth provider, or another team's API. These break in ways unit tests can't see and manual QA can't reliably catch every regression on, because the break isn't in your code, it's in the contract between your code and someone else's.

Anything that has actually broken in production before

A regression test for a real incident is close to free-value. You already paid the cost of discovering the bug the hard way; the marginal cost of locking it down is low, and the confidence it buys is real, because you know exactly what it's protecting against.

The critical path a customer would notice immediately

Checkout, login, the core action your product exists to perform. This is where end-to-end coverage earns its cost, specifically because a break here is not a minor bug, it's a revenue-stopping incident, and the few minutes an E2E suite takes to run is cheap insurance against that.

What's Usually Safe to Skip (or Test Manually)

• Pure UI layout and styling. A snapshot test for CSS spacing breaks on every intentional design change and catches almost nothing real. A human glancing at a PR preview catches visual regressions faster and cheaper.
• Third-party library internals. You don't need to test that a library you depend on works, that's the maintainer's job, and their test suite is bigger than yours will ever be.
• Exhaustive input permutation testing for low-stakes forms. Testing every combination of a contact form's optional fields has a maintenance cost that rarely justifies the bug it might catch.
• One-off scripts and internal tooling with a tiny blast radius. If it breaks, one person notices immediately and fixes it. That's an acceptable failure mode; it doesn't need a CI gate.

A Practical Shape for the Pyramid

The classic testing pyramid (lots of unit tests, some integration tests, few end-to-end tests) is still directionally right, but the ratio matters less than the reasoning behind each layer:

If your suite is inverted, mostly E2E tests with thin unit coverage, you'll know it from the symptoms: CI takes forever, failures are hard to localize, and engineers start ignoring red builds because "it's probably flaky." That pattern is a strategy problem, not a discipline problem, and more tests written the same way won't fix it.

Tie It to Delivery Velocity, Not Coverage Percentage

Code coverage percentage is a vanity metric that's trivially gameable and correlates weakly with actual defect rates. The metrics that matter are the same ones that show up in DORA: change failure rate (did this deploy cause an incident) and mean time to resolution (how fast did we find out). A testing strategy is working when those two numbers improve, regardless of what the coverage badge says.

Key takeaways:

1. The real cost of a test is its maintenance burden over years, not the ten minutes it took to write.
2. Automate business logic, cross-system integration points, past incidents, and true critical paths. Everything else is a candidate for manual review instead.
3. An inverted pyramid (heavy E2E, thin unit coverage) produces slow, flaky CI that engineers learn to ignore, which is worse than no tests.
4. Measure change failure rate and MTTR, not coverage percentage. Coverage tells you what ran; it doesn't tell you what's actually safe.

If your CI pipeline has gotten slow enough that people are quietly ignoring red builds, that's usually a strategy problem worth fixing before it costs you a real incident. Book a discovery session and we'll look at where your suite's actually earning its keep.