---
title: "Smoke Testing: The Check That Should Never Be Slow"
excerpt: "A smoke test answers one question: is this build broken badly enough that nothing else is worth running? Its whole value is speed, and the usual failure is letting it grow until nobody wants to wait for it."
metaDescription: "Smoke testing explained: what belongs in a smoke suite, how it differs from regression and sanity testing, and how to keep it fast enough to stay useful."
publishedAt: 2026-07-30
author: Shiplight AI Team
categories:
 - Guides
 - Best Practices
tags:
 - smoke-testing
 - test-strategy
 - e2e-testing
 - ci-testing
 - qa-process
metaTitle: "Smoke Testing: Keep It Fast to Keep It Useful"
featuredImage: ./cover.png
featuredImageAlt: "Illustrated Shiplight blog cover: a small fast gate at the front of a delivery pipeline catching a broken build before the longer stages run."
---
Before spending forty minutes running a full suite, it is worth spending ninety seconds finding out whether the build is fundamentally broken. That is the entire job of a smoke test, and almost every problem teams have with them comes from forgetting it.

## What a smoke test is

**A smoke test is a small, fast set of checks that confirms a build is functional enough to be worth testing further.** It is a gate, not an assessment.

The name comes from hardware: power on the board, and if smoke comes out, stop. Nobody runs a full diagnostic on a component that is on fire.

Applied to software, a smoke test answers one question. *Does the application start, and do its most critical paths work at all?* If the answer is no, the detailed suite would only produce a long list of failures caused by the same root problem, and the time spent producing that list is wasted.

## What belongs in one

The test for inclusion is severity, not importance. Ask: **if this were broken, would running anything else be pointless?**

For a typical web application that usually means:

- The application loads
- A user can authenticate
- The main navigation works
- The core action of the product succeeds once, on a happy path
- Critical integrations respond

That is roughly it. Five to ten checks, finishing in a minute or two.

What does not belong: edge cases, error handling, validation messages, permutations, anything unusual. Those matter and they belong in the regression suite. Putting them in the smoke suite is the single most common mistake, because each one is individually defensible and collectively they destroy the thing that made the suite useful.

## How it differs from neighbors

| | Smoke | Sanity | Regression |
|---|---|---|---|
| Question | Is the build viable? | Does this specific fix work? | Did anything previously working break? |
| Scope | Broad, shallow | Narrow, focused | Broad, deep |
| Duration | Seconds to minutes | Minutes | Long |
| When | Every build | After a targeted fix | Before release, or on every PR |
| On failure | Stop everything | Reject the fix | Investigate the specific break |

**Smoke** is broad and shallow: touch everything critical, examine nothing closely.

**Sanity** is narrow and focused: a quick check that one particular change did what it claimed, without running the world.

**Regression** is broad and deep, and it is what most people mean when they say "the test suite."

The distinction that matters operationally is what happens on failure. A smoke failure stops the pipeline. A regression failure means investigating one thing.

## The failure mode: suite creep

Every smoke suite grows. The mechanism is always the same and it is worth naming because it is preventable.

A bug reaches production. In the retrospective someone asks how it escaped. Someone else proposes adding a check to the smoke suite, because that runs first and would have caught it earliest. It is a reasonable suggestion and nobody objects.

Repeat that fifteen times over a year, and the ninety-second gate takes twelve minutes. Developers start pushing without waiting for it. The gate that was valuable because it was fast is now a gate people route around, which is worse than not having one.

Two rules keep it honest:

**A time budget, enforced.** Pick a number, two minutes is a common one, and treat exceeding it as a defect. When something new must go in, something else comes out or gets faster.

**The severity test, applied every time.** "Would running anything else be pointless if this broke?" A validation message failing does not make the rest of the suite pointless. Login failing does.

## Where it runs

Smoke tests earn the most in three places:

**On every pull request, first.** Fail fast before the expensive stages start.

**Against a preview deployment**, so you are checking the built artifact in a realistic environment rather than a developer's machine. See [how to test Vercel preview deployments automatically](/blog/test-vercel-preview-deployments).

**Immediately after a production deploy.** The highest-value smoke run of all, because it is the one that tells you to roll back. A deploy nobody verified is a deploy you find out about from a customer.

## What changes when agents write the code

The economics shifted, and it cuts in a specific direction for this kind of test.

A smoke suite was historically hand-written and hand-maintained, so keeping it small was partly a resource constraint dressed up as discipline. When the agent that writes a feature can also exercise it in a real browser, generating checks stops being the bottleneck. See [shift left testing](/blog/shift-left-testing).

That does not mean the smoke suite should grow. It means the *regression* suite can grow, absorbing the checks that used to get pushed into smoke for lack of anywhere better. The gate stays small because there is now a good home for everything else.

The other change is what needs smoke-testing. Agents refactor aggressively, so the "did the app fundamentally survive that" question gets asked more often per day than it used to. A characteristic agent failure is a quietly removed safeguard rather than an obvious break, which smoke will not catch. Smoke is a viability gate, not a safety net. See [regression risk in AI-generated code](/blog/regression-risk-ai-generated-code).

[Shiplight](/plugins) fits this by letting the agent verify a change in a real browser and commit the check as intent-based YAML, so smoke and regression suites both grow without the maintenance cost that used to cap them. Tests are tagged and run in whichever stage you choose. Web applications; native mobile is not in it.

## Frequently Asked Questions

### What is smoke testing?
A small, fast set of checks confirming a build works well enough to be worth testing further. It is a viability gate rather than an assessment of correctness.

### What is the difference between smoke and sanity testing?
Smoke is broad and shallow, run on every build to confirm the whole thing is viable. Sanity is narrow, run after a specific fix to confirm that one change worked.

### How long should a smoke test take?
A couple of minutes at most. Speed is the entire value, and a suite people wait too long for is one they will push around.

### How many tests should a smoke suite have?
Usually five to ten. The test for inclusion is whether running anything else would be pointless if that check failed.

### Should smoke tests run in production?
Yes, right after a deploy. That is the run that tells you to roll back, and skipping it means learning about the failure from a customer.

## Related Reading

- [End-to-end testing](/blog/end-to-end-testing): the fuller suite smoke sits in front of
- [Shift left testing](/blog/shift-left-testing): why generating checks got cheap
- [How to test Vercel preview deployments](/blog/test-vercel-preview-deployments): where smoke runs best
- [E2E testing in GitHub Actions](/blog/github-actions-e2e-testing): wiring the gate into CI