Definition of Done for AI-Generated Code

The definition of done is one of the oldest working agreements in software delivery. A team writes down what has to be true before a piece of work counts as finished, everyone holds each other to it, and the list stops the slow drift where "done" quietly comes to mean "I stopped working on it."

Most versions of that list look similar. Code written. Tests written and passing. Code reviewed. Documentation updated. Merged to the main branch. Deployed to staging. It worked for two decades because of an assumption nobody had to state: the person who wrote the code and the person asserting it was finished were the same human, and that human had a reputation to protect and a colleague who would read their work.

That assumption is what broke. Not the list.

When an agent implements a feature, the same process writes the code, writes the tests for the code, runs them, and produces the summary that says the work is complete. Each of those steps is genuinely useful. Together they form a loop with no independent observer in it. The summary is not a lie, but it is a self-report, and a self-report is not evidence.

The volume makes it worse rather than better. A reviewer who used to read four pull requests a day carefully now faces forty, each with a confident description and a green check. The scarce resource stopped being the code and became the attention required to confirm any of it is true.

So a definition of done written for AI-generated code has a different job from the classic one. It is no longer a shared reminder among people who trust each other. It is a specification of what counts as proof, and who or what produced it.

Done is three states, not one

The most useful change is to stop treating done as a single boolean and split it into three states that can be true independently. Work is finished only when all three hold.

Specification complete. There is a written statement of the intended behavior, it describes the current accepted state of the product, and the plan and task list agree with it. This is the state most teams skip, and skipping it makes the other two unfalsifiable: you cannot prove software matches its intent if the intent was a chat message that scrolled away.

Implementation complete. The code implements that accepted scope, and the checks the repository already runs are passing. Type checks, unit tests, linters, build. This is the state agents are genuinely good at reaching, and it is the state most teams mistake for done.

Evidence complete. Someone or something has exercised the built software and recorded what happened, against the acceptance criteria in the specification, in a form another person can read later. Not "the tests passed" as a claim, but a durable artifact showing which behaviors were checked and what the result was.

The value of separating them is that a partial result becomes reportable. "Specification and implementation complete, evidence phase blocked because the test tooling was unavailable" is a useful status. "Done" is not.

Why passing tests are not automatically evidence

The obvious objection is that tests already cover the third state. Sometimes they do. The question is what the test was derived from.

A test written by the same agent that wrote the implementation, in the same session, from the same understanding, tends to encode the behavior that was built rather than the behavior that was requested. If the agent misread a requirement, the code implements the misreading and the test asserts the misreading, and both go green. The loop is closed and internally consistent and wrong.

This is the specific reason acceptance criteria matter more than they used to. A criterion written before implementation, in the specification, is the one artifact in the chain that was not produced by the process being checked. Every criterion is a statement about observable behavior, which is exactly the shape an end-to-end test needs. Deriving the test from the criterion rather than from the implementation is what turns a passing test back into evidence.

The second thing that separates evidence from a claim is whether it survives the session. A summary in a terminal is gone when the window closes. A recorded result that lives in the repository, next to the specification that produced it, is something a reviewer can open next month when the same flow breaks.

A definition of done you can actually enforce

Here is a version that holds up when most of the code is agent-written. The point is not the exact wording but that every line names an artifact rather than a feeling.

Specification complete

• A feature specification exists and describes current accepted behavior, with superseded behavior removed rather than appended.
• Acceptance criteria are written as observable outcomes, phrased so that a person or a browser could check them.
• The change was filed against an existing feature where one covers it, rather than creating an overlapping specification.

Implementation complete

• The code implements the accepted scope, and scope creep beyond it is either specified or removed.
• The repository's existing checks pass without any assertion having been weakened to achieve it.
• Anything the agent could not do is stated, not silently omitted.

Evidence complete

• Each acceptance criterion has been exercised against the running application.
• The result is recorded in a durable artifact in the repository, not only in a chat summary.
• Failures are attributed: an application bug, a stale test, infrastructure, or configuration. An unattributed red result is not evidence either.

The third bullet in that last group is the one teams add after their first bad month. The expensive part of a failing suite was never the fix. It is the morning a senior engineer spends deciding which of those four things went wrong.

The rule that keeps it honest

One rule matters more than the rest: a broken application must never be resolved by editing the test.

Once agents maintain tests, this stops being a hypothetical. A test fails, and the cheapest path to green is to soften the assertion or update the expected value. An agent optimizing for a passing suite will take that path, and the change will look reasonable in a diff. Whatever your definition of done says, it needs a line that makes weakening a check to reach green an explicit failure rather than a repair, and a triage step that distinguishes "the test drifted" from "the product broke."

Where this is more than you need

Honest scope. Three states and a written artifact per state is real overhead, and it does not pay for itself everywhere.

• Prototypes and spikes. If the code is going to be thrown away, the specification state costs more than it returns. Implementation complete is a reasonable place to stop.
• Small internal tools with one user. When the person who requested the change is the person running it, they are the evidence phase.
• Teams with strong existing practice. If your acceptance criteria are already written before implementation and your suite is already trusted, this formalizes what you do rather than adding to it.
• Changes with no observable behavior. A dependency bump or an internal refactor has nothing for the evidence phase to exercise beyond the existing suite. Do not invent a browser check to satisfy a checklist.

Making the third state cheap

The reason evidence gets skipped is rarely disagreement. It is cost. If proving a feature works means a human clicking through the application, the third state loses to shipping velocity every time, and the definition of done quietly becomes a two-state list again.

That calculation changes when the agent that built the feature can also drive a browser. Shiplight gives a coding agent eyes and hands in a real browser, so the acceptance criteria in the specification become checks the agent runs against the rendered application and then keeps. It installs as a browser MCP server plus a set of skills, with a one-line install for Claude Code, Cursor, Codex, VS Code, and more.

/shiplight cover decides what proof a change needs before writing anything, on the principle of choosing the cheapest sufficient proof: a deterministic end-to-end test for a flow a browser can walk, or an agent verification when the state behind the interface also has to be judged. What it produces is readable YAML in your own repository, authored from intent rather than selectors, plus a recorded test report that stays with the specification. When a test fails because the application is broken, /shiplight fix reports the bug rather than editing the test to pass, which is the rule above enforced in tooling rather than in a document.

For the workflow that wires all three states together, including the handoff into the evidence phase, see Spec Project.

Frequently Asked Questions