AI Testing Glossary

Agent-native QA describes quality assurance tools designed so AI coding agents can invoke them directly as peers — through agent-callable interfaces (typically MCP) — rather than human dashboards. The AI agent is a first-class user, not just an internal feature.

Agentic QA testing is a model of software quality assurance where AI agents drive the testing loop end-to-end — deciding what to test, generating tests, executing them, interpreting results, and healing broken tests — without human intervention at each step.

AI test debt is the accumulated quality liability that results when AI coding agents author production code faster than tests can keep up — including untested code paths, brittle scripts that survived a redesign by accident, and quarantined tests left unfixed. Like financial debt, it compounds.

AI-augmented testing is software testing where AI assists humans inside a fundamentally human-driven workflow — smart locators, suggested test cases, auto-complete for scripts, intelligent test selection. The human still drives every step; AI is a feature, not the operator. Distinct from AI-native testing.

AI-native testing is software testing built from the ground up around AI as the primary operator — AI authors tests from intent, executes them, interprets results, and heals broken tests, with humans setting policy and reviewing outcomes. Distinct from AI-augmented testing, where AI assists a human-driven workflow.

Coverage decay is the gradual erosion of test coverage that happens when a codebase changes faster than its tests can keep up — new code paths ship without tests, old tests cover paths that no longer exist, and the gap widens silently. AI coding agents accelerate decay because they ship code faster than humans write tests.

A flaky test is an automated test that produces inconsistent results — passing and failing intermittently against the same code — without any change to the system under test. Flakiness is a signal-quality problem, not a code-correctness problem.

Intent-based testing is a test authoring style where each step describes what the user is trying to do (intent) rather than how to perform it (selectors and actions). Tests survive UI changes because the system re-resolves the correct element from intent at runtime.

Intent-cache-heal is a deterministic execution pattern for AI-native E2E testing: the test author records user intent, the runtime caches a fast deterministic locator for each step, and AI resolution is invoked only when the cached locator fails. Combines the speed of script-based tests with the survivability of intent-based ones.

MCP testing is the practice of exposing browser automation, test generation, and verification capabilities as Model Context Protocol (MCP) tools so that AI coding agents — like Claude Code, Cursor, Codex, and GitHub Copilot — can invoke them directly during development.

PR-time verification is the practice of running automated tests at the moment a pull request is opened — before code review begins — so reviewers see verified, passing changes rather than guessing whether the change works. AI coding agents make this possible at scale by invoking the test layer themselves.

Quarantining is the practice of removing a flaky test from the merge-blocking suite while keeping it running in a separate, non-blocking lane until the underlying flake is fixed. Quarantine preserves history and signal without letting flakiness corrupt the main pipeline.

A self-healing test is an automated test that adapts to UI changes at runtime — when a target element moves, renames, or restructures, the test re-resolves the correct element instead of failing. Self-healing eliminates the maintenance tax of brittle selectors.

A test flakiness budget is an explicit, team-agreed maximum flake rate for the test suite — a quality SLO. When the budget is exceeded, new feature work pauses until flake-rate drops back under the threshold, the same way SRE error budgets gate deployment.

A verification agent is an AI agent whose specialized role is to confirm that a code change behaves correctly — opening a real browser, exercising the change, comparing observed behavior to expected outcomes, and reporting structured results. It is distinct from the coding agent that authored the change.