Testing AI Features

A normal assertion assumes the same input produces the same output, which stops being true the moment a model is in the path. These pages cover what to assert instead, and how to tell a real regression from the model wording something differently.

10 articles, newest first.

Illustrated Shiplight blog cover: a glossy test card stamped with a bright green checkmark while a small crack in the code block beside it goes unnoticed, and a second card catching the same crack.
AI TestingEngineering

Do AI-Generated Tests Actually Catch Bugs?

Illustrated Shiplight blog cover: a glossy chat window emitting three differently worded responses that all pass through one structural assertion gate into a bright green checkmark.
AI TestingEngineering

How to Test AI Features in Your Product

a glossy indigo verification shield with a green checkmark protecting a floating panel of AI-generated code on a soft lavender background.
AI TestingEngineering

Can You Trust AI-Generated Code?

Illustrated Shiplight blog cover: a glossy code window where one fabricated hallucinated element glows red and is caught by a green verification checkmark magnifier.
AI TestingEngineering

How to Catch Hallucinations in AI-Generated Code

Illustrated Shiplight blog cover: a glossy magnifying glass with a green checkmark inspecting a floating browser window that is running a piece of AI-generated code.
AI TestingBest Practices

How to Verify AI-Generated Code: A Practical Checklist

Illustrated Shiplight blog cover: a glossy central code edit sending ripples outward to connected app-flow nodes, a couple turning red while a green safety net of tests catches them.
EngineeringAI Testing

The Regression Risk of AI-Generated Code (and How to Contain It)

Marketing cover with the headline 'Testing Strategy for AI-Generated Code.' on the left and a 7-layer defense-in-depth stack on the right: bands ramping from light to dark indigo labeled Spec before generation, Behavioral over unit, Contract tests at boundaries, Treat rewrites as untested, Blocking PR-time gate, Human review, and Regression learning
AI TestingBest Practices

How to Build a Testing Strategy for AI-Generated Code (2026)

Marketing cover for 'Catch What Code Review Misses' with a Shiplight indigo 'Bug Detection 2026' pill badge and five icon tiles representing the five detection techniques (Detect, Browser, E2E, Gates, Review)
AI TestingEngineering

How to Detect Hidden Bugs in AI-Generated Code (2026)

Illustrated Shiplight blog cover: a glossy block of AI-generated code with several small glossy red bug icons surfacing from it, under a magnifier.
Engineering

AI-Generated Code Has 1.7x More Bugs: Here's the Fix

Illustrated Shiplight blog cover: two glossy test cards balanced side by side, one AI-generated marked with a chip and one hand-written marked with a pen.
AI TestingTesting Strategy

AI-Generated Tests vs Hand-Written Tests: When to Use Each

Other topics

Agent-Native TestingFlaky TestsSelf-Healing TestsMCP and Agent ToolingPlaywrightAI Coding AgentsEnd-to-End TestingCI and Release GatesTool ComparisonsEnterprise and ComplianceQA FoundationsSpec-Driven DevelopmentNo-Code and Natural-Language TestingAI Testing StrategyAgentic QA

See testing that keeps up with your AI coding agents.