AI-Generated Code Has 1.7x More Bugs — Here's the Fix

The data is in, and it's not what AI optimists hoped for.

CodeRabbit's "State of AI vs Human Code Generation" report, analyzing 470 real-world GitHub pull requests, found that AI-generated code produces approximately 1.7x more issues than human-written code. Not in toy benchmarks — in production repositories.

That's the headline. Here's what makes it worse:

• Logic and correctness errors are 75% more common in AI-generated PRs
• Readability issues spike more than 3x
• Error handling gaps are nearly 2x more frequent
• Security vulnerabilities are up to 2.74x higher

And this isn't an isolated finding. Uplevel's study of 800 developers found a 41% increase in bug rates for teams with GitHub Copilot access. GitClear's analysis of 211 million lines of code found that code churn — code rewritten or deleted within two weeks of being committed — nearly doubled from 3.1% to 5.7% between 2020 and 2024, with AI-assisted coding identified as a key driver.

The pattern is consistent across every major study: AI makes developers faster, but the code it produces breaks more often.

So why are some teams shipping AI-generated code with fewer bugs than before?

The Problem Isn't AI. It's the Missing Feedback Loop.

When a human developer writes code, they typically:

1. Write the code
2. Run it locally
3. Click through the UI to check it works
4. Write or update tests
5. Push to CI

When an AI coding agent writes code, most teams:

1. Prompt the AI
2. Review the diff visually
3. Push to CI

Steps 2-4 just vanished. The developer didn't run the app. Didn't click through the flow. Didn't verify the UI actually works. The AI generated plausible-looking code, the developer skimmed it, and it went straight to review.

This is where the 1.7x bug multiplier comes from. Not because AI writes worse code in absolute terms — but because the human verification step that catches bugs disappears when AI writes code fast enough that reviewing feels like enough.

What the Data Actually Shows

Let's look at what types of bugs increase most in AI-generated code:

Source: CodeRabbit, Dec 2025

Notice what's at the top: logic and correctness. Not syntax errors. Not type mismatches. The kind of bugs that only show up when you actually run the application and verify the UI behaves as expected.

Unit tests don't catch these. Linters don't catch these. Code review often doesn't catch these either — because the code looks correct. It compiles, the types check, the logic reads plausibly. You have to click through the flow to discover the bug. That's what end-to-end testing is for — and it's exactly the step that disappears in AI-assisted workflows.

Meanwhile, Technical Debt Is Compounding

GitClear's 2025 research reveals a deeper structural problem:

• Code duplication rose 8x in AI-assisted repositories
• Refactoring dropped from 25% to under 10% of code changes between 2021-2024
• Copy-pasted code blocks rose from 8.3% to 12.3% of all changes

AI tools generate new code instead of reusing existing abstractions. The result: repositories that grow faster but become harder to maintain. Each duplicated block is a future bug — when you fix one copy, the others remain broken.

What High-Performing Teams Do Differently

The teams shipping AI-generated code without the 1.7x bug penalty all share one practice: they verify AI output in a real browser before it reaches main.

Not with unit tests. Not with code review alone. With actual end-to-end verification — the same kind of "click through the app" checking that human developers do naturally, but automated so it scales with AI's speed.

Here's what that looks like at three companies using Shiplight:

Warmly: From 60% Maintenance Time to Zero

> "I used to spend 60% of my time authoring and maintaining Playwright tests for our entire web application. I spent 0% of the time doing that in the past month. I'm able to spend more time on other impactful/more technical work. Awesome work!"

— Jeffery King, Head of QA, Warmly

The 60% number is staggering but common. Industry data shows that test maintenance is one of the largest hidden costs in software development, often consuming more time than writing the tests in the first place. When tests break every time the UI changes, teams either burn cycles fixing them or stop running them entirely — leaving AI-generated code unverified.

Warmly eliminated this by switching to self-healing test automation — intent-based tests that adapt when the UI changes. The time freed up went to higher-impact engineering work, not more test maintenance.

Jobright: Reliable Coverage Within Days

> "Within just a few days, we achieved reliable end-to-end coverage across our most critical flows, even with complex integrations and data-driven logic. QA no longer slows the team down as we ship fast."

— Binil Thomas, Head of Engineering, Jobright

The key phrase: "within just a few days." Traditional E2E test suites take weeks or months to build. By the time they're ready, the AI-assisted codebase has already moved on. Jobright closed that gap by generating tests directly from their AI coding workflow — the same agent that writes code also verifies it.

Daffodil: 80% Regression Coverage in Weeks

> "We automated over 80% of our core regression flows within the first few weeks. Most manual checks are gone, ongoing maintenance is minimal, and shipping changes feels significantly safer now."

— Ethan Zheng, Co-founder & CTO, Daffodil

80% coverage of core regression flows means 80% fewer places for AI-generated bugs to hide. When every PR triggers automated verification of the most critical user paths, the 1.7x bug multiplier gets absorbed before it reaches production.

The Fix: Make AI Verify Its Own Work

The solution isn't to stop using AI coding tools. The productivity gains are real — teams using AI assistants ship features significantly faster. The solution is to close the verification gap with agentic QA testing — letting the AI agent verify its own output.

With MCP (Model Context Protocol), AI coding agents can now:

1. Write the code — same as before
2. Open a real browser — navigate to the running app
3. Verify the change works — click through flows, check the UI
4. Save the verification as a test — YAML file in your repo
5. Run tests in CI — every future PR is verified automatically

The agent that generates the code also proves it works. The verification step that humans skip when AI writes code fast enough becomes automated.

goal: Verify checkout flow after AI-generated payment update
base_url: http://localhost:3000
statements:
  - navigate: /products
  - intent: Add first product to cart
    action: click
    locator: "getByRole('button', { name: 'Add to cart' })"
  - navigate: /checkout
  - VERIFY: Cart shows correct item and price
  - intent: Fill payment details
    action: fill
    locator: "getByLabel('Card number')"
    value: "4242424242424242"
  - intent: Submit payment
    action: click
    locator: "getByRole('button', { name: 'Pay now' })"
  - VERIFY: Order confirmation page appears with order number

This test is readable by anyone on the team. It lives in your repo. When the UI changes, intent-based steps self-heal automatically — the same pattern described in AI-generated tests vs hand-written tests. And it catches exactly the type of bugs that multiply 1.7x in AI-generated code — logic errors, flow breakages, and UI regressions that unit tests miss.

The Numbers Add Up

The Bottom Line

AI coding tools are here to stay. The 1.7x bug multiplier doesn't have to be.

The teams that will win are the ones that treat AI-generated code the same way they'd treat code from a very fast junior developer: verify everything, automate the verification, and never ship without testing.

The tools to do this exist today. Get started with Shiplight Plugin — it takes one command to add automated verification to your AI coding workflow. The question is whether your team adopts it before the technical debt compounds — or after the production incident.

---

Sources:

• CodeRabbit: State of AI vs Human Code Generation (Dec 2025) — 470 GitHub PRs analyzed, AI code produces 1.7x more issues
• CodeRabbit press release (BusinessWire)
• Uplevel: Copilot 41% bug increase study — 800 developers over 3 months
• GitClear: AI Copilot Code Quality 2025 — 211M lines of code analyzed
• GitClear: Coding on Copilot (2024 projections)
• Stack Overflow: Are bugs inevitable with AI coding agents?
• Rainforest QA: The unexpected costs of test automation maintenance
• The Register: AI-authored code needs more attention