How to Test Code Written by Gemini CLI

Code an AI agent writes in your terminal still has to run in a browser, and testing it means confirming the running application behaves the way a real user expects, not just that the source compiles. When an agent edits files, refactors components, and touches adjacent features in one command, the fastest reliable way to verify its work is a loop that opens the actual app, exercises the changed flow, and records that check as a test that survives the next refactor.

That loop has three parts: live confirmation that a change looks and works right, durable end-to-end tests that catch regressions later, and a CI gate that blocks any change breaking a flow that already worked. This guide walks through each part, then shows how to wire the whole loop into a terminal-based coding agent so verification happens where the code gets written.

Why Gemini CLI output needs verification

Gemini CLI is Google's open-source AI agent that runs in your terminal, reads and edits your codebase, runs shell commands, and completes multi-step tasks from a natural-language prompt. It is licensed Apache 2.0 and extensible through the Model Context Protocol, which matters later. Like every capable coding agent, it optimizes for code that satisfies the task you described, which is not the same as code that is correct across every path a user can take.

Four failure modes recur with agent-written code, and none of them are Gemini CLI-specific:

• Unmentioned edge cases. The agent implements what the prompt described. Empty states, error paths, and unusual inputs that were never spelled out are frequently left unhandled.
• Cross-browser behavior. Generated CSS and JavaScript can render or execute differently across browser engines, and a terminal session never opens a browser to notice.
• Side effects in adjacent code. A change scoped to one feature can alter behavior in another feature it touched indirectly, especially when the agent refactors shared components.
• Real user flows under real conditions. A feature that works in isolation can still fail once authentication, live data, or a specific browser state is involved.

Research on AI-generated code points to the same root issue: bug rates climb when the verification step cannot keep pace with the generation step. Gemini CLI can rewrite a dozen files before you finish reading the diff, and manual click-through does not scale to that speed. Verification has to be automated and live in the same loop as the code. Every terminal agent hits this, which is why the workflow here mirrors what we cover for OpenAI Codex testing and for adding testing to Cursor, Copilot, and Codex.

Setup: connect verification to Gemini CLI through MCP

Gemini CLI reads MCP server definitions from a settings.json file, either the user-level ~/.gemini/settings.json or a project-level .gemini/settings.json. Servers are declared under an mcpServers object, and the CLI supports stdio, Server-Sent Events, and HTTP streaming transports. A stdio server, the common case for a locally installed tool, looks like this:

{
  "mcpServers": {
    "shiplight": {
      "command": "npx",
      "args": ["-y", "shiplight", "mcp"]
    }
  }
}

Once a server is registered, its tools become available inside the agent's session, and MCP servers can also expose slash commands and packaged extensions. That extension model makes MCP the right integration point: it turns "give the agent a browser" from a custom scripting project into a one-line config entry. MCP for testing covers why the protocol fits verification work, and why intent-level tools matter more than raw browser primitives.

Shiplight's browser MCP server installs as an MCP server plus a set of Skills, with a one-line install across Gemini CLI, Claude Code, Cursor, Codex, VS Code, and 40 or more agents. Local browser automation and test authoring need no account or token, so the loop below runs entirely on your machine before any hosted service is involved.

The verification loop: verify, create-tests, triage

With the MCP server connected, Gemini CLI gets three commands that map onto the three parts of the loop. Each one addresses a distinct failure mode.

Verify a change as soon as it is written

After the agent implements a change, /shiplight verify has it open the running application in a real browser, navigate to the affected feature, walk the user journey end to end, and confirm the expected result is on screen, capturing screenshots as evidence. This is the step a terminal session cannot do on its own: the agent gets eyes on the change in the same loop that produced it, with no switch to a separate test environment. Integration bugs that unit tests miss surface here, at the point of implementation, when they are cheapest to fix.

Create tests that outlive the next refactor

One-time verification catches a bug now. A persistent test catches the regression a future change introduces. /shiplight create-yaml-tests has the agent walk the app and write end-to-end tests as readable YAML, expressed as user intent rather than brittle DOM selectors:

goal: Verify project creation and collaborator invite
base_url: https://app.example.com
statements:
  - URL: /dashboard
  - intent: Click "New Project" to open the creation dialog
  - intent: Enter a project name and invite a collaborator by email
  - intent: Click "Create Project"
  - VERIFY: New project appears in the dashboard project list

Intent-based tests matter more, not less, with an agent that refactors aggressively. Gemini CLI renames classes and reorganizes component trees as part of ordinary work, and tests pinned to a CSS selector break every time it does. A test that describes what the user is doing survives, because intent does not change when the DOM does. When a cached locator goes stale, the test resolves the intent against the current page instead of failing, which is the intent-cache-heal pattern: intent as the source of truth, cached locators for speed, AI resolution when the cache misses. Heals surface as reviewable pull-request diffs, not silent rewrites.

These tests live in your own git repository and run locally with npx shiplight test. They are Playwright-compatible and run alongside an existing Playwright suite, so there is no rip-and-replace to adopt them.

Triage failures without editing away the signal

When a test fails, /shiplight fix reproduces the failure in a real browser and works out the root cause. The discipline that matters: if the application is genuinely broken, triage reports the bug rather than quietly rewriting the test to pass. A change that broke a real flow should fail loudly, which is what keeps a self-maintaining suite honest as the agent keeps shipping into it.

Gate every change in CI

A test suite that runs only when someone remembers is advisory. To make it a real quality bar, the suite has to run automatically on every change and block the ones that break a working flow. Because the YAML tests live in your repository, they run in CI like any other check. With GitHub Actions:

name: E2E Regression Tests
on:
  pull_request:
    branches: [main, staging]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run E2E suite
        run: npx shiplight test

When a change from a Gemini CLI session breaks a flow, the pull request goes red, and the agent reads the failure output and fixes the issue before a human opens the diff. That closes the loop: the agent implements, verifies in a browser, writes tests, and answers to CI, all without waiting for someone to click through the feature by hand.

For teams that need managed execution, Shiplight-hosted runners run the same YAML in parallel across concurrent pull requests, and the enterprise tier adds SOC 2 Type II, a 99.99% uptime SLA, and private-cloud or VPC deployment. The tests stay in your repository either way, so there is no vendor-cloud lock-in on what matters most.

What to automate versus what to review by hand

The point is not to remove human judgment. It is to make sure that by the time a change from Gemini CLI reaches review, you already know it did not break anything that worked before. Reviewers then focus on whether the implementation is right for the requirement, not on whether it silently broke the login flow. The same split applies to any terminal or editor agent; if you also drive Cursor for coding, our Cursor testing guide covers the editor-based variant of the same loop.

Key Takeaways

• Gemini CLI produces code that satisfies the prompt, not code guaranteed to hold up across edge cases, browsers, and adjacent flows. Verification has to be automated to keep pace.
• MCP is the integration point: a few lines in ~/.gemini/settings.json give the agent a real browser and test-authoring tools in its own session.
• The loop is verify, create-tests, triage: confirm a change live, capture it as an intent-based YAML test, and root-cause failures instead of editing them away.
• Intent-based tests survive the agent's frequent refactors because they describe user behavior, not DOM structure. A CI gate turns the suite from advisory into blocking.

Frequently Asked Questions

How do I test code written by Gemini CLI?

Connect a verification tool to Gemini CLI as an MCP server, then run a three-step loop: verify each change in a real browser as it is written, capture that check as a durable end-to-end test in your repository, and gate the test suite in CI so any change that breaks a working flow fails the pull request. This keeps testing at the same speed the agent generates code.

What is Gemini CLI and why does its output need extra testing?

Gemini CLI is Google's open-source, terminal-based AI agent that reads and edits your codebase and runs commands from a natural-language prompt. Because it works in the terminal, it never opens a browser to see whether a UI change actually renders and behaves correctly, and it can touch many files at once faster than a human can review. That combination makes automated browser verification and regression tests necessary rather than optional.

How do I connect a testing tool to Gemini CLI?

Gemini CLI reads MCP server definitions from ~/.gemini/settings.json or a project-level .gemini/settings.json, under an mcpServers object, and supports stdio, SSE, and HTTP streaming transports. Adding a browser-verification server such as Shiplight is a one-line install, after which the agent gains verify, create-tests, and triage commands, with no account or token needed for local use.

Will tests break every time Gemini CLI refactors the UI?

Not if the tests describe user intent instead of CSS selectors. Gemini CLI renames classes and reorganizes components as part of normal work, which breaks selector-based tests constantly. Intent-based YAML tests describe what the user does, so they survive refactors, and when a cached locator goes stale the test resolves the intent against the current page and surfaces the heal as a reviewable pull-request diff.

Do the tests run in CI like the rest of my checks?

Yes. The YAML tests live in your git repository and run with npx shiplight test, so a GitHub Actions job runs them on every pull request and blocks any change that breaks a flow. When a Gemini CLI change fails a test, the agent can read the failure output and fix the issue before a human opens the diff.

---

References: Gemini CLI (google-gemini/gemini-cli), MCP servers with Gemini CLI, Introducing Gemini CLI, an open-source AI agent, Gemini CLI Extensions documentation, Model Context Protocol