Shiplight vs testRigor: Intent-Based Testing Compared

Shiplight puts your tests where you can own them and lets your coding agents write them; testRigor keeps them in a vendor console authored by QA staff. That is the decision in one line, and it follows from opposite architecture bets made a decade apart.

Where the two diverge

Shiplight is built for teams that ship with AI coding agents: the agent authors tests as readable YAML in your git repo, they run locally with no account or on hosted CI runners, and the runtime is built on Playwright, so nothing is locked to a proprietary engine.

testRigor is a cloud-hosted platform from the pre-agent generation of no-code testing (founded 2015), designed so manual QA staff can author tests in a constrained plain-English command language inside testRigor's web console, run on testRigor's hosted runners, and leave only under paid-customer agreements.

The two diverge at every layer that matters: where tests live, who authors them, what the escape hatch is when logic gets complex, how failures heal, and what you can take with you.

We build Shiplight, so read this as our perspective, checked against both vendors' documentation. Last verified: 2026-07-13.

Quick Comparison

How They Work, Side by Side

Shiplight: agent-authored YAML in your repo

Shiplight tests are YAML files with natural-language intent statements. They live in your git repo, are reviewable in PRs, and run anywhere Node.js runs:

goal: Verify user can create a new project
statements:
 - intent: Log in as a test user
 - intent: Navigate to the dashboard
 - intent: Click "New Project" in the sidebar
 - intent: Enter "My Project" in the project name field
 - intent: Click the Save button
 - VERIFY: the project appears in the project list

The authoring model is the decisive difference. Shiplight installs into your coding agent as an MCP server plus Skills (one-line install for Claude Code, Cursor, Codex, VS Code, and more). The agent that builds a feature verifies it in a real browser and writes the regression test as a byproduct of shipping. Locators are a cache committed to the repo; cached steps replay deterministically and call no LLM, AI re-resolves only when the UI changes, and larger changes arrive as a reviewable PR diff from the triage agent, so every heal is reviewable before it lands. When a flow needs real logic, the escape hatch is inline JavaScript in a YAML step, hand-tuned in a local debugger with screenshots and traces.

Shiplight's trade-offs, stated plainly: it is web-focused, with no native mobile or desktop testing. And if your team has very strong engineers and a Playwright suite that genuinely works, Playwright is not your bottleneck and Shiplight is not the tool to reach for.

testRigor: a constrained plain-English DSL in a vendor console

testRigor's design center is making manual QA staff productive without engineers. Tests are written as sequences of commands in a constrained plain-English language:

login
click "New Project"
check that page contains "Project created successfully"
enter "My Project" into "Project Name"
click "Save"
check that page contains "My Project"

This reads like free English but is not: the language is a defined command set with its own syntax (testRigor's docs say the parsed English "has some syntax to it"), and free-form phrasing is translated by an LLM into that command set. Tests are created, stored, and executed in testRigor's cloud console. Element location works by matching visible attributes, with an AI screenshot fallback. When a flow exceeds what the command language expresses, the escape hatch is embedded ECMAScript 5.1 JavaScript invoked as strings, a decade-old JavaScript dialect, rather than modern inline code.

testRigor's design center is the non-technical QA tester in a manual-QA-heavy organization, often a non-software company, working across web, mobile, desktop, and API surfaces from one console. That buyer profile is distinct from engineering-led teams.

On reliability, independent review-site feedback is worth reading with its small sample size in mind: recurring complaint themes on G2 and Capterra include nondeterministic failures on their hosted runners (tests that fail, then pass unchanged on re-run) and the absence of built-in test management.

The Core Difference: Agent-Native vs Agent-Integrated

Both products now have MCP servers, which makes the distinction worth being precise about.

testRigor's MCP server is agent-integrated: it wraps the cloud console so an agent can drive it. The tests it produces still live in testRigor's cloud, in testRigor's format, running on testRigor's runners.

Shiplight is agent-native: the agent itself authors and maintains the tests, the tests live in your repo as YAML, and they run locally with no account. There is no per-action round-trip to a vendor console, which is why regression suites get built in days rather than months.

The combination that matters is the whole loop: agent authors tests as YAML in your git repo, intent-level heals arrive as reviewable PR diffs, and the YAML transpiles to Playwright at run time.

Test Ownership and Portability

testRigor

Tests are created and stored in testRigor's cloud platform and executed on testRigor's infrastructure. The plain-English format is proprietary to testRigor's interpreter. Per the founder's public statements, Selenium conversion is available only under paid-customer agreements; there is no self-serve export.

Shiplight

Tests are YAML files committed to your repository. The source of truth lives in git, not in a vendor's cloud. Shiplight Cloud adds managed execution, dashboards, and scheduling for the same repo-based files.

If you leave Shiplight, your test specs stay with you, and because Shiplight is built on Playwright, the flows they describe are standard browser automation, not a proprietary interpreter's dialect.

Pricing

testRigor

testRigor does not publish paid plan pricing as of this writing; budgeting requires a sales conversation. Capacity is sold in virtual machines, with machines added to reduce execution time as suites grow. Shiplight's plans and rates are published, which is the only reason this section can compare anything.

Shiplight

Shiplight starts free: local browser automation and test authoring need no account or token at all, and the Free plan is $0 forever with every core feature included, from AI actions and auto-healing to MCP access and CI integrations. Pro is $60 per month. Enterprise includes SOC 2, VPC deployment, RBAC.

Stated honestly: testRigor does not publish full platform pricing. Shiplight does, at Free $0 forever and Pro $60/month, and the MCP server, Skills, browser automation and test authoring need no Shiplight account at all, so that part is evaluable without a sales call.

When testRigor's Design Center Applies

testRigor was built for a specific buyer: manual-QA-heavy organizations where non-technical QA staff own testing and no engineers are available to support them, from the pre-agent generation of no-code testing. That buyer profile is distinct from engineering-led teams, and it rarely overlaps with teams that ship using AI coding agents.

Whatever the buyer, the operating model is the same: tests are created and stored in the vendor's console, run on the vendor's runners, and leave only under paid-customer agreements.

When to Choose Shiplight

Shiplight fits when the mechanisms line up:

• Your coding agent should author the tests. Shiplight connects to Claude Code, Cursor, Codex, and more; the agent verifies its own work in a real browser during development and writes the regression test.
• Tests must live in your repo. YAML test files sit alongside your code, are version-controlled, produce clean diffs, and are reviewable in PRs.
• Heals must be reviewable. Intent-level self-healing surfaces as PR diffs, not silent rewrites in a vendor console.
• You need local runs with no account. npx shiplight test runs the suite on your machine or in your CI with no vendor runner in the loop.
• You need enterprise controls. SOC 2, VPC deployment, RBAC, hosted runners.

And where Shiplight does not fit: native mobile or desktop testing, or teams whose existing Playwright investment already works well.

Final Verdict

The choice reduces to mechanisms, not adjectives.

If tests must live in your repo, your coding agent is the author, heals must arrive as reviewable PR diffs, and you want a runtime built on Playwright with local runs with no account, that combination is what Shiplight ships. Shiplight's scope is honest too: web only, and not the right call for teams whose Playwright suite already works.

testRigor's design center is different: non-technical QA staff authoring tests in a vendor console, a model from the pre-agent era of no-code testing, with the suite living on the vendor's runners and leaving only under paid-customer agreements. Weighing more tools? Our roundup of testRigor alternatives covers the wider field.

Book a demo to see the agent-native loop end to end.

Get Started

• Try Shiplight: free, no account needed
• Book a demo
• YAML Test Format
• Best AI Testing Tools in 2026
• Documentation

References: Playwright Documentation, SOC 2 standard, Google Testing Blog

Frequently Asked Questions