The Two-Speed E2E Testing Strategy: Fast by Default, Adaptive When the UI Changes

End-to-end testing usually breaks down in one of two ways.

In the first, tests are written “the right way” with stable selectors and careful waits, but they become a tax. Every UI refactor creates a backlog of broken tests, and the team quietly starts ignoring failures. In the second, teams try to move faster with record-and-replay or brittle scripts, and flakiness becomes the norm.

Shiplight AI takes a different approach: run tests as deterministic Playwright actions when you can, and fall back to intent-aware AI execution when you must. That combination turns UI change from a recurring fire drill into a recoverable event, without giving up speed in CI.

Below is a practical strategy you can adopt immediately, whether you are starting from scratch or modernizing an existing Playwright suite.

The core idea: treat locators like a cache, not a contract

Traditional automation treats selectors as the contract. If the selector breaks, the test fails, and a human fixes it. That works until your product velocity increases, your design system evolves, or your frontend stack changes how it renders DOM.

Shiplight’s model is closer to how resilient systems are built:

1. Write the test in human-readable intent.
2. Enrich steps with Playwright locators for fast replay.
3. When the UI changes, recover by re-resolving the intent.
4. Optionally update the cached locator after a successful recovery in Shiplight Cloud.

That “locator cache” framing is not a metaphor. In Shiplight’s YAML test flows, you can run natural language steps, you can run action entities with explicit Playwright locators, and you can combine both.

How Shiplight implements two-speed execution

Shiplight runs on top of Playwright, with an AI layer that can interpret intent at runtime. In practice, you get two execution modes:

1) Fast Mode for performance-critical regression

Fast Mode uses cached, pre-generated Playwright actions and fixed selectors. It is optimized for quick, repeatable runs.

2) AI Mode for adaptability

AI Mode evaluates the action description against the current browser state, dynamically finds the right element, and adapts when IDs, classes, or layout change. It trades some speed for resilience.

Auto-healing: the bridge between speed and stability

Shiplight can automatically recover from failures by retrying a failed Fast Mode action in AI Mode. In cloud execution, if AI Mode succeeds, the run continues without permanently modifying the test configuration.

This matters because it changes the economics of maintenance. You can keep your suite optimized for CI while still surviving real-world UI churn.

A practical authoring pattern for modern teams

A strong E2E suite is not just “more tests.” It is a set of workflows that stay readable, reviewable, and resilient as the app changes. Here is a pattern that consistently works.

Step 1: Start with intent in YAML

Shiplight tests are written in YAML with natural language steps, including VERIFY: assertions for AI-powered verification.

A minimal flow looks like this:

goal: Verify user can reset their password
url: https://app.example.com/login

statements:
- Click "Forgot password"
- Enter "qa.user@example.com" in the email field
- Click "Send reset link"
- "VERIFY: A confirmation message is visible"


This is the right level of abstraction for collaboration. Product, design, QA, and engineering can all review the intent without parsing framework-specific code.

Step 2: Enrich high-value steps for speed

Once the flow is correct, convert the most frequently executed actions into deterministic steps with explicit locators, while keeping verification intent clear. Shiplight’s documentation calls out that natural language steps can take longer, while locator-backed actions replay quickly.

This is where two-speed testing starts paying off:

• Your suite stays fast for everyday regressions.
• Your suite stays recoverable when the UI moves.

Step 3: Design for UI change, not against it

When the inevitable happens (a button is renamed, a component is replaced, a layout shifts), you want graceful degradation:

• AI fallback to resolve intent
• Clear failure artifacts when the behavior truly changed

Shiplight supports auto-healing by switching to AI Mode when Fast Mode actions fail, both in the editor and during cloud execution.

Debugging that produces decisions, not just logs

Most teams do not struggle to run E2E tests. They struggle to interpret failures quickly enough to keep shipping.

Shiplight’s cloud debugging workflow includes real-time visibility, screenshots, and step-level context. The Live View panel and screenshot gallery are designed to shorten the “what happened?” loop.

On top of that, Shiplight can generate AI summaries of failed test results, including root cause analysis, expected vs actual behavior, and recommendations. Summaries are cached after generation so subsequent views load instantly.

If you want a north star for E2E maturity, it is this:

• A failing test should be a high-signal quality event, not an investigation project.

Operationalizing the strategy: local-first and CI-native

Two-speed execution becomes even more valuable when it fits cleanly into daily engineering workflows.

Local development in the repo

Shiplight’s YAML flows are designed to be run locally with Playwright using the shiplightai CLI, and the docs emphasize “no lock-in” with the YAML format as an authoring layer.

For teams that live in their editor, Shiplight’s VS Code extension supports stepping through YAML statements, inspecting action entities inline, and iterating without switching browser tabs.

CI integration that matches how teams ship

Shiplight provides a GitHub Actions integration via ShiplightAI/github-action@v1, supporting suite execution and common patterns like preview deployments.

And for teams that want automated monitoring beyond PR gates, Shiplight Cloud supports suites and schedules that can run on recurring cadences (including cron-based schedules).

Where this approach is most valuable

Two-speed E2E testing is especially effective when:

• Your UI changes frequently (design system updates, rapid iteration, A/B tests)
• You need fast CI feedback, but cannot afford constant selector maintenance
• Multiple roles contribute to test coverage, not just specialists
• You want enterprise-grade readiness, including SOC 2 Type II compliance and deployment options like private cloud or VPC for stricter environments.

A simple way to evaluate Shiplight AI

If you are assessing whether this model fits your team, run a small pilot:

1. Pick one critical workflow with frequent UI movement.
2. Author it in intent-first YAML.
3. Enrich only the highest-frequency actions for Fast Mode speed.
4. Run it in CI, then introduce a controlled UI change and observe recovery behavior.
5. Measure what matters: time-to-diagnosis and maintenance hours avoided.

Shiplight is built to get teams up and running quickly, with minimal setup and a clear path from local testing to cloud execution.