Yes, AI test automation tools can run on-premise or in a private cloud, but only some vendors offer it, and the term hides three very different architectures.
The three are vendor-hosted SaaS with a private tenant, a full deployment inside your own VPC, and local-first tools where test execution never depended on the vendor's cloud in the first place.
Shiplight sits in the last two of those: local execution needs no account at all, tests run on your own CI runners on any plan, you can point it at your own model provider keys, and private cloud and VPC deployment are available on Enterprise.
Before you shortlist tools, you need to know which architecture a vendor actually means, because the security, cost, and maintenance profiles are not interchangeable.
This guide breaks down the deployment models available for AI-powered test automation, what each one actually keeps inside your network, the questions that separate real private deployments from marketing language, and the trade-offs vendors are less eager to discuss.
Why deployment model matters more for AI testing than it did for traditional automation
Traditional test automation was easy to self-host. A test framework was a library in your repo; the browser grid was infrastructure you already ran. The vendor question barely existed.
AI test automation changed the shape of the product. Most AI-native testing tools now involve some combination of:
- A model call. Test generation, self-healing, and visual analysis usually mean sending page content to a large language model or vision model.
- A cloud execution grid. Many tools run your tests on vendor-managed browsers.
- A vendor-side test store. Low-code and no-code platforms typically keep the tests themselves in the vendor's database, not in your repo.
- Result and artifact storage. Screenshots, videos, DOM snapshots, and logs from every run.
Each of these is a place where your application's data can leave your environment. So "can it run on-premise" is really four questions: where do tests execute, where do tests live, where do artifacts go, and where does the AI inference happen. A vendor can answer "private cloud" to one of these and "our multi-tenant SaaS" to the other three.
The four deployment models, compared
| Deployment model | Where tests execute | Where tests are stored | AI inference | Typical ops burden | Who it fits |
|---|---|---|---|---|---|
| Multi-tenant SaaS | Vendor cloud | Vendor cloud | Vendor cloud | None | Teams testing public or staging apps with no data restrictions |
| Single-tenant / private SaaS | Dedicated vendor instance | Dedicated vendor instance | Vendor cloud | None | Teams that need tenant isolation but accept vendor hosting |
| VPC / private cloud deployment | Your cloud account | Your cloud account | Varies: in-VPC or egress to a model API | Medium to high | Enterprises with data residency or network isolation requirements |
| Local-first / repo-based | Your machines and your CI | Your git repo | Via your own tooling or configurable endpoints | Low | Engineering teams that treat tests as code |
| Shiplight | Your CI or ours, your choice on any plan | Your git repo, as readable YAML | Your own provider keys, or our proxy | Low | Teams that want the choice rather than one fixed answer |
A few things this table understates:
Single-tenant is not on-premise. A dedicated instance in the vendor's cloud isolates you from other customers, which helps with noisy-neighbor and tenancy concerns. It does not keep your data inside your network boundary. Security teams that require "no application data leaves our environment" will reject it.
VPC deployments vary in completeness. Some vendors ship the full product into your virtual private cloud, including execution and storage. Others deploy only the browser runners into your VPC while control-plane traffic, test definitions, and AI calls still flow to their cloud. Both get sold as "VPC deployment."
Local-first tools sidestep the question for authoring and execution. If tests are plain files in your repository and run as a process on your own machines and CI, the core workflow was never in the vendor's cloud. The remaining question is what the AI layer transmits during generation and healing, which is narrower and easier to audit.
What "on-premise" has to cover: the four data paths
When a security team evaluates an AI testing tool, these are the four paths to trace.
Path 1: Test execution
Where does the browser actually run when a test executes? If the answer is a vendor grid, your application, including any test data you type into it, is being driven from outside your network.
For internal apps behind a VPN this often fails at a practical level too: the vendor's browsers cannot reach the app at all without a tunnel or agent, which is itself a new piece of attack surface to review.
Tools that execute locally or in your CI avoid both problems. The browser runs where your code already runs, against whatever environments your network can already reach.
Path 2: Test definitions
Where do the tests live? Vendor-database storage means your test suite, which encodes your product's workflows, URLs, and often credentials or credential references, sits in someone else's system, and leaving the vendor means exporting or rewriting it. Repo-based storage means tests are versioned files you control, reviewable in pull requests and portable by default.
Path 3: Run artifacts
Screenshots and DOM snapshots are the most sensitive artifacts an AI testing tool produces, because they can capture real interface states: names, account numbers, internal dashboards. Ask where artifacts are stored, for how long, and whether storage location is configurable. A tool can execute locally and still upload every screenshot to its cloud for reporting.
Path 4: AI inference
This is the path teams most often miss. Self-healing and test generation typically send page structure, and sometimes screenshots, to a model. Relevant questions: which model provider, what exactly is in the payload, is it retained or used for training, and can you route it through your own model endpoint or API keys instead of the vendor's.
For a deeper treatment of this path, see our guide to AI testing data security and SOC 2.
Questions to ask a vendor before believing "private cloud"
Use these verbatim in an evaluation call. Vague answers are answers.
- In your VPC deployment, which components run in our account and which still call your cloud? Ask for an architecture diagram.
- Can the product execute tests with zero network egress from our environment? If not, list every egress destination.
- Where do AI inference calls go, what is in the payload, and can we supply our own model API keys or endpoint?
- Where are test definitions stored, and in what format do we get them back if we leave?
- Where are screenshots and DOM snapshots stored, and is retention configurable?
- Is the on-premise or VPC version the same build as the SaaS version, and how far behind SaaS do its releases lag?
- What is our operational responsibility: upgrades, scaling, monitoring, incident response?
- Which compliance attestations cover the deployment we are buying, not just your SaaS? SOC 2 reports are typically scoped to specific services.
The honest trade-offs
Private deployment is not free, and vendors who offer it will privately agree with most of this list.
- Feature lag is normal. Self-hosted and VPC builds usually trail the SaaS release train. If a vendor ships weekly to SaaS and quarterly to VPC, you are buying a product several months old.
- You inherit ops. Inside your VPC, capacity planning, upgrades, and uptime for the testing stack become at least partly your job. That is engineering time with a real cost, which belongs in any cost comparison of AI test automation.
- The AI layer may still egress. Very few vendors run frontier models inside customer networks. In most "private" deployments, model inference either leaves the VPC to a model API or drops to a smaller local model with weaker results. Get the specific answer.
- Pricing moves upmarket. VPC and on-premise options are usually gated behind enterprise, quote-only tiers.
- SaaS is genuinely fine for many teams. If you test a marketing site or a staging environment seeded with synthetic data, a multi-tenant SaaS tool with a clean SOC 2 report is a defensible choice. Private deployment is a requirement for regulated data and internal apps, not a universal best practice. Teams in that situation should start with our guide to AI testing for regulated industries.
Where Shiplight fits
Shiplight is local-first by architecture rather than by an enterprise add-on, which changes what "deployment" means.
Tests are plain YAML files that live in your git repository, and they run locally or in your existing CI with npx shiplight test. The browser execution happens on your machines, against environments your network can already reach.
The MCP plugin that gives coding agents browser automation and test authoring runs locally and requires no Shiplight account or token, so the authoring loop works before any procurement conversation happens.
For enterprise teams that want managed execution, Shiplight offers hosted CI runners plus private cloud and VPC deployment, backed by SOC 2, a dedicated customer success manager. The practical result is a spectrum: start fully local with nothing leaving your environment, add hosted runners where convenient, or run the platform inside your own VPC where policy requires it.
Where Shiplight is not the right fit: it is web-focused, so native mobile or desktop apps need a different tool, and teams that want a fully managed, no-engineering-involvement QA service are better served by a managed QA provider than by any self-hosted product.
Frequently Asked Questions
Can AI test automation tools run on-premise or in a private cloud?
Yes. Several vendors offer private cloud or VPC deployments on enterprise plans, and local-first tools run execution on your own machines by default. The critical detail is scope: confirm whether execution, storage, artifacts, and AI inference all stay in your environment, or only some.
Many "private cloud" offerings still route test definitions and AI calls to the vendor, so demand an architecture diagram first.
What testing tools can deploy inside your own VPC?
Three categories can. Open-source frameworks like Playwright and Selenium are libraries you host entirely yourself. Enterprise tiers of commercial AI platforms ship runners or the full product into your cloud account, usually quote-only. Local-first tools such as Shiplight keep tests as files in your repo running in your own CI, with VPC deployment for managed components.
Is an on-premise AI testing deployment more secure than SaaS?
Only if you operate it well. On-premise moves data risk inside your boundary but also moves patching, access control, and monitoring onto your team; a neglected self-hosted deployment can be weaker than a vendor's audited SaaS. The stronger question is what actually leaves your environment under each model.
A local-first tool with minimal, documented egress can satisfy a review with less operational burden.
Does the AI model itself run inside my network in a private deployment?
Usually not. Most AI testing products call an external model API for generation, healing, and visual analysis even when everything else runs in your VPC. Some vendors let you bring your own keys or route inference through your own cloud endpoints, keeping it under your existing agreements. Fully local inference exists but usually means smaller models and weaker results.
Can I run AI-generated tests without any vendor cloud at all?
Yes, if the tool stores tests as portable code. Shiplight tests, for example, are YAML files in your repository that run with npx shiplight test on your own hardware, and the local MCP authoring needs no account.
Tools that store tests in a vendor database can't: no vendor cloud, no test suite. Make portable test format a hard requirement in your evaluation criteria.
What should regulated companies choose: on-premise, VPC, or local-first?
Regulated teams usually need auditability and data residency more than a specific hosting model. VPC deployment satisfies residency; a local-first, repo-based tool adds a reviewable audit trail, since every test and change is a git commit.
Many finance and healthcare teams combine the two: local-first execution day-to-day, private cloud components where managed infrastructure helps. See AI testing in regulated industries for the full list.
