E2E Coverage
How GitAuto tells E2E from unit coverage
Key Requirements
- Coverage report must be in LCOV format
- GitHub Actions: upload it as an artifact named exactly
e2e-coverage - CircleCI: store it at path
e2e-coverage/lcov.info - Keep it separate from the artifact your unit coverage already publishes
An LCOV file records which lines ran, not what kind of test ran them, so the published name is what distinguishes the two. Anything published under the E2E name above is recorded as end-to-end coverage; everything else is recorded as unit coverage.
The match is exact, not a search for "e2e" anywhere in the name. A report is only labeled end-to-end when you have deliberately published it under that name, so a pipeline that happens to include those letters is never mislabeled.
Both suites can report from the same CI run. GitAuto keeps a separate coverage record per test type for every file, so unit and E2E numbers never merge into a single figure.
Making browser tests emit coverage
Every browser-based E2E framework can produce standard LCOV. Coverage does not come from the test runner, so the framework you use does not limit it. It comes from instrumenting the application build with babel-plugin-istanbul, which makes the running app expose a window.__coverage__ object. Your tests read that object out of the browser and write it to disk.
Each framework reads it through its own JavaScript evaluation mechanism:
- Playwright: collect it in a fixture after each test
- Cypress: add
@cypress/code-coverage, which wires this up for you - Puppeteer: read it with
page.evaluate - WebdriverIO / Selenium: read it with
executeScript - TestCafe / Nightwatch: read it through the client-function or
.executeAPI
Collecting it in Playwright looks like this. The other frameworks differ only in how they reach into the page:
import { test as base, expect } from "@playwright/test";
import { mkdir, writeFile } from "fs/promises";
import { randomUUID } from "crypto";
// Writes one istanbul JSON file per test; nyc merges them into lcov afterwards
export const test = base.extend({
page: async ({ page }, use) => {
await use(page);
const coverage = await page.evaluate(() => window.__coverage__);
if (!coverage) return;
await mkdir(".nyc_output", { recursive: true });
await writeFile(`.nyc_output/${randomUUID()}.json`, JSON.stringify(coverage));
},
});
export { expect };
Choosing the target URL
End-to-end tests drive a real browser against a running copy of your app, so GitAuto needs an address that the CI runner can open. The runner is a fresh machine in the cloud: it is not your laptop, and nothing is listening on its own localhost unless the job starts a server itself. A localhost address is therefore rejected when you save it.
If your app is already deployed somewhere
Point at that deployment, and prefer a staging or development environment over production. Browser tests sign up, submit forms and complete checkouts, so running them against production creates real accounts, sends real email and can charge real cards.
You may not need to set anything at all. If your repository already declares a base URL, GitAuto reads it and uses that. It looks in .env.development, .env.stage, .env and cypress.env.json, in that order, for CYPRESS_BASE_URL, PLAYWRIGHT_BASE_URL, baseUrl or BASE_URL. Failing that it reads the most recent GitHub Deployment for the repository. Set the dashboard field only to override that order, for example to pin staging when the repo defaults to dev.
If the value in your env file interpolates a variable, such as a version segment in the path, put the resolved URL in the dashboard rather than the template.
If your app has no deployed environment
Have the workflow build and start the app on the runner, then point the browser at it. This is the better option when it is available: the tests exercise the code on the branch rather than whatever happens to be deployed, and it needs no hosting and no secrets. Playwright supports it directly through the webServer option in its config, which starts the command and waits for the URL before the run begins.
Leave the dashboard field blank in that case. The URL belongs in the workflow and the Playwright config, not in the dashboard, because it only exists for the lifetime of the job.
Publishing the report
Point the browser at a deployed environment rather than a local dev server, since the app has to be reachable for the whole run.
GitHub Actions
name: E2E Coverage
# Run on the target branch so coverage history tracks the deployed app
on:
push:
branches:
- main
workflow_dispatch:
jobs:
e2e-coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v6
with:
node-version: 22
cache: npm
- run: npm ci
- run: npx playwright install --with-deps chromium
# Instrument the app so the browser exposes window.__coverage__
- name: Build instrumented app
run: npm run build
env:
BABEL_ENV: instrumented
- name: Run E2E tests
run: npx playwright test
env:
PLAYWRIGHT_BASE_URL: ${{ vars.E2E_BASE_URL }}
# Convert the collected istanbul JSON into LCOV
- name: Generate lcov report
run: npx nyc report --reporter=lcovonly --report-dir=coverage
- uses: actions/upload-artifact@v5
with:
name: e2e-coverage
path: coverage/lcov.info
CircleCI
CircleCI has no artifact names, so the stored path carries the same meaning. Copy the LCOV file into an e2e-coverage directory and store that.
jobs:
e2e-coverage:
docker:
- image: cimg/node:22.11-browsers
steps:
- checkout
- run: npm ci
- run: npx playwright install --with-deps chromium
# Instrument the app so the browser exposes window.__coverage__
- run:
name: Build instrumented app
command: npm run build
environment:
BABEL_ENV: instrumented
- run:
name: Run E2E tests
command: npx playwright test
- run:
name: Generate lcov report
command: npx nyc report --reporter=lcovonly --report-dir=coverage
# The stored path is what marks this report as end-to-end
- run: mkdir -p e2e-coverage && cp coverage/lcov.info e2e-coverage/lcov.info
- store_artifacts:
path: e2e-coverage/lcov.info
workflows:
build:
jobs:
- e2e-coverage
Letting GitAuto set this up for you
You do not have to write any of this by hand. GitAuto detects which E2E framework your repository already uses, or sets up Playwright if there is none, and opens a pull request that adds the instrumentation and the pipeline above. Review it like any other pull request before merging.
Not Sure Your E2E Tests Can Report Coverage?
Most teams assume end-to-end tests can't produce coverage at all, so this part usually gets skipped. They can, and we're happy to look at your setup with you.
Contact us and let's get your E2E coverage on the board!