Run TestVibe in CI

testvibe run --all --wait runs every generated feature and exits 1 when any of them fails — that's the whole integration. Point it at the environment you just deployed and let the exit code gate the pipeline.

GitHub Actions example

.github/workflows/e2e.yml

e2e:
  runs-on: ubuntu-latest
  needs: deploy
  steps:
    - uses: actions/setup-node@v4
      with: { node-version: 20 }
    - run: npm install -g testvibe   # or, without npm: curl -fsSL "$TESTVIBE_SERVER/api/v1/install.sh" | sh
    - run: testvibe run --all --url "$TARGET_URL" --wait --timeout 1800
      env:
        TARGET_URL: ${{ vars.TEST_TARGET_URL }}
        TESTVIBE_SERVER: https://testvibe.app
        TESTVIBE_API_KEY: ${{ secrets.TESTVIBE_API_KEY }}
        TESTVIBE_PROJECT: ${{ vars.TESTVIBE_PROJECT }}

The same shape works anywhere — only the env-var syntax changes. --timeout 1800 caps the wait at 30 minutes for the whole step (one budget, not 30 minutes per feature) so a wedged run can't hang the job; leave it off and the ceiling is 5400 s. See the wait contract.

Key the step off the exit code

--wait / --watch report four distinct outcomes, and the fourth is the one that matters for a pipeline: "we stopped waiting" is not "your tests failed."

Exit code	Meaning	What the pipeline should do
0	Everything finished and passed.	Continue.
1	It finished and something actually failed.	Fail the build — this is your gate.
2	A generation parked to ask a question (generate --watch only).	Needs a human; don't retry it unattended. Answer it, or decide up front with --gherkin-changes accept|deny.
3	The --timeout ceiling elapsed, or contact with the server was lost.	No verdict. The work may still be running. Stop the pipeline if you like, but report it as infrastructure — then raise --timeout or open the run in TestVibe.

Collapsing 3 onto 1 is the failure mode to avoid: once a red build can mean "the ceiling elapsed," nobody trusts a red build.

Gate on real failures, flag no-verdict separately

set +e
testvibe run --all --url "$TARGET_URL" --wait --timeout 1800
code=$?
set -e

case $code in
  0) echo "All features passed." ;;
  1) echo "::error::TestVibe reported failing tests."; exit 1 ;;
  3) echo "::warning::No verdict — the wait ended before the runs did. Check them in TestVibe."; exit 1 ;;
  *) echo "::error::testvibe exited $code."; exit 1 ;;
esac

Both 1 and 3 stop the pipeline here — a run with no verdict is not evidence that the deploy is good — but the log says which one happened, so nobody goes hunting for an application bug that doesn't exist.

Practical notes

• Use env vars, not testvibe login, in CI — TESTVIBE_API_KEY / TESTVIBE_SERVER / TESTVIBE_PROJECT (config precedence). Create a dedicated key for CI so it can be revoked independently.
• --url targets the deployment under test (a preview URL, staging) without touching the project's configured URL.
• --all runs the whole suite (one cloud run per generated feature, summarized at the end); name features instead (testvibe run Login Checkout --wait) or keep a single broad "smoke" feature when you want a faster gate.
• Tests run in TestVibe's sandboxes, not your runner — the CI job is just dispatch + poll, so it's cheap; the target must be reachable from the execution browser, directly or through the project's configured tunnel.
• Artifacts on failure: testvibe runs artifacts <runId> prints signed trace/screenshot links; treat them as credentials and avoid public job logs; the run is also waiting in the TestVibe app.
• testvibe verify --git is the narrower gate. Rather than the whole suite, it maps uncommitted tracked changes against HEAD by default (or changes against an explicit --git <ref>) to the tests that cover them and runs only those — same wait contract, same exit codes.

info

Want the run to create or update tests too? That's a generation concern, not a CI one — keep generate interactive (it consumes credits and takes minutes) and let CI only run what's already generated. If you do run generate --watch unattended, remember it exits 2 when the generation stops to ask a question — that needs a human, and it is not a failure.