Diagnose a failed generation

A generation came back failed — here's how to find out why and fix it.

A feature stuck at failed almost always means the AI couldn't make the Gherkin's scenarios pass against the real site. The transcript tells you why.

1

Confirm what happened

curl -H "Authorization: Bearer $TESTVIBE_API_KEY" \
  "$TESTVIBE_SERVER/api/v1/ops/projects/$PROJECT/features/12/generation"

get_generation_status shows which section concluded failure and when.

2

Read the transcript

get_generation_log returns the per-section log — what the agent explored, what it generated, and what verification said. Look at the section's statusReason and final log lines first.

3

Fix the cause

The common failure modes, in order of frequency:

Symptom in the log	Likely cause	Fix
Couldn't find a page or element the scenario names	Gherkin describes UI that doesn't exist (yet) or names it differently	Reword the scenario to match the real UI, or deploy the feature first.
Stuck at a login wall	The flow needs credentials the agent doesn't have	Put test credentials in the scenario (When I sign in as "[e-mail]" / "secret").
Wrong site explored	Project base URL points elsewhere (staging moved?)	Pass siteUrl on the next generate, or fix the project URL in the app.
Verification flaky/timeouts	Slow environment	Often passes on retry; consider a calmer test environment.

4

Update and retry

testvibe features update 12 --file login.feature
testvibe generate 12 --watch

update_feature then generate_feature — each retry is a fresh generation (and fresh credits), so fix the Gherkin before re-rolling.

Doing this from an MCP client? The assistant can run this whole loop itself — see Your first test from Claude Code.