Diagnose a failed generation

A failed generation may be caused by the test contract, target application, missing environment values, model/provider failure, or capacity. Read the failure reason before deciding what to change.

1. Confirm what happened

   curl -H "Authorization: Bearer $TESTVIBE_API_KEY" \
     "$TESTVIBE_SERVER/api/v1/ops/projects/$PROJECT/features/12/generation"

   get_generation_status shows which section concluded failure and when.

2. Read the transcript

   get_generation_log returns the per-section log — what the agent explored, what it generated, and what verification said. Look at the section's statusReason and final log lines first.

3. Fix the cause

   Examples to investigate:

   Symptom in the log	Likely cause	Fix
   Couldn't find a page or element the scenario names	Gherkin describes UI that doesn't exist (yet) or names it differently	Reword the scenario to match the real UI, or deploy the feature first.
   Stuck at a login wall	The flow needs credentials the agent doesn't have	Store credentials as variables/secrets, then reference {{var:LOGIN_USERNAME}} and {{secret:LOGIN_PASSWORD}} in the scenario.
   Wrong site explored	Project base URL points elsewhere (staging moved?)	Pass siteUrl on the next generate, or fix the project URL in the app.
   Verification flaky/timeouts	Slow environment	Inspect timing and environment evidence; fix the cause before paying for another generation.

4. Update and retry

   testvibe features update 12 --file login.feature
   testvibe generate 12 --watch

   update_feature then generate_feature — each retry is a fresh generation (and fresh credits), so fix the Gherkin before re-rolling.

   --watch exits 1 if the retry fails again, so a script can tell a fixed Gherkin from one that still doesn't generate — and 2 if it stopped to ask you a question instead. Full table: the CLI wait contract.

info

Doing this from an MCP client? The assistant can run this whole loop itself — see Your first test from Claude Code.

Start with isTerminal, terminalReason, and root statusReason / logLines. If awaitingInput is true, answer the existing question instead of regenerating: testvibe generate answer 12 --option "<exact option>" --watch. Use testvibe generate log 12 --json for the full transcript. Capacity or credential problems may need configuration changes rather than a Gherkin edit.