Skip to main content

Diagnose a failed generation

A feature stuck at failed almost always means the AI couldn't make the Gherkin's scenarios pass against the real site. The transcript tells you why.

  1. Confirm what happened

    curl -H "Authorization: Bearer $TESTVIBE_API_KEY" \
    "$TESTVIBE_SERVER/api/v1/ops/projects/$PROJECT/features/12/generation"

    get_generation_status shows which section concluded failure and when.

  2. Read the transcript

    get_generation_log returns the per-section log — what the agent explored, what it generated, and what verification said. Look at the section's statusReason and final log lines first.

  3. Fix the cause

    The common failure modes, in order of frequency:

    Symptom in the logLikely causeFix
    Couldn't find a page or element the scenario namesGherkin describes UI that doesn't exist (yet) or names it differentlyReword the scenario to match the real UI, or deploy the feature first.
    Stuck at a login wallThe flow needs credentials the agent doesn't havePut test credentials in the scenario (When I sign in as "user@example.com" / "secret").
    Wrong site exploredProject base URL points elsewhere (staging moved?)Pass siteUrl on the next generate, or fix the project URL in the app.
    Verification flaky/timeoutsSlow environmentOften passes on retry; consider a calmer test environment.
  4. Update and retry

    testvibe features update 12 --file login.feature
    testvibe generate 12 --watch

    update_feature then generate_feature — each retry is a fresh generation (and fresh credits), so fix the Gherkin before re-rolling.

info

Doing this from an MCP client? The assistant can run this whole loop itself — see Your first test from Claude Code.