Skip to main content

Troubleshoot Failed Generation

Generation can fail because of setup, project access, session startup, site behavior, or unclear test intent.

Start with the failure details in TestVibe. They usually tell you whether to fix configuration, the target site, or the feature instructions — a generation that was cut off (timeout, credits, or an interrupted session) says so plainly rather than leaving the run stuck with no explanation.

Common Causes​

CauseWhat to check
Project permission issueConfirm you have access to the project and its files.
Session did not startRetry; repeated startup failures are usually temporary service issues.
Site unreachableConfirm the target site URL opens from the cloud test session, or use a tunnel device for private apps.
Login or test data missingAdd secrets, prerequisites, or clearer setup steps.
Vague instructionsImprove Gherkin with visible actions and outcomes.
App behavior changedUpdate the feature to match the current UI.
Heavy feature hit the generation timeoutRaise Generation timeout (minutes) in Settings > AI Model, up to 120.

First Checks​

  1. Open the failed generation details.
  2. Read the summary or failure message.
  3. Check whether any output was generated.
  4. Confirm the target site URL.
  5. Confirm the reasoning effort and generation timeout in Settings > AI Model.
  6. Confirm the cloud test session can start.
  7. Decide whether to edit settings, improve Gherkin, or retry.

Out Of Credits​

Generation runs on TestVibe's metered AI service. If it stops because the credit balance ran out, top up or change the plan in Settings > Billing. See Billing. A credit-walled generation now shows an Out of credits message with a Buy credits button that opens billing directly.

If a test secret is reported missing (a {{secret:NAME}} token), add the value in Settings > Variables & Secrets. See Add API keys and secrets.

Site Access Problems​

If the agent or generated test cannot reach the site:

CheckWhat to do
URLMake sure the URL is complete and opens in a browser.
LocalhostAvoid localhost; the cloud test session cannot reach your machine. Use a tunnel device instead.
Private networkUse a tunnel device with network access to the app.
AuthenticationProvide test credentials through secrets ({{secret:NAME}}) or clear setup steps.
Environment availabilityConfirm staging or QA is running before generation starts.

See Runners for execution environment setup.

Project Or Run Problems​

If the run fails before meaningful generation starts:

SymptomPossible fix
Cannot write filesCheck your access to the project.
Run stays queuedGive TestVibe a moment to start a cloud test session, then retry.
Secrets unavailableConfirm secrets are created in Settings > Variables & Secrets.
Generated path missingReview the run failure details.

Feature Clarity Problems​

If generation completes but the output is poor, treat it as an instruction problem.

Improve the feature by adding:

  • a clear starting page
  • visible UI labels
  • required test data
  • expected messages or page changes
  • one focused behavior per scenario
  • stronger Then assertions

Then regenerate. See Write good test instructions.

Retry Or Regenerate?​

SituationBest next step
Temporary session or network issueRetry after the environment is healthy.
Missing key or permissionFix setup, then regenerate.
Vague or wrong outputEdit Gherkin, then regenerate.
Partial output existsReview it before deciding what to keep.
App is brokenFix the application or target environment before retrying.