Troubleshoot Failed Generation

Recover from failed AI generation by checking keys, permissions, site access, session setup, and feature clarity.

Generation can fail because of setup, project access, session startup, site behavior, or unclear test intent.

Start with the failure details in TestVibe. They usually tell you whether to fix configuration, the target site, or the feature instructions — a generation that was cut off (timeout, credits, or an interrupted session) says so plainly rather than leaving the run stuck with no explanation.

Common Causes

Cause	What to check
Project permission issue	Confirm you have access to the project and its files.
Session did not start	Retry; repeated startup failures are usually temporary service issues.
Site unreachable	Confirm the target site URL opens from the cloud test session, or use a tunnel device for private apps.
Login or test data missing	Add secrets, prerequisites, or clearer setup steps.
Vague instructions	Improve Gherkin with visible actions and outcomes.
App behavior changed	Update the feature to match the current UI.
Heavy feature hit the generation timeout	Raise Generation timeout (minutes) in Settings > AI Model, up to 120.

First Checks

1. Open the failed generation details.

2. Read the summary or failure message.

3. Check whether any output was generated.

4. Confirm the target site URL.

5. Confirm the reasoning effort and generation timeout in Settings > AI Model.

6. Confirm the cloud test session can start.

7. Decide whether to edit settings, improve Gherkin, or retry.

Out Of Credits

Generation runs on TestVibe's metered AI service. If it stops because the credit balance ran out, top up or change the plan in Settings > Billing. See Billing. A credit-walled generation now shows an Out of credits message with a Buy credits button that opens billing directly.

If a test secret is reported missing (a {{secret:NAME}} token), add the value in Settings > Variables & Secrets. See Add API keys and secrets.

Site Access Problems

If the agent or generated test cannot reach the site:

Check	What to do
URL	Make sure the URL is complete and opens in a browser.
Localhost	Avoid localhost; the cloud test session cannot reach your machine. Use a tunnel device instead.
Private network	Use a tunnel device with network access to the app.
Authentication	Provide test credentials through secrets ({{secret:NAME}}) or clear setup steps.
Environment availability	Confirm staging or QA is running before generation starts.

See Runners for execution environment setup.

Project Or Run Problems

If the run fails before meaningful generation starts:

Symptom	Possible fix
Cannot write files	Check your access to the project.
Run stays queued	Give TestVibe a moment to start a cloud test session, then retry.
Secrets unavailable	Confirm secrets are created in Settings > Variables & Secrets.
Generated path missing	Review the run failure details.

Feature Clarity Problems

If generation completes but the output is poor, treat it as an instruction problem.

Improve the feature by adding:

• a clear starting page

• visible UI labels

• required test data

• expected messages or page changes

• one focused behavior per scenario

• stronger Then assertions

Then regenerate. See Write good test instructions.

Retry Or Regenerate?

Situation	Best next step
Temporary session or network issue	Retry after the environment is healthy.
Missing key or permission	Fix setup, then regenerate.
Vague or wrong output	Edit Gherkin, then regenerate.
Partial output exists	Review it before deciding what to keep.
App is broken	Fix the application or target environment before retrying.

Related Help

• Review generation progress

• Regenerate a test

• Missing secrets

• Troubleshoot runners