Skip to main content

TestVibe 0.4.6

All release notes · Self-hosted upgrade guide

Tagged but never published. These changes shipped in 0.4.7; there are no published 0.4.6 downloads.

A generation that stops to ask you a question now carries on where it left off, a failing test can retry before the whole run goes red, and the encryption key for your stored secrets looks after itself. Containers and Helm chart — no appliance image yet (below). Docker and Kubernetes users upgrade from 0.4.5.

  • A parked generation resumes instead of starting over. Its workspace and the agent's conversation are packed away when it stops to ask you something and restored when you answer, on all three generation engines. The Agent Log survives the pause too, so answering no longer blanks the drawer. Answering from the command line or the API previously resumed nothing at all, and testvibe generate --watch waited forever on a generation that was waiting on you. A resumed generation also keeps the model and provider it started with, instead of switching to whatever the project points at now.
  • A failed test can retry itself before the whole run goes red. A test that only passed on a retry is reported as flaky — never as clean.
  • Import a Jira story or a GitHub issue straight into a feature by pasting its key, number or link. You can also file an issue about a single failing scenario from the scenario drawer, instead of only about the whole run. Issues TestVibe filed in Jira listed as #0, imported under a GitHub banner, and linked back to the run with a dead URL — all three are fixed.
  • Nobody could connect Jira — or any other tracker — from Settings. The Connectors panel never received its data, so the whole panel was unusable. It works now.
  • Settings groups your AI models by the company that supplies them and says which model does each job. The command line can choose the model for a generation too, including auto.
  • Administration console. The Models page is a list of the providers you connect from — Anthropic, OpenAI, OpenCode, OpenRouter or a custom endpoint — and the model list now answers which model is current, how big it is and what it costs, sortable and ten to a page. The environment switcher moved into the left nav, and you can set an API key on a provider that already exists.
  • Security. The LLM key proxy now speaks every provider's dialect and takes its credential from the database, so no generation sandbox holds a real API key; it is still behind TV_LLM_KEY_PROXY and still off by default. The generation sandbox no longer sends a credential it cannot vouch for, and a saved workspace never contains one — an allowlist plus a second, independent exclusion gate, with snapshots expiring on a retention sweep.

TESTVIBE_SECRET_PROTECTION_KEY is no longer required. The app generates its own key into its own database the first time it needs one, and the administration console reads it back over the same connection — so the two can no longer disagree, and there is nothing to set or to keep a copy of beyond the database backup you already take. Compose no longer refuses to start without it. If you already have one, leave it exactly where it is: an explicit key still wins over the automatic one, and removing it makes everything already encrypted under it permanently unreadable. New installs should simply not set one. TV_ADMIN_PASSWORD is still required and still fails closed.

Nothing about AI routing moves by itself. V0276 only widens which provider kinds an administrator may register. It registers nothing and seeds no rows: an install that upgrades and configures nothing keeps its existing models, keys and generation engine exactly as they were.

Parked-generation snapshots are transient — swept automatically once a park can no longer be answered, and excluded from the per-account storage policy. Expect a modest, self-limiting increase in run-bundle usage. TV_GEN_SNAPSHOT_DISABLE=1 turns the feature off.

Optional, on-prem: with TV_LLM_KEY_PROXY=1 and TESTVIBE_PUBLIC_BASE_URL set to an address the sandbox can reach, no provider API key is placed inside the generation sandbox at all. It is off by default and applies no migration.

Upgrading: Docker — pull the 0.4.6 tag and docker compose up -d; the one-shot migrate service applies the migrations before web starts. Take the updated compose file and .env.example from this release, because TESTVIBE_SECRET_PROTECTION_KEY is now passed as ${VAR:-} rather than ${VAR:?}. Your existing .env needs no edit. Kubernetes — helm upgrade to chart 0.4.6; its Secret is still annotated to survive a helm uninstall. Keep passing the secrets.protectionKey you already pass, and do not remove it — an upgrade that drops it can remove the active key, and an upgrade that drops it leaves everything encrypted under it unreadable.

warning

Known defect in chart 0.4.6: a fresh Kubernetes install cannot follow the documentation. The chart's values document secrets.protectionKey as "leave this empty", but the chart still refuses to render without it, so helm install stops with secrets.protectionKey is required. Pass a value you then keep forever, or supply secrets.existingSecret (which skips the check), or install chart 0.4.7, where the requirement is removed and the value is optional as documented. Compose is unaffected — it was correctly changed to ${VAR:-} in this release.

Appliance users: there is no 0.4.6 VHDX or OVA at the time of writing. The appliance build was repaired in this release — the workflow could not be parsed at all, which is why no image has been produced since 0.4.3 and why 0.4.5 shipped containers and chart only. Until an image is published, stay on the 0.4.3 appliance. When one is, attach the new OS disk and keep your existing data disk: the database, artifacts, install identity and license all live on disk 2.

Schema migrations: this release applies V0273 (a single-use claim record for a parked generation's saved workspace), V0274 (the self-resolving encryption key), V0275 (a "first published by the provider" column on tv_llm_model) and V0276 (a fourth provider kind, opencode). V0275 and V0276 are pure widenings that touch no existing row, and V0273/V0274 create new tables — but take a database backup first regardless. They run automatically and are one-way.