What broke for other teams

The migration atlas pools anonymous verdict outcomes from teams that chose to share them: for each model switch, how many teams hit a regression and in which kind of case. Updated daily.

0migrations published

No migration has reached 5 teams yet. The atlas fills up as teams run verdicts with swapvera run --share. Until a migration has 5 different teams behind it, we only show how many have shared.

Ask before you run

Before spending a verdict, see what happened to teams with cases like yours. Only counts of your case kinds are sent (for example 25 JSON cases, 5 tool calls), never the cases.

Example of the output format, not real data:

$ swapvera predict
12 teams have migrated openai:gpt-4o -> openai:gpt-5.6.
42% hit at least one regression (95% interval 19% to 68%).

Risk for the kinds of cases in your suite:
  JSON output      your cases:   25   teams that saw regressions: 50% (24% to 76%, 10 teams)
  tool calls       your cases:    5   not enough teams yet

The atlas tells you where to look. Only a verdict on your own prompts tells you whether your app breaks.

What is shared, exactly

Nothing unless you pass --share or set share_anonymous_stats: true, and only with a license. The engine writes the exact request body to swapvera-atlas.json so you can read it. This is all of it:

{
  "from_model": "openai:gpt-4o",
  "to_model": "openai:gpt-5.6",
  "verdict": "review",
  "pref": 0.04,
  "det_change": -0.03,
  "judge_consistency": 0.91,
  "engine_version": "0.1.0",
  "cases": [
    {
      "format": "json",
      "turns": "single",
      "input_len": "s",
      "checks": [
        "json_schema"
      ],
      "outcome": "worse",
      "failed_new": [
        "json_schema"
      ],
      "reps": 14
    },
    {
      "format": "text",
      "turns": "multi",
      "input_len": "m",
      "checks": [
        "contains"
      ],
      "outcome": "same",
      "failed_new": [],
      "reps": 2
    }
  ]
}
  • Never sent: prompts, system prompts, inputs, outputs, case ids, the text of your checks, the judge’s reasons, repository or file names.
  • The server accepts only these fields and these values. Anything else, including free text, is rejected.
  • Only public models of Anthropic, OpenAI and Google. Fine-tuned and self-hosted model names are never sent, since they can identify a company.
  • One vote per team and migration: sharing again replaces your previous result.
  • A migration is published once 5 different teams have shared it, and within it, no group of cases with fewer than 5 teams.
  • Snapshot dates are removed from model names, so a pinned version does not single you out.
  • Withdraw everything your license shared at any time: curl -X DELETE -H "Authorization: Bearer $SWAPVERA_LICENSE" https://swapvera.com/v1/atlas/mine