which AI model should you point your coding agent at

The short answer: match the model to the ticket, not the leaderboard

Claude Fable 5 tops the vals.ai SWE-bench Verified leaderboard at 95%, resolving 475 of 500 real GitHub issues. GPT-5.6 Sol edges it on that specific benchmark at 96.2% (481 of 500). Claude Opus 4.8 sits at 88.6%. Sonnet 5 lands at 79.6%. Gemini 3.1 Pro comes in at 78.8%. None of those numbers tell you which model to actually run today, because the gap between the top and the middle of that list costs real money per task, and most coding tickets don't need the top of the list.

If you want the fast version: browse live demand signals to see what people are actually building right now, then come back to this table to figure out which model should build it.

The benchmark table (verified against vals.ai's independent eval)

vals.ai runs the same 500 SWE-bench Verified tasks against every model and buckets them by how long the fix took a human: under 15 minutes, 15 minutes to an hour, 1 to 4 hours, and over 4 hours. That bucketing matters more than the headline number, because it shows where a model's accuracy actually falls off.

Sonnet 5's intro pricing ($2/$10) holds through August 31, 2026, then moves to $3/$15. Gemini 3.1 Pro's rate above 200K input tokens jumps to $4/$18. Fable 5 and Sol both hold flat pricing regardless of context length up to their max.

Why the ranking flips on harder tasks

Look at the 1-4 hour bucket specifically, the tickets that take a competent human developer most of an afternoon:

• GPT-5.6 Sol: 41/42 resolved (98%)
• Claude Fable 5: 39/42 resolved (93%)
• Claude Opus 4.8: 31/42 resolved (74%)
• Claude Sonnet 5: 32/42 resolved (76%)
• GPT 5.5: 21/42 resolved (50%)

That 20+ point drop between the top two models and everything below them is the actual decision point. If your backlog is mostly small, well-scoped tickets, Sonnet 5 or GPT-5.6 Terra clears the bar fine. If you're handing an agent a genuinely hard multi-file bug and walking away, only Fable 5, Sol, or Opus 4.8 resolve it more than 3 times out of 4.

On the trickier SWE-bench Pro benchmark (harder, less contaminated by public training data), the order changes again: Claude Mythos 5 scores 80.3% against Sol's 64.6%, a 15-point swing in the other direction. No single benchmark settles it. Run both if the task is expensive enough to fail.

Running each model from the CLI

All four labs expose the same basic pattern: pick a model string, point your agent at it. Here's the actual invocation for each, assuming Claude Code, Codex, and Gemini CLI are already installed.

# Claude Code - Sonnet 5 (default on Free/Pro plans)
claude --model claude-sonnet-5

# Claude Code - Opus 4.8 for a hard bug
claude --model claude-opus-4-8

# Claude Code - Fable 5 for an overnight multi-file refactor
claude --model claude-fable-5

# Codex CLI - GPT-5.6 Sol
codex --model gpt-5.6-sol "fix the failing test in payments/checkout.py"

# Codex CLI - Terra for routine tickets (cheaper mid tier)
codex --model gpt-5.6-terra

# Gemini CLI - 3.1 Pro
gemini --model gemini-3.1-pro-preview "refactor the auth middleware to use the new session store"

For API-direct calls instead of a CLI wrapper, the model ID goes straight in the request body:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 4096,
    "messages": [{"role": "user", "content": "write a pytest suite for utils/parser.py"}]
  }'

The cost-per-task math, worked out

A million tokens sounds abstract until you run it against a real coding session. A typical multi-file feature ticket burns somewhere around 40k-80k input tokens (reading the relevant files, the test suite, prior commits) and 8k-15k output tokens (the diff plus explanation).

Sonnet 5 (intro): 60k in x $2/M + 10k out x $10/M = $0.12 + $0.10 = $0.22 per ticket
Opus 4.8:         60k in x $5/M + 10k out x $25/M = $0.30 + $0.25 = $0.55 per ticket
Fable 5:          60k in x $10/M + 10k out x $50/M = $0.60 + $0.50 = $1.10 per ticket

Fable 5 costs 5x Sonnet 5 per ticket for a 15-point accuracy jump on the hardest bucket and effectively no jump on the easy bucket (94% vs 96% on sub-15-minute tickets, within noise). Run the cheap model first. Anthropic's own advisor-strategy setup formalizes this: pairing a cheap executor model with an expensive model as an on-demand advisor scored 2.7 points higher than the expensive model running solo on SWE-bench Multilingual, at 11.9% less cost, and on BrowseComp a Haiku-plus-Opus-advisor pair more than doubled a Haiku-solo score at 85% less cost than running Sonnet the whole way through.

Picking a model by task type, not habit

• Single-file bug fix, clear repro steps: Haiku 4.5 or Sonnet 5. Don't pay Opus 4.8 rates for a ticket a $1/$5 model resolves the same way.
• New feature spanning 3-5 files, existing test coverage: Sonnet 5 as the default. It's the current default Claude Code model on Free and Pro plans for a reason, it clears the 15-minute-to-1-hour bucket at 77%.
• Refactor touching 10+ files, or a bug you've already failed twice on: Opus 4.8 or Fable 5. The 1-4 hour bucket is where the cheap models start missing 1 in 4.
• Overnight or multi-day autonomous run with no one watching: Fable 5 or GPT-5.6 Sol. Both are built to plan across stages and check their own work in an agent harness without a human in the loop every step.
• CI bot doing small mechanical edits at volume (lint fixes, dependency bumps): Haiku 4.5. At $1/$5 per million tokens it's the only one of these six where running it 500 times a day doesn't show up on the invoice.

Cut the bill further with caching and batch mode before you downgrade the model

Before dropping from Opus 4.8 to Sonnet 5 to save money, check whether you're actually using prompt caching. Every lab now discounts repeated context heavily: Anthropic gives up to 90% off cached input tokens, Gemini's caching does the same. On a coding agent that re-reads the same file tree, test suite, and CLAUDE.md-style rules file on every single turn, caching alone can cut the effective input cost by more than half without touching which model you're running.

{
  "model": "claude-opus-4-8",
  "system": [
    {
      "type": "text",
      "text": "<full repo context, coding rules, style guide>",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "messages": [{"role": "user", "content": "add rate limiting to the /api/upload route"}]
}

Run that same repo-context system prompt across 20 tickets in a session and the first call pays full price, the next 19 pay the cached rate. For a 60k-token repo context at Opus 4.8 rates, that's the difference between $0.30 and roughly $0.03 per subsequent call on the input side alone. Batch mode adds another 50% off on top for anything that doesn't need a live response, useful for overnight bulk-refactor runs where you queue up 200 tickets and check the diffs in the morning.

GPT-5.6's three-tier split shows the same pattern from OpenAI's side

OpenAI didn't ship one model with GPT-5.6, it shipped three: Sol at $5/$30 per million tokens (the 96.2% SWE-bench Verified flagship above), Terra at $2.50/$15 (the everyday-ticket tier), and Luna at $1/$6 (the cheapest, closest to Haiku 4.5's price point). That's the same three-tier shape as Anthropic's Fable 5 / Opus 4.8 / Sonnet 5 / Haiku 4.5 lineup and Google's Gemini 3.1 Pro / Flash split. Every lab has converged on the same idea: one frontier model priced for the hardest 10% of tickets, one mid-tier model priced for daily driving, and one cheap model priced for volume. The mistake is running everything through the frontier tier by default because it topped a benchmark chart once.

Where the demand data comes from, if you're building the agent instead of just running it

If you're not just picking a model but building the thing the model executes against, Clawsmith tracks what builders are actually asking for across GitHub, Reddit, HN, and X in real time, currently around 1,503 demand signals pulled from roughly 2,615 public sources. One live example: Anthropic's advisor-strategy launch (the cheap-executor plus expensive-advisor pattern above) is sitting in the tracker as a hype signal with real engagement numbers behind it, the kind of thing that turns into a product idea once three or four similar signals cluster around the same complaint. The pricing page covers what the paid tier adds if you want the agent-ready brief instead of just the raw signal, and the model comparisons category has the rest of this series if a different lineup ships next month and this table goes stale.