Claude Fable 5 vs GPT-5.6 Sol on an NP-Hard Problem: Does /goal Help?

We're drowning in benchmark scores. LMSYS leaderboards, Artificial Analysis indices, swe-bench results — every week a new number claims to tell you which model is "best." What we don't get enough of is real work: someone picking a genuinely hard problem neither model was trained on, running both flagships on it, and publishing the raw numbers.

Charles Azam did exactly that. He gave Claude Fable 5 and GPT-5.6 Sol the same unpublished NP-hard optimization problem — a fiber-network design task called KIRO that he'd previously spent a week writing C++ to solve — and ran each model in plain mode and with their native /goal mode. The results are the most practical head-to-head I've seen all month.

The headline: Fable 5 is a beast. /goal is not a game changer. But the reasons why are more interesting than either of those statements alone.

The Problem: Why NP-Hard Makes a Better Test

KIRO is an operations research problem: given directed distance matrices for Grenoble, Nice, and Paris, connect distribution points to terminals using loops and short chains while minimizing total cable length. The search space is comically large — 11^532 possible hub-to-terminal assignments alone, before even considering ordering and branching.

This matters because most coding benchmarks test things models have seen variations of: writing CRUD endpoints, fixing syntax errors, refactoring Python. Models can pattern-match their way through those. NP-hard optimization doesn't give ground to pattern matching. An agent either builds a competent solver or it doesn't, and there's no amount of "let me try another approach" that helps if the first approach is fundamentally wrong.

That makes it a near-ideal probe for understanding what /goal mode actually does — because /goal's entire job is to keep the agent working. And on a problem where more work can mean more wrong, that's a feature with a sharp edge.

The Results: Fable 5 Dominates, but Here's the Twist

Azam ran three trials per model per mode — plain and /goal — with a 30-minute optimization budget and Harbor 0.1.43 as the execution layer. Here's the best single run:

Fable 5 with /goal: 31,934 (lowest cable length = best) Fable 5 plain: 32,197 (best plain run) GPT-5.6 Sol plain: 33,581 (best Sol run) GPT-5.6 Sol with /goal: 32,703 (best Sol /goal run)

Fable 5 plain is already 1,875 points ahead of Sol plain. That's a significant gap — roughly 5.5% better on a problem where every point of cable length is real optimization surface. More importantly, Fable 5's consistency is dramatic: its three plain runs span a range of just 319 points, while Sol's plain runs range 1,958 points. Azam calls Fable 5's stability "unlike anything I have seen from a model on this problem" — and he's been running models on KIRO as a personal benchmark for years.

Sol is not bad — 33,581 on a problem with a 10^1223 search space is impressive. But Fable 5 is operating in a different tier of reliability, and on an optimization problem where you're paying per-agent-token, that consistency matters.

The /goal Paradox: Winning Trials, Losing the Average

Here's the part that stopped me.

/goal won 4 out of 6 matched trials. If you just looked at win rates, you'd say it's clearly helpful. But when you average all three runs per mode, both models performed worse with /goal:

How can a feature that wins most trials make average performance worse? Because the losses are catastrophic while the gains are incremental. When /goal sustains a good strategy — Fable's fast compiled portfolio, Sol's successful chain repartition — it shaves off a few hundred points. When it extends a bad strategy — Fable building a slow interpreted solver and sticking with it, Sol committing to an exhaustive anchor sweep — the damage compounds.

The median moves slightly in the right direction. The tail moves dramatically in the wrong one. On a normal coding task, extra time usually helps — another turn can fix a test or complete a migration. On optimization, extra time amplifies whatever decision the agent already made, good or bad.

This is the finding every team adopting agentic coding tools should internalize. /goal is not a generic "try harder" button. It changes the control loop, and that change has asymmetric risk.

Two Very Different /goal Implementations

The most valuable part of Azam's analysis is the implementation dig. "Anthropic and OpenAI both support /goal" sounds like a feature parity statement, but the mechanisms couldn't be more different.

Claude Code's /goal uses an external evaluator — by default, a Haiku-class model that reads the session transcript after every turn and returns a yes/no verdict on whether the goal is met. The evaluator doesn't have access to tools or files; it judges only the conversation history. This means Claude Code has a genuinely independent quality gate, but one that's blind to what's actually on disk.

Codex's /goal gives the working model itself a set of tools — create_goal, get_goal, update_goal — and persits goal state in SQLite. When the thread goes idle with an active goal, Codex injects a continuation turn with the objective and a completion audit. The model grades its own work, with full access to files and tools, but no external oversight.

These are philosophically opposite approaches. Claude Code separates "doing" from "evaluating" at the cost of evaluator blindness. Codex trusts the model to self-assess, at the cost of potential confirmation bias. Azam doesn't declare a winner here — and I think that's right. Each tradeoff is appropriate for different workloads. The point is that they're different, and developers need to understand which one matches their risk profile.

What This Means for Developers

Three practical takeaways:

1. Fable 5 is the current optimization king, but that's a specific claim. This is one problem, tested for 30-minute sessions. On code generation, tool use, or creative work, the balance may shift. What this benchmark validates is that Fable 5's notorious deliberateness pays off on problems where premature commitment is expensive.

2. Use /goal with explicit constraints, not as a default. Azam's data suggests that /goal is most useful when you can specify a clear, bounded condition — "reduce the score below 32,000" — rather than an open-ended "keep improving." Open-ended goals risk amplifying early bad decisions, while bounded goals give the evaluator a crisp stopping criterion.

3. Watch the mechanism, not the name. Claude Code's separated evaluator and Codex's self-grading agent have different failure modes. A separated evaluator that can't read files might declare victory too early on tasks with invisible failures. A self-grading model might never declare victory at all on open-ended optimization. If you're building a pipeline that depends on /goal for reliability, test with both approaches.

The original article is worth a full read — Charles Azam goes deep on the implementation details, including the Harbor execution setup and per-run analysis. The HN discussion is also picking up steam as I write this.

What I really want to see next: someone run the same test on Fable 5 against itself — plain vs /goal — across 30+ trials to get statistically meaningful distributions. The single-trial data here is suggestive, but the hypothesis that /goal has a fat left tail deserves a larger sample. If Anthropic or OpenAI are reading: this is the kind of benchmark that would actually improve your products. Run it at scale.