Paper Brief - Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: The /goal Command Is Not a Magic Switch

What happens when you put the two most advanced AI coding agents on the market - Claude Fable 5 and GPT-5.6 Sol - head-to-head on a problem so hard that the search space contains roughly 10^1223 possible configurations? Charles Azam, an operations researcher and engineer, ran exactly this experiment. The results are the most detailed, honest comparison of frontier AI model reasoning we have seen this year. And the headline finding is not about which model won. It is about how the /goal command - a feature both Anthropic and OpenAI market as a way to keep agents on track - can quietly make your results worse even when it appears to help.

The benchmark is called KIRO, a fiber-network design problem Azam originally tackled as an engineering student in 2018. It involves connecting distribution points and terminals in three French cities - Grenoble, Nice, and Paris - using directed loops and short branches, while minimizing total cable length. A single lower bound on the search space: even ignoring ordering and branch structure, simply assigning each of 532 terminals to one of 11 distribution hubs yields 11^532 possible assignments. A more realistic bound for a restricted family of valid solutions clocks in at 10^1223 combinations. This is not a synthetic benchmark cooked up to show off. It is a real constraints satisfaction problem with a known human baseline that Azam spent a week writing C++ to solve.

The Experiment: Head-to-Head on Hard Optimization

Azam tested six models in total - Claude Fable 5, Opus 4.8, Sonnet 5, and GPT-5.6 Sol, Terra, and Luna - but concentrated his analysis on the flagship pair: Fable 5 versus Sol. Each model received the same problem with no hints, a 30-minute optimization budget, and maximum reasoning settings. The key variable was whether the model ran in plain mode or with its native /goal command enabled, which is Anthropic's objective-setting feature that prompts the agent to keep working toward a stated goal.

The methodology was rigorous. Azam repeated the flagship comparison until he had three matched runs for each model in each mode. Every scored output was validated against the problem's structural constraints. The containers ran with eight CPUs, which favored Fable 5's parallel portfolio approach, and the entire benchmark suite ran on Harbor 0.1.43 via Docker with subscription authentication. The code, prompts, result tables, and exclusion notes are all published in the CLIArena repository on GitHub.

What the Results Actually Show

Fable 5 clearly outperformed GPT-5.6 Sol on this problem. Fable 5's plain mean score was 32,386 across three runs, while Sol's plain mean was 34,261 - a gap of 1,875 points. More importantly, Fable 5 was dramatically more consistent. Its plain runs stayed within a 319-point range. Sol's plain runs spanned 1,958 points. The best individual score across the entire experiment was Fable 5 with /goal enabled, achieving 31,934 - the lowest total cable length of any run.

But here is where the story gets counterintuitive. The /goal command won 4 out of 6 individual trials, meaning it produced a better score than the plain run in two-thirds of cases. Yet for both models, the average score with /goal was worse than the average without. Fable 5's /goal mean was 33,145 (759 points worse than plain). Sol's /goal mean was 35,129 (868 points worse than plain). How can a feature win most of the time but make the average worse? The answer: when /goal fails, it fails spectacularly. In one run, Fable 5 with /goal produced a score of 35,178 - a 2,732-point regression from its plain run. Sol with /goal hit 39,371 in its worst run - 5,790 points worse than its plain equivalent. The median moved slightly in the right direction; the bad tail moved much further in the wrong one.

Azam explains this pattern elegantly: on a normal coding task, progress is often legible - another turn can fix a test or complete a migration. Optimization is different. Once an agent chooses a solver strategy, extra time can amplify either a good decision or a bad one. The /goal command gave good strategies more time to find better solutions, but it also gave bad strategies more time to dig deeper into dead ends.

The /goal Command: Two Different Implementations

One of the most valuable parts of Azam's analysis is his deep dive into how /goal is actually implemented in Claude Code versus Codex (OpenAI's CLI). These are not the same feature under the hood, and the difference matters for anyone building on top of either platform.

Claude Code implements /goal as a session-scoped Stop hook. After each main-model turn, a small evaluator model (Haiku by default) reads the goal condition and the conversation transcript, then returns yes or no with a reason. A no starts another turn; a yes clears the goal. The evaluator cannot use tools or inspect files - it can only judge evidence in the transcript. That can catch an early exit, but it cannot know whether another ten million solver iterations are worthwhile.

Codex, by contrast, treats a goal as persisted thread state. The TUI saves the objective, SQLite stores its status and budget accounting, and the working model receives create_goal, get_goal, and update_goal tools it can call directly. If the thread becomes idle while the goal is active, Codex injects a continuation turn with the objective and a completion audit. Claude delegates completion judgment to a separate model; Codex lets the working model declare completion itself, then resumes it while the persisted goal stays active. Claude's evaluator is independent but sees only the transcript; Codex sees the files and tools but effectively grades its own work.

Neither approach is obviously superior. Claude's design is safer against self-deception but blind to file-level progress. Codex's design is more capable of understanding real task state but introduces a principal-agent problem where the model evaluates its own completion.

What This Means for Builders

Three practical takeaways emerge from this benchmark. First, Fable 5's raw reasoning on hard constraint satisfaction problems is exceptional - it outperforms GPT-5.6 Sol on this specific class of operations research problem with dramatically better consistency. If you are building a product that depends on model reasoning quality - code generation, route optimization, scheduling, logistics - the choice of model matters more than ever, and Fable 5 appears to have a meaningful edge.

Second, the /goal feature is not a generic try-harder switch. It changes how the model explores solutions, which is useful knowledge for anyone building agentic workflows. On problems where iteration is inherently valuable (debugging, refactoring, writing), /goal may help. On problems where the quality of the first approach determines the ceiling (optimization, creative direction, architecture decisions), /goal can amplify mistakes. The right call depends on the task, not on a blanket recommendation.

Third, we are entering an era where meaningful model-to-model comparisons are becoming possible as frontier models diverge in reasoning styles. Two years ago, comparing GPT-4 and Claude 3 was mostly about speed and price. Today, the differences are structural - different architectures, different agent loop designs, different failure modes. Builders who invest in their own benchmarking pipelines, rather than relying on public leaderboards, will have a genuine competitive advantage in choosing the right model for their specific use case.