OpenAI shipped GPT-5.5 and GPT-5.5 Pro to the API this week. Within 48 hours, Reddit was flooded with benchmark comparisons against Claude Opus 4.6, Opus 4.7, and Gemini. Sam Altman called it "a good week." Perplexity already rolled GPT-5.5 out as their default orchestrator. Community sentiment, as usual, is divided.
Here is the thing about those Reddit benchmarks. Most of them test the wrong thing.
They test raw capability on isolated prompts. They test speed on single-turn completions. They test coding puzzles and trivia questions and creative writing samples. These are fine for consumer use cases. They tell you almost nothing about how these models perform in the workflows that enterprise teams actually care about.
I have been running GPT-5.5, Opus 4.6, and Gemini on the same production-adjacent enterprise workflows for the past week. Multi-document reasoning. Long-context retrieval. Multi-turn agent sessions. Tool call accuracy at depth. The results are more nuanced than any leaderboard ranking would suggest, and they point to a conclusion that the benchmark discourse is not ready to hear: the right model depends entirely on your workload profile.
Table of Contents#
- What I tested and how
- Speed versus coherence: the core tradeoff
- Multi-document reasoning at scale
- Tool call accuracy across depth
- Why Perplexity's decision makes sense for Perplexity
- The three-way evaluation framework
- Cost per correct answer: the metric nobody tracks
- What the Claude Code community is saying
- How to run your own practitioner evaluation
- FAQ
What I Tested and How#
Let me be specific about the evaluation setup, because methodology matters more than results in model comparisons.
I ran three workload profiles against all three models. Each workload was tested with 50 runs to get stable averages. Temperature was set to 0 for all tests. System prompts were identical across models. Context was drawn from the same document corpus.
Workload A: Simple retrieval and summarization. Query a document corpus of 15-20 documents, retrieve relevant sections, produce a structured summary. This is the bread and butter of enterprise RAG applications. Single-turn, relatively straightforward, and latency-sensitive.
Workload B: Multi-document reasoning. Reason across 15+ documents simultaneously, identify contradictions, synthesize conclusions, and cite sources accurately. This is the compliance and financial analysis use case. It requires holding many documents in working memory and reasoning across them, not just retrieving from them.
Workload C: Multi-turn agent sessions. A 20-turn agent session with 10+ tool calls per session. The agent needs to maintain context, make accurate tool calls, handle tool failures gracefully, and produce coherent outputs at depth. This is the workflow that breaks most models eventually.
These are not academic benchmarks. They are simplified versions of actual enterprise workflows I have deployed. The results tell you something different from what MMLU scores tell you.
Speed Versus Coherence: The Core Tradeoff#
GPT-5.5 is fast. There is no hedging on this. On Workload A, simple retrieval and summarization, GPT-5.5 completed tasks 35-40% faster than Opus 4.6 and roughly 25% faster than Gemini. First-token latency was noticeably lower. Total generation time was shorter. For latency-sensitive applications where good-enough answers delivered quickly beat perfect answers delivered slowly, GPT-5.5 is a genuine step forward.
This is why Perplexity moved to it immediately. Search-adjacent workloads are Workload A at scale. Speed matters. Precision matters less than coverage. GPT-5.5 is purpose-built for this tradeoff.
But on Workload B, multi-document reasoning, the picture flips. Opus 4.6 produced more accurate synthesis across 15+ source documents. Specifically, Opus made fewer errors in identifying which document a claim originated from, was better at flagging contradictions between sources, and maintained more consistent reasoning quality as the number of source documents increased.
The delta was not small. On a 20-document reasoning task, GPT-5.5's source attribution accuracy was roughly 12-15 percentage points below Opus 4.6. That gap grows as the number of documents increases. On a 30-document task, it widened to 18-20 points.
Speed means nothing if you have to retry because the synthesis missed a critical contradiction in the source material. When you factor in retry costs, the speed advantage of GPT-5.5 on complex reasoning tasks disappears entirely.
Multi-Document Reasoning at Scale#
Let me dig deeper into the multi-document reasoning results because this is where the practical implications live.
Enterprise compliance, financial analysis, legal review, and audit workflows all share a common structure. You have a large corpus of documents. You need the model to reason across them, not just retrieve from them. The difference is crucial. Retrieval is finding the relevant paragraph. Reasoning is determining that paragraph A from document 3 contradicts paragraph B from document 17, and that the contradiction has implications for the conclusion in document 22.
On this specific capability, Opus 4.6 has a meaningful edge. The model holds context across longer document sets without the quality degrading in the way I saw with GPT-5.5. It is not that GPT-5.5 fails. It produces plausible outputs. The problem is that "plausible" and "correct" diverge when you need the model to track 30 inter-document relationships simultaneously.
For teams building production-ready multi-agent systems, this distinction matters because multi-agent pipelines often require exactly this kind of cross-document reasoning in their orchestration layer.
Gemini occupies an interesting middle ground. Its long-context handling is technically strong, and on pure retrieval from very long documents (200K+ tokens), it performs well. But the reasoning quality on cross-document tasks sits between GPT-5.5 and Opus, which makes it hard to recommend as the primary model for reasoning-heavy workflows when Opus is available.
Tool Call Accuracy Across Depth#
Workload C is where the story gets most interesting for anyone building agent systems.
At tool call 1-5, all three models perform similarly. Tool call accuracy is high. Context is fresh. Outputs are coherent. You could pick any of them and be happy.
At tool call 10-15, divergence appears. GPT-5.5 starts making more errors in tool parameter selection. Not dramatically more. Enough to notice in aggregate. Opus 4.6 maintains higher accuracy through this range. Gemini sits in between but shows a different failure mode: it tends to make redundant tool calls, repeating calls it already made with slightly different parameters.
At tool call 15-20, the differences are significant. Opus 4.6 maintains the highest coherence and tool call accuracy. GPT-5.5's error rate increases more steeply. Gemini's redundant call pattern becomes more pronounced.
This tracks with what the Claude Code community is reporting on Reddit: Opus handles deep sessions better, but the token usage is higher. There is always a tradeoff. Opus gives you better outputs at depth, but it costs more per token and generates more tokens per response.
For context on why tool call depth matters so much, see my deep dive on context window management as a cost lever. The same principles about sending less, better context apply to how models handle deep tool call chains.
Why Perplexity's Decision Makes Sense for Perplexity#
Perplexity switching to GPT-5.5 as their default orchestrator is not a statement about model quality. It is a statement about workload fit.
Perplexity's core use case is search. Search queries are overwhelmingly Workload A: retrieve relevant information, synthesize it quickly, present a coherent answer. Latency matters enormously. Users expect answers in seconds. The quality bar is "accurate enough for a general query," not "precise enough for a compliance audit."
For this workload profile, GPT-5.5 is the right choice. It is faster, cheaper per query, and accurate enough for the median search use case. Choosing it does not mean GPT-5.5 is the best model. It means GPT-5.5 is the best model for Perplexity's workload.
The mistake is extrapolating from Perplexity's choice to your own use case. If you are building a search product, follow Perplexity's lead. If you are building a compliance pipeline, do not.
The Three-Way Evaluation Framework#
Enterprise buyers now face a genuine three-way evaluation between OpenAI, Anthropic, and Google. None of the three models dominates across all workload types. Here is how I think about the mapping.
GPT-5.5: Best for speed-first use cases. Search, summarization, customer-facing chatbots where latency matters more than depth. Strong at Workload A. Competitive at Workload B with moderate document counts. Falls off at Workload C depth.
Opus 4.6: Best for coherence-first use cases. Compliance, financial analysis, multi-agent orchestration, any workflow where context depth and reasoning accuracy matter more than speed. Strong at Workload B and C. Slower and more expensive at Workload A.
Gemini: Best for multimodal and very-long-context retrieval. When you need to process 200K+ token documents, images alongside text, or video content, Gemini has capabilities the other two do not match. Mixed results on cross-document reasoning and agent tool calls.
The practical implication is that most enterprise teams will end up using at least two models. Speed-sensitive surfaces get GPT-5.5. Depth-sensitive pipelines get Opus. The orchestration layer needs to route intelligently between them.
This is, incidentally, another argument for the model-agnostic orchestration approach I have been advocating. If your architecture is locked to one model, you cannot take advantage of the strengths of each.
Cost per Correct Answer: The Metric Nobody Tracks#
Most model evaluations compare cost per token. This is the wrong metric for enterprise use cases. The right metric is cost per correct answer.
Here is why the distinction matters. GPT-5.5 is cheaper per token than Opus 4.6. On Workload A, where accuracy is comparable across models, cheaper per token means cheaper per correct answer. GPT-5.5 wins clearly.
On Workload B, the math changes. GPT-5.5 is cheaper per token, but if it gets the answer wrong 15% more often, and each wrong answer requires a retry, the cost per correct answer can actually be higher than Opus despite the lower per-token rate. With Opus at a higher per-token cost but higher first-pass accuracy, you run fewer retries. Fewer retries means lower total cost for the correct answer.
I have been tracking this metric across deployments. On simple retrieval tasks, GPT-5.5's cost per correct answer is 20-30% lower than Opus. On multi-document reasoning tasks, Opus's cost per correct answer is 10-25% lower than GPT-5.5, even though it costs more per token. The crossover point depends on the complexity of the reasoning required.
If your finance team is comparing model costs by looking at per-token pricing, they are optimizing the wrong variable.
What the Claude Code Community Is Saying#
The Claude Code subreddit has been active this week with GPT-5.5 comparisons. The community sentiment maps to what my evaluations show, but from a developer workflow perspective.
Developers using Claude Code for terminal-native workflows report that Opus handles long coding sessions better. The model maintains better context about the codebase as the session goes deeper. GPT-5.5, tested via competing tools, is faster for quick tasks but loses track of project context more quickly.
There are also Sonnet 4.6 budget burn stories emerging, people using the cheaper model for routine tasks and getting surprised by how many tokens accumulate. This reinforces the cost-per-correct-answer framing. The cheapest model per token is not always the cheapest model per task.
One interesting data point: a hardware usage monitor for Claude Code was shared, showing real-time resource consumption during agent sessions. The community is starting to instrument their development workflows the same way you would instrument a production system. That is a healthy instinct.
For more on this comparison from a practitioner perspective, see my post on Claude Code vs Cursor for production workflows.
How to Run Your Own Practitioner Evaluation#
If this post has one takeaway, it is this: run your own evaluations. Leaderboard rankings are marketing. Reddit benchmarks are anecdotes. Practitioner evaluations on your actual workloads are the only thing that matters.
Here is a minimal evaluation framework:
-
Define 3-5 representative workloads from your actual pipeline. Not toy examples. Real tasks with real data.
-
Run 50+ trials per workload per model. Small sample sizes produce noise that looks like signal. You need enough runs to see stable patterns.
-
Measure cost per correct answer, not cost per token. Track first-pass accuracy. Track retry rates. Multiply per-token cost by total tokens including retries.
-
Test at depth. If your production workload involves multi-turn sessions or long tool call chains, test at the actual depth you need. Models that perform well at turn 3 may degrade significantly by turn 15.
-
Track latency at the task level, not the token level. Tokens per second is a vendor metric. Time to complete a task including retries is a practitioner metric.
-
Document your results and share them. The community needs more practitioner evaluations and fewer isolated benchmark runs.
The evaluation takes about a week to run properly. That week will save you months of running the wrong model for your workload.
FAQ#
How does GPT-5.5 compare to Opus 4.6 on speed?#
GPT-5.5 completes simple retrieval and summarization tasks 35-40% faster than Opus 4.6. First-token latency is noticeably lower across all workload types. For latency-sensitive applications where speed matters more than depth, GPT-5.5 represents a genuine improvement. However, on multi-document reasoning and deep agent sessions, the speed advantage disappears when retry costs from lower accuracy are factored in.
Is GPT-5.5 better than Opus 4.6 for enterprise use cases?#
It depends entirely on the workload. GPT-5.5 is better for speed-first use cases like search, summarization, and customer-facing chatbots. Opus 4.6 is better for coherence-first use cases like compliance analysis, financial reasoning, and multi-agent orchestration. Most enterprise teams will benefit from using both models, routing requests to the appropriate model based on workload characteristics.
Why did Perplexity switch to GPT-5.5?#
Perplexity's core use case is search, which is a speed-first workload. Search queries require fast retrieval, quick synthesis, and low latency. GPT-5.5 excels at this profile. Their decision is a statement about workload fit, not a general endorsement of GPT-5.5 over other models. Teams with different workload profiles should not automatically follow Perplexity's choice.
What is cost per correct answer and why does it matter?#
Cost per correct answer factors in retry rates along with per-token pricing. A model that is cheaper per token but produces wrong answers 15% more often may actually cost more per correct answer than a more expensive model with higher first-pass accuracy. On simple tasks, GPT-5.5's cost per correct answer is 20-30% lower than Opus. On complex reasoning tasks, Opus's cost per correct answer is 10-25% lower, despite higher per-token pricing.
How do the models compare on tool call accuracy at depth?#
At tool calls 1-5, all three models perform similarly. At tool calls 10-15, GPT-5.5 begins showing more errors in tool parameter selection. Opus 4.6 maintains higher accuracy. Gemini shows a different failure mode with redundant tool calls. By tool calls 15-20, Opus maintains the highest coherence and accuracy, while GPT-5.5 shows steeper accuracy degradation and Gemini's redundancy pattern becomes more pronounced.
Should I run my own model evaluation?#
Yes, and this is the single most important recommendation in this post. Leaderboard rankings and Reddit benchmarks test toy workloads. Practitioner evaluations on your actual data and workflows are the only reliable way to make model selection decisions. A proper evaluation with 50+ trials per workload per model takes about a week and will save months of running the wrong model.
What about Gemini in the three-way comparison?#
Gemini occupies a distinct niche. It excels at multimodal processing and very-long-context retrieval (200K+ tokens). For use cases involving mixed media, images alongside text, or extremely long documents, Gemini has capabilities GPT-5.5 and Opus do not match. On cross-document reasoning and agent tool call accuracy, it sits between the other two models, making it harder to recommend as a primary model for reasoning-heavy or agent-heavy workflows.