Frontier Agent Benchmarks: DeepSeek V4 vs Qwen 3.8

The most consequential open-weight benchmark battle of the summer is a head-to-head between two labs that shipped within ten days of each other. DeepSeek's V4 family went generally available on August 13 with its 0813 build. Alibaba's Qwen3.8-Max followed its August 3 launch with a full, published benchmark table.

Both claim the same thing: that the frontier has moved from knowledge and reasoning into agentic execution — long-horizon tool use, autonomous terminal work, and real-world issue resolution. The benchmark data supports that shift. It also tells a more complicated story about which numbers you can actually trust.

Here's the catch you need before routing production traffic to either model. DeepSeek's headline scores have been partly reproduced on independent, neutral harnesses. Qwen's entire table is still Alibaba's word alone. And on August 16, DeepSeek repriced its API upward by up to 12x on some token types — so any TCO math you did before that date is wrong.

The Contenders

DeepSeek V4 Pro (0813) and V4 Flash (0731)

DeepSeek's V4 line shipped as a text preview on April 24 and reached general availability on August 13 with the 0813 checkpoint. Both models carry a 1M-token context window and a 384K max output — enough for entire repositories or long-running agent conversations without serial truncation.

V4-Flash is the efficiency play: 284B total parameters with 13B active per token, and DeepSeek's own notes flag a much narrower gap to Pro than the parameter count suggests. It's the model the release notes recommend for subagent workloads.

V4-Pro is the reasoning tier. Its agentic gains are the story of the release: DeepSWE jumped from 12.8 on the preview to 62.7 on the official card — roughly a 5x improvement in a single update.

Two structural details matter for builders. First, the legacy model names (deepseek-chat, deepseek-reasoner) were fully retired on July 24 — stale configs now silently route to the wrong thing. Second, DeepSeek moved to peak/off-peak billing on August 16, which changes the cost math below.

Qwen3.8-Max

Qwen3.8-Max is a 2.4-trillion-parameter sparse Mixture-of-Experts model with ~95B active parameters per query — roughly 4% of the network firing per token. It's multimodal on input (text, image, video) and returns text, with a 1M context window and 131K max output.

Alibaba positions it as a long-horizon autonomous worker. The release demos include the model reproducing a research paper's results over five days (7,600 lines of code, 33 GPU training jobs) and beating the paper's method on AIME24 by 2.7 points. A reasoning_effort parameter trades speed for thoroughness across three levels.

Two things make it deployable today: it speaks both OpenAI's and Anthropic's API protocols, so it plugs directly into Claude Code, Codex, and Qoder — and it ships a sibling checkpoint, Qwen3.8-27B, that's the realistic local-first option.

One thing doesn't: the open weights. Alibaba promised Max and 27B files the week of August 10, but the license was still unpublished at the time of writing. Until that text exists, "going open-weight" is a press strategy, not a property of the model.

How to Read Agentic Benchmarks

Agentic benchmarks don't measure the model alone. They measure the model plus a scaffold plus a harness, and harness choice can swing results by several points. Three names dominate this matchup:

• SWE-bench Pro / Verified — resolve real GitHub issues: read a bug report, navigate a repository, edit the right files, and pass the tests. Verified is a human-validated 500-task subset; Pro is the harder 500.
• Terminal-Bench 2.1 — autonomous terminal work: run commands, parse output, and recover from errors over hours of execution.
• OSWorld-Verified — computer-use agents that interact with graphical interfaces like a person would.

There's one more layer you have to keep straight, and it's the whole point of this article. Vendor-reported scores are claims; independently reproduced scores are facts. Alibaba ran most of Qwen's coding rows inside Claude Code's harness — fair, but it means those numbers are really "Qwen inside Anthropic's harness" results. DeepSeek's official card is vendor-reported too, but two of its key rows have since been rerun on neutral harnesses.

The Benchmark Breakdown

SWE-bench Pro is Qwen's win, and it's a real one. Qwen3.8-Max posts 67.7 against DeepSeek's 55.4, and it also edges GPT-5.6 Sol (64.6) on this row. But Alibaba's own table shows the honest ceiling: Claude Fable 5 sits at 80.0, a 12-point gap on the hardest software-engineering eval in the field. On this row, Qwen is the pick — with the caveat that it's Alibaba's harness, not a neutral one.

SWE-bench Verified is DeepSeek's strongest, most trustworthy claim — precisely because DeepSeek never made it. Vals AI's neutral bash-only harness measured V4 Pro 0813 at 96.4%, second overall behind only Claude Opus 5 (97.0%) and ahead of every other closed model, at $0.022 per test versus Opus 5's $1.29. There's no vendor chart to discount. This is patch-shaped work — bug fixes, PR-scale edits — and it's independently the best cost-to-performance number on any board right now.

Terminal-Bench 2.1 is where you see the reproduction gap quantified. DeepSeek's official card claims 87.9 — and its own docs flag that figure as a "provider run." An independent run measured 78.7, a nine-point gap. Qwen sits at 86.6, ahead of Claude Opus 4.8 and Claude Fable 5 (both 84.6) and behind GPT-5.6 Sol at max effort (88.8). On vendor numbers the two are near-tied; on the one independent DeepSeek data point, Qwen would take the row. Take this as the template for how much any unverified agentic number is worth.

OSWorld-Verified is a Qwen-only row in this matchup. DeepSeek publishes no computer-use score, which is itself information: if your agents need to drive GUIs or operate web applications, Qwen3.8-Max at 86.1 is the clear leader of the published table (Claude Fable 5: 85.0, GPT-5.6 Sol: 83.2). Just remember that 86.1 is still an unverified claim.

The pattern across the rows is the same shape both labs have been reporting all summer: the gains landed in tool use and long-horizon execution, not in static reasoning. GPQA Diamond is essentially saturated — Qwen at 92.6, DeepSeek's official card at 92.8, everyone within a point or two.

What the Table Doesn't Tell You

Three things the scorecards can't show you.

Calibration. Qwen's HLE score (43.6) is the weakest of the four flagships, and DeepSeek's official card puts its HLE in the low-40s too — both well behind Claude Fable 5's 53.3. But neither lab publishes hallucination or calibration rates for these builds. Capability gains and truthfulness are separate axes, and neither vendor is volunteering the second one.

Harness sensitivity. Alibaba ran its coding rows inside Anthropic's Claude Code harness. That's a reasonable methodology — same tool, all models — but it means Qwen's coding scores would be different, possibly by several points, inside a different scaffold. Wait for an independent rerun before you anchor a decision on the delta.

The license. DeepSeek's open weights are out and self-hostable. Qwen3.8-Max's weights were promised for the week of August 10 with no license published. If you're planning self-hosting, the practical move right now is the Qwen3.8-27B checkpoint — or wait for the license text.

Pricing & TCO

This is where the matchup stops being close. The table uses DeepSeek's post-August-16 rates (off-peak to peak) and Qwen's standard API pricing.

DeepSeek is roughly 3x cheaper than Qwen at list price, and the gap only widens when you account for caching. Agent workloads reuse the same system prompt and context across calls, which is exactly what prefix caching rewards. DeepSeek's cache-hit input runs $0.022–$0.044 versus $0.66–$1.32 on a miss — a 30x discount. Qwen's is 8x ($0.25 vs $2.00). For a long-running agent loop, DeepSeek's cache economics are the difference between viable and not.

Two pricing facts changed the calculus recently, and you should have both in mind:

1. DeepSeek repriced on August 16. V4 Pro input went from a flat $0.435 to $0.66 off-peak / $1.32 peak, and output from $0.87 to $1.98 / $3.96 — plus new peak/off-peak billing (peak hours 01:00–04:00 and 06:00–10:00 UTC). Cache-hit input rose the most, up to 12x, which is the cost that hits agent workloads hardest. The article's earlier "$0.14–$0.43" range is the pre-increase number and no longer holds.
2. Qwen is still the mid-market price anchor. At $2/$6 it undercuts Kimi K3 ($3/$15) and sits at a fraction of the Western flagships (Claude Opus 5, GPT-5.6 Sol at $5/$30+). DeepSeek undercuts everyone. The market now has a clean three-tier open-weight structure: DeepSeek at the bottom, Qwen in the middle, Kimi K3 above.

For high-volume subagent work, don't ignore Flash. It posts an independently measured 88.8% on SWE-bench Verified (Vals AI) at $0.22–$0.44 input — and, counterintuitively, Pro is often cheaper per successful task because Flash burns far more tokens. See our Flash vs Pro pricing breakdown for the exact math.

Which Model for Your Workload

This isn't a "which is better" decision. It's a workload decision.

Choose DeepSeek V4 (Pro or Flash) when:

• You run cost-sensitive, high-volume agent loops — DeepSeek's price and 30x cache discount win at scale
• Your work is patch-shaped: bug fixes and PR-scale edits, where the independently verified 96.4% SWE-bench Verified matters
• You want 1M-token context without a per-token premium
• You need self-hosted open weights today, not next week
• You want the numbers to be third-party verified rather than a vendor's word

Our DeepSeek V4 + Claude Code setup covers the exact env vars, and cost optimization patterns is worth reading before you scale. Related coverage: DeepSeek V4-Flash-0731.

Choose Qwen3.8-Max when:

• Your agents work in the terminal or drive GUIs — Terminal-Bench and OSWorld-Verified are its leading rows
• You need multimodal input (screenshots, documents, video) — DeepSeek has no comparable row
• You're doing long-horizon autonomous research — PaperBench at 93.0 is the top published score
• You can afford enterprise-grade orchestration and are willing to treat vendor numbers as provisional until independent runs land

For the local path, the Qwen3.6-27B local coding guide and the Qwen3.6-27B benchmark comparison are the practical next reads — the 27B-class checkpoints are where the agentic gains actually reach machines people own.

The Bottom Line

The head-to-head settles into a clean split. DeepSeek wins on verified evidence and economics: it's the only one of the two with independently reproduced scores, it's three times cheaper, and its cache math is built for agent loops. Qwen3.8-Max wins on capability breadth: multimodal input, computer-use agents, terminal autonomy, and research-scale long-horizon work — all of it, at this point, on the vendor's word.

The two Chinese labs are telling the same story, and it's worth hearing twice: the frontier's gains are now in execution, not knowledge. GPQA is saturated. DeepSWE, FrontierSWE, and JobBench are where the generations moved. If you're choosing an open-weight base for agent infrastructure in late 2026, DeepSeek V4 is the lower-risk default, and Qwen3.8-Max is the higher-ceiling bet that needs an independent benchmark run to earn production trust.

The question I'm watching for the rest of August: when the first neutral evaluator scores Qwen3.8-Max, does the reproduction gap show up the way it did for DeepSeek's 87.9 Terminal-Bench claim — and for Kimi K3 before it? The answer will tell you whether the gap between the two scorecards is a marketing artifact or the real thing.

Sources: Vals AI — SWE-bench Verified leaderboard, OrcaRouter — DeepSeek V4 Pro benchmarks (official vs independent), NIST CAISI — Evaluation of DeepSeek V4 Pro, DeepSeek API docs — Models & Pricing, Mercatus — DeepSeek V4 Pro pricing increase, MarkTechPost — Qwen3.8-Max release, The Decoder — Qwen3.8-Max, Thorsten Meyer — Qwen3.8-Max reality check, Emergent — Qwen 3.8 benchmarks explained, S5 Labs — Qwen3.8-Max pricing