Verified AI benchmark

SWE-bench Verified Leaderboard

Explore verified SWE-bench Verified Unclassified results, rankings, methodology and sources on AI Orbit.

Semantic classUnclassified CategorySoftware Engineering DirectionHigher is better VersionVerified Verified entities16

How to interpret this leaderboard

What this benchmark does—and does not—measure

This leaderboard preserves the benchmark's own metric, source and test date. Results are comparable only within the same benchmark definition and semantic class; missing data is never treated as a zero.

Method & source

Unit: %
Results: 16
Latest test: Aug 22, 2026
Distinct source links: 6

Open official methodology / source

Leading score85.20 %

Entities at top score1

Evidence freshnessAug 22, 2026

Verified results

SWE-bench Verified rankings

Only the latest verified result for each listed entity is shown. Equal scores receive the same displayed rank.

Rank	Model / Tool	Score	Tested	Evidence
#1	Claude Sonnet 5	85.20%	Aug 22, 2026	Anthropic Claude Sonnet 5 System Card
#2	GPT-5	74.90%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#3	Claude Opus 4.1	74.50%	Aug 5, 2025	Anthropic Claude Opus 4.1 announcement
#4	Claude Sonnet 4	72.70%	May 22, 2025	Anthropic Claude 4 announcement
#5	Claude Opus 4	72.50%	May 22, 2025	Anthropic Claude 4 announcement
#6	GPT-5 mini	71.00%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#7	o3	69.10%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#8	o4-mini	68.10%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#9	DeepSeek-V3.1	66.00%	Aug 21, 2025	DeepSeek API changelog
#10	Gemini 2.5 Pro	63.80%	Mar 25, 2025	Google Gemini 2.5 announcement
#11	Claude 3.7 Sonnet	62.30%	May 22, 2025	Anthropic Claude 4 announcement
#12	GPT-5 nano	54.70%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#13	GPT-4.1	54.60%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#14	GPT-4o	33.20%	Apr 14, 2025	OpenAI GPT-4.1 announcement
#15	GPT-4.1 mini	23.60%	Aug 7, 2025	OpenAI GPT-5 developer announcement
#16	GPT-4o mini	8.70%	Apr 14, 2025	OpenAI GPT-4.1 announcement

Limitations

What to keep in mind

A benchmark is one evidence signal, not a universal product verdict. Test conditions, benchmark versions, source methodology and date can all affect interpretation. Compare shared benchmarks and current product evidence before making a model or tool decision.

Freshness

Why dates and sources are shown

AI Orbit retains the source and tested date for each verified result so readers can judge whether evidence is current enough for their use case. Older evidence is not silently presented as a new test.