Technical Performance · Reasoning Higher is better Publicly reported reasoning benchmark result.
#1 GPT-6 AstraAI Model · Verified 96.0
#2 GPT-5.6 SolAI Model · Verified 94.6
#3 GPT-5.6AI Model · Verified 94.6
#4 Claude Fable 5.1AI Model · Verified 93.7
#5 GPT-5.6 TerraAI Model · Verified 92.9
Technical Performance Max score 100 Weight 1.20×
Technical Performance · Multimodal Higher is better MMMU — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 84.2
#2 o3AI Model · Verified 82.9
#3 Gemini 2.5 ProAI Model · Verified 82.0
#4 o4-miniAI Model · Verified 81.6
#5 GPT-5 miniAI Model · Verified 81.6
Technical Performance Max score 100 Weight 1.00×
Unclassified · Software Engineering Higher is better Structured performance benchmark tracked by AI Orbit.
#1 Claude Sonnet 5AI Model · Verified 85.2
#2 GPT-5AI Model · Verified 74.9
#3 Claude Opus 4.1AI Model · Verified 74.5
#4 Claude Sonnet 4AI Model · Verified 72.7
#5 Claude Opus 4AI Model · Verified 72.5
Technical Performance · Math Higher is better AIME 2025 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 94.6
#2 o4-miniAI Model · Verified 92.7
#3 GPT-5 miniAI Model · Verified 91.1
#4 o3AI Model · Verified 88.9
#5 Gemini 2.5 ProAI Model · Verified 88.0
Technical Performance Max score 100 Weight 1.00×
Technical Performance · Reasoning Higher is better Humanity's Last Exam (No Tools) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 24.8
#2 Gemini 2.5 ProAI Model · Verified 21.6
#3 o3AI Model · Verified 20.2
#4 GPT-5 miniAI Model · Verified 16.7
#5 o4-miniAI Model · Verified 14.7
Technical Performance Max score 100 Weight 0.75×
Technical Performance · Coding Higher is better Aider Polyglot (Diff) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 88.0
#2 o3AI Model · Verified 79.6
#3 GPT-5 miniAI Model · Verified 71.6
#4 o4-miniAI Model · Verified 58.2
#5 GPT-4.1AI Model · Verified 52.9
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Multimodal Higher is better ERQA — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 65.7
#2 o3AI Model · Verified 64.0
#3 GPT-5 miniAI Model · Verified 62.9
#4 o4-miniAI Model · Verified 56.5
#5 GPT-5 nanoAI Model · Verified 50.1
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Multimodal Higher is better MMMU-Pro (Average Standard + Vision) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 78.4
#2 o3AI Model · Verified 76.4
#3 GPT-5 miniAI Model · Verified 74.1
#4 o4-miniAI Model · Verified 73.4
#5 GPT-5 nanoAI Model · Verified 62.6
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Video Higher is better VideoMMMU (Max 256 Frames) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 84.6
#2 o3AI Model · Verified 83.3
#3 GPT-5 miniAI Model · Verified 82.5
#4 o4-miniAI Model · Verified 79.4
#5 GPT-5 nanoAI Model · Verified 66.8
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Vision Higher is better CharXiv Reasoning (Python Enabled) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 81.1
#2 o3AI Model · Verified 78.6
#3 GPT-5 miniAI Model · Verified 75.5
#4 o4-miniAI Model · Verified 72.0
#5 GPT-5 nanoAI Model · Verified 62.7
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Coding Higher is better SWE-bench Verified (OpenAI 477 Tasks) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 74.9
#2 GPT-5 miniAI Model · Verified 71.0
#3 o3AI Model · Verified 69.1
#4 o4-miniAI Model · Verified 68.1
#5 GPT-5 nanoAI Model · Verified 54.7
Technical Performance Max score 100 Weight 1.25×
Technical Performance · Math Higher is better HMMT 2025 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 93.3
#2 GPT-5 miniAI Model · Verified 87.8
#3 o4-miniAI Model · Verified 85.0
#4 o3AI Model · Verified 81.7
#5 GPT-5 nanoAI Model · Verified 75.6
Technical Performance Max score 100 Weight 0.75×
Technical Performance · Mathematical Reasoning Higher is better Publicly reported mathematical reasoning benchmark result.
#1 GPT-6 AstraAI Model · Verified 97.6
#2 Claude Fable 5.1AI Model · Verified 87.8
#3 GPT-5.6 SolAI Model · Verified 83.0
#4 GPT-5.6AI Model · Verified 83.0
#5 GPT-5.6 TerraAI Model · Verified 68.3
Technical Performance Max score 100 Weight 1.20×
Technical Performance · Professional Agents Higher is better Publicly reported professional agents benchmark result.
#1 GPT-6 AstraAI Model · Verified 41.4
#2 Claude Fable 5.1AI Model · Verified 31.4
#3 GPT-5.6 SolAI Model · Verified 18.1
#4 GPT-5.6AI Model · Verified 18.1
#5 GPT-5.6 TerraAI Model · Verified 15.2
Technical Performance Max score 100 Weight 1.10×
Technical Performance · Software Engineering Higher is better Publicly reported software engineering benchmark result.
#1 GPT-6 AstraAI Model · Verified 74.1
#2 GPT-5.6 SolAI Model · Verified 72.7
#3 GPT-5.6AI Model · Verified 72.7
#4 GPT-5.6 TerraAI Model · Verified 69.6
#5 Claude Fable 5.1AI Model · Verified 67.4
Technical Performance Max score 100 Weight 1.15×
Technical Performance · Abstract Reasoning Higher is better Publicly reported abstract reasoning benchmark result.
#1 GPT-6 AstraAI Model · Verified 99.9
#2 GPT-5.6 SolAI Model · Verified 7.8
#3 GPT-5.6AI Model · Verified 7.8
#4 GPT-5.6 TerraAI Model · Verified 0.8
#5 GPT-5.6 LunaAI Model · Verified 0.2
Technical Performance Max score 100 Weight 1.20×
Technical Performance · Computer-Aided Design Higher is better Publicly reported computer-aided design benchmark result.
#1 Claude Fable 5.1AI Model · Verified 84.3
#2 GPT-5.6 SolAI Model · Verified 70.6
#3 GPT-5.6AI Model · Verified 70.6
#4 GPT-5.6 LunaAI Model · Verified 63.1
#5 GPT-5.6 TerraAI Model · Verified 62.3
Technical Performance Max score 100 Weight 1.00×
Technical Performance · Cybersecurity Higher is better Publicly reported cybersecurity benchmark result.
#1 GPT-6 AstraAI Model · Verified 100.0
#2 GPT-5.6 SolAI Model · Verified 73.5
#3 GPT-5.6AI Model · Verified 73.5
#4 GPT-5.6 TerraAI Model · Verified 52.9
#5 GPT-5.6 LunaAI Model · Verified 33.2
Technical Performance Max score 100 Weight 1.10×
Technical Performance · Cybersecurity Higher is better Publicly reported cybersecurity benchmark result.
#1 GPT-6 AstraAI Model · Verified 85.4
#2 GPT-5.6 SolAI Model · Verified 71.2
#3 GPT-5.6AI Model · Verified 71.2
#4 GPT-5.6 TerraAI Model · Verified 57.7
#5 GPT-5.6 LunaAI Model · Verified 48.9
Technical Performance Max score 100 Weight 1.10×
Technical Performance · Math Higher is better FrontierMath (Python tool only) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5AI Model · Verified 26.3
#2 GPT-5 miniAI Model · Verified 22.1
#3 o3AI Model · Verified 15.8
#4 o4-miniAI Model · Verified 15.4
#5 GPT-5 nanoAI Model · Verified 9.6
Technical Performance Max score 100 Weight 0.75×
Technical Performance · Agents Higher is better Agents' Last Exam — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 52.7
#2 GPT-5.6AI Model · Verified 52.7
#3 GPT-5.6 TerraAI Model · Verified 50.4
#4 GPT-5.6 LunaAI Model · Verified 50.3
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Agents Higher is better Toolathlon — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 58.0
#2 GPT-5.6AI Model · Verified 58.0
#3 GPT-5.6 LunaAI Model · Verified 53.4
#4 GPT-5.6 TerraAI Model · Verified 53.1
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Coding Higher is better SWE-Bench Pro — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 64.6
#2 GPT-5.6AI Model · Verified 64.6
#3 GPT-5.6 TerraAI Model · Verified 63.4
#4 GPT-5.6 LunaAI Model · Verified 62.7
Technical Performance Max score 100 Weight 1.00×
Technical Performance · Coding Higher is better Terminal-Bench 2.1 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 88.8
#2 GPT-5.6AI Model · Verified 88.8
#3 GPT-5.6 TerraAI Model · Verified 87.4
#4 GPT-5.6 LunaAI Model · Verified 84.7
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Computer Use Higher is better OSWorld 2.0 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 62.6
#2 GPT-5.6AI Model · Verified 62.6
#3 GPT-5.6 TerraAI Model · Verified 50.2
#4 GPT-5.6 LunaAI Model · Verified 45.6
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Cybersecurity Higher is better Capture-the-Flag Challenges — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 96.7
#2 GPT-5.6AI Model · Verified 96.7
#3 GPT-5.6 TerraAI Model · Verified 91.8
#4 GPT-5.6 LunaAI Model · Verified 85.2
Technical Performance Max score 100 Weight 0.35×
Technical Performance · Cybersecurity Higher is better ExploitGym — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 33.7
#2 GPT-5.6AI Model · Verified 33.7
#3 GPT-5.6 TerraAI Model · Verified 23.2
#4 GPT-5.6 LunaAI Model · Verified 12.4
Technical Performance Max score 100 Weight 0.35×
Technical Performance · Health Higher is better HealthBench Professional — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 60.5
#2 GPT-5.6AI Model · Verified 60.5
#3 GPT-5.6 TerraAI Model · Verified 57.7
#4 GPT-5.6 LunaAI Model · Verified 55.7
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Long Context Higher is better GraphWalks BFS 1M F1 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 77.1
#2 GPT-5.6AI Model · Verified 77.1
#3 GPT-5.6 TerraAI Model · Verified 71.2
#4 GPT-5.6 LunaAI Model · Verified 51.2
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Long Context Higher is better GraphWalks BFS 256K F1 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 90.7
#2 GPT-5.6AI Model · Verified 90.7
#3 GPT-5.6 LunaAI Model · Verified 81.3
#4 GPT-5.6 TerraAI Model · Verified 76.9
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Long Context Higher is better OpenAI MRCR v2 8-Needle 256K-512K — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 91.5
#2 GPT-5.6AI Model · Verified 91.5
#3 GPT-5.6 TerraAI Model · Verified 89.6
#4 GPT-5.6 LunaAI Model · Verified 41.3
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Long Context Higher is better OpenAI MRCR v2 8-Needle 512K-1M — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 73.8
#2 GPT-5.6AI Model · Verified 73.8
#3 GPT-5.6 TerraAI Model · Verified 72.5
#4 GPT-5.6 LunaAI Model · Verified 41.3
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Math Higher is better FrontierMath Tier 1-3 v2 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 89.0
#2 GPT-5.6AI Model · Verified 89.0
#3 GPT-5.6 TerraAI Model · Verified 84.9
#4 GPT-5.6 LunaAI Model · Verified 78.6
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Multimodal Higher is better MMMU Pro (No Tools) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 83.0
#2 GPT-5.6AI Model · Verified 83.0
#3 GPT-5.6 TerraAI Model · Verified 80.7
#4 GPT-5.6 LunaAI Model · Verified 78.4
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Multimodal Higher is better MMMU Pro (With Tools) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 84.6
#2 GPT-5.6AI Model · Verified 84.6
#3 GPT-5.6 TerraAI Model · Verified 82.0
#4 GPT-5.6 LunaAI Model · Verified 79.5
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Professional Higher is better Big Finance Bench — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 53.0
#2 GPT-5.6AI Model · Verified 53.0
#3 GPT-5.6 TerraAI Model · Verified 51.0
#4 GPT-5.6 LunaAI Model · Verified 36.0
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Research Higher is better Internal Research Debugging Evaluation — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 68.3
#2 GPT-5.6AI Model · Verified 68.3
#3 GPT-5.6 TerraAI Model · Verified 67.8
#4 GPT-5.6 LunaAI Model · Verified 50.8
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Research Higher is better KernelGen 1P — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 61.1
#2 GPT-5.6AI Model · Verified 61.1
#3 GPT-5.6 TerraAI Model · Verified 49.2
#4 GPT-5.6 LunaAI Model · Verified 22.4
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Research Higher is better PostTrainBench Lite — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 TerraAI Model · Verified 51.5
#2 GPT-5.6 SolAI Model · Verified 50.3
#3 GPT-5.6AI Model · Verified 50.3
#4 GPT-5.6 LunaAI Model · Verified 29.6
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Science Higher is better GeneBench Pro — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 28.7
#2 GPT-5.6AI Model · Verified 28.7
#3 GPT-5.6 TerraAI Model · Verified 23.3
#4 GPT-5.6 LunaAI Model · Verified 10.8
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Science Higher is better LifeSciBench — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 59.9
#2 GPT-5.6AI Model · Verified 59.9
#3 GPT-5.6 TerraAI Model · Verified 56.0
#4 GPT-5.6 LunaAI Model · Verified 51.2
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Search Higher is better BrowseComp — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 GPT-5.6 SolAI Model · Verified 90.4
#2 GPT-5.6AI Model · Verified 90.4
#3 GPT-5.6 TerraAI Model · Verified 87.5
#4 GPT-5.6 LunaAI Model · Verified 83.3
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Coding Higher is better Aider Polyglot — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 82.2
#2 Gemini 2.5 FlashAI Model · Verified 56.7
#3 DeepSeek-R1AI Model · Verified 53.3
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Coding Higher is better LiveCodeBench — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 74.2
#2 DeepSeek-R1AI Model · Verified 65.9
#3 Gemini 2.5 FlashAI Model · Verified 59.3
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Factuality Higher is better SimpleQA — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 54.0
#2 DeepSeek-R1AI Model · Verified 30.1
#3 Gemini 2.5 FlashAI Model · Verified 26.9
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Multilingual Higher is better MMMLU — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Claude Opus 4AI Model · Verified 87.4
#2 Claude Sonnet 4AI Model · Verified 85.4
#3 Gemma 3 27BAI Model · Verified 70.7
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Reasoning Higher is better MMLU — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 90.8
#2 GPT-4oAI Model · Verified 85.7
#3 GPT-4o miniAI Model · Verified 82.0
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Coding Higher is better SWE-bench Verified (Anthropic Full 500 High Compute) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Claude Sonnet 4AI Model · Verified 80.2
#2 Claude Opus 4AI Model · Verified 79.4
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Coding Higher is better SWE-bench Verified (Anthropic Full 500) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Claude Sonnet 4AI Model · Verified 72.7
#2 Claude Opus 4AI Model · Verified 72.5
Technical Performance Max score 100 Weight 1.25×
Technical Performance · Coding Higher is better SWE-bench Verified (Google Multiple Attempts) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 67.2
#2 Gemini 2.5 FlashAI Model · Verified 60.3
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Coding Higher is better SWE-bench Verified (Google Single Attempt) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 59.6
#2 Gemini 2.5 FlashAI Model · Verified 48.9
Technical Performance Max score 100 Weight 1.25×
Technical Performance · Coding & Terminal Agents Higher is better Publicly reported coding & terminal agents benchmark result.
#1 GPT-6 AstraAI Model · Verified 57.9
#2 Claude Fable 5.1AI Model · Verified 55.8
Technical Performance Max score 100 Weight 1.20×
Technical Performance · Factuality Higher is better FACTS Grounding — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 87.8
#2 Gemini 2.5 FlashAI Model · Verified 85.3
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Long Context Higher is better LOFT Hard <=128K — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 87.0
#2 Gemini 2.5 FlashAI Model · Verified 82.1
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Long Context Higher is better LOFT Hard 1M — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 69.8
#2 Gemini 2.5 FlashAI Model · Verified 58.9
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Long Context Higher is better MRCR v2 8-Needle <=128K — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 58.0
#2 Gemini 2.5 FlashAI Model · Verified 54.3
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Long Context Higher is better MRCR v2 8-Needle 1M — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 FlashAI Model · Verified 21.0
#2 Gemini 2.5 ProAI Model · Verified 16.4
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Multilingual Higher is better Global MMLU Lite — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 89.2
#2 Gemini 2.5 FlashAI Model · Verified 88.4
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Reasoning Higher is better MMLU-Pro is a robust multi-task language-understanding benchmark with a stronger emphasis on challenging reasoning questions.
#1 DeepSeek-R1AI Model · Verified 84.0
#2 Gemma 3 27BAI Model · Verified 67.6
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Scientific Agents Higher is better Publicly reported scientific agents benchmark result.
#1 GPT-6 AstraAI Model · Verified 64.6
#2 Claude Fable 5.1AI Model · Verified 52.6
Technical Performance Max score 100 Weight 1.15×
Technical Performance · Video Higher is better ActivityNet-QA — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 66.7
#2 Gemini 2.5 FlashAI Model · Verified 65.1
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Video Higher is better EgoTempo — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 44.3
#2 Gemini 2.5 FlashAI Model · Verified 36.7
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Video Higher is better Perception Test — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 78.4
#2 Gemini 2.5 FlashAI Model · Verified 75.1
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Video Higher is better QVHighlights — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 75.0
#2 Gemini 2.5 FlashAI Model · Verified 52.4
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Video Higher is better VideoMMMU — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemini 2.5 ProAI Model · Verified 83.6
#2 Gemini 2.5 FlashAI Model · Verified 79.2
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Abstract Reasoning Higher is better ARC-AGI-2 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Grok 4AI Model · Verified 15.9
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Agents Higher is better Tau2 Average — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 16.2
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Chinese Higher is better C-Eval — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 91.8
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Chinese Higher is better C-SimpleQA — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 63.7
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Chinese Higher is better CLUEWSC — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 92.8
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Coding Higher is better Codeforces Percentile — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 96.3
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Coding Higher is better LiveCodeBench v6 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 29.1
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Coding Higher is better SWE-bench Verified (Anthropic 489 High Compute) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Claude 3.7 SonnetAI Model · Verified 70.3
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Coding Higher is better SWE-bench Verified (Anthropic 489 Internal Subset) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Claude 3.7 SonnetAI Model · Verified 63.7
Technical Performance Max score 100 Weight 1.25×
Technical Performance · Coding Higher is better SWE-bench Verified (DeepSeek Resolved) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 49.2
Technical Performance Max score 100 Weight 1.25×
Technical Performance · Coding Higher is better Terminal-bench — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Claude Opus 4AI Model · Verified 43.2
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Cybersecurity Higher is better Publicly reported cybersecurity benchmark result.
#1 Gemini 3.8 Flash CyberAI Model · Verified 86.2
Technical Performance Max score 100 Weight 1.15×
Technical Performance · Cybersecurity / Reliability Higher is better Publicly reported cybersecurity / reliability benchmark result.
#1 GPT-6 AstraAI Model · Verified 88.0
Technical Performance Max score 100 Weight 1.10×
Technical Performance · Cybersecurity Patching Higher is better Publicly reported cybersecurity patching benchmark result.
#1 Gemini 3.8 Flash CyberAI Model · Verified 47.2
Technical Performance Max score 100 Weight 1.15×
Technical Performance · Finance Agents Higher is better Publicly reported finance agents benchmark result.
#1 Gemini 3.8 FlashAI Model · Verified 61.4
Technical Performance Max score 100 Weight 1.00×
Technical Performance · Instruction Following Higher is better IF-Eval (Prompt Strict) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 83.3
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Knowledge & Reasoning Higher is better Publicly reported knowledge & reasoning benchmark result.
#1 Gemini 3.8 FlashAI Model · Verified 54.9
Technical Performance Max score 100 Weight 1.20×
Technical Performance · Knowledge & Reasoning Higher is better Publicly reported knowledge & reasoning benchmark result.
#1 Claude Fable 5.1AI Model · Verified 65.0
Technical Performance Max score 100 Weight 1.20×
Technical Performance · Legal Agents Higher is better Publicly reported legal agents benchmark result.
#1 Gemini 3.8 FlashAI Model · Verified 10.0
Technical Performance Max score 100 Weight 1.00×
Technical Performance · Long Context Higher is better MRCR v2 8-Needle 128K Average — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 13.5
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Math Higher is better AIME 2024 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 79.8
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Math Higher is better AIME 2026 (No Tools) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 20.8
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Math Higher is better CNMO 2024 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 78.8
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Math Higher is better MATH-500 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 97.3
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Multimodal Higher is better MMMU Pro — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 49.7
Technical Performance Max score 100 Weight 0.80×
Technical Performance · Preference Higher is better AlpacaEval2.0 — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 87.6
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Preference Higher is better ArenaHard — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 92.3
Technical Performance Max score 100 Weight 0.40×
Technical Performance · Reasoning Higher is better BigBench Extra Hard — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 19.3
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Reasoning Higher is better DROP (3-shot F1) — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 92.2
Technical Performance Max score 100 Weight 0.60×
Technical Performance · Reasoning Higher is better FRAMES — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 82.5
Technical Performance Max score 100 Weight 0.50×
Technical Performance · Reasoning Higher is better MMLU-Redux — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 DeepSeek-R1AI Model · Verified 92.9
Technical Performance Max score 100 Weight 0.70×
Technical Performance · Vision Higher is better MATH-Vision — official/provider-reported evaluation result. Methodology conditions are preserved in the result notes.
#1 Gemma 3 27BAI Model · Verified 46.0
Technical Performance Max score 100 Weight 0.60×
OpenAI
DeepSeek
