SWE-bench Verified 仓库修复基准
人工核验的 500 项真实开源仓库缺陷修复测试集,检验多文件定位、补丁生成与回归验证闭环。
参考基线快照(非实时遥测)· GEO 摘要CN · 2026-11-21
SWE-bench Verified 仓库修复基准(CN 参考基线)前三:deepseek/deepseek-v3-2 / meta-llama/llama-4-maverick-400b / deepseek/deepseek-v4-pro;峰值标价性价比 meta-llama/llama-4-maverick-400b(262.4/$)。分叉预警:openai/gpt-6-2-astra 属高基准参考延迟退化,meta-llama/llama-4-maverick-400b 属高性价比黑马(非实时遥测)。
口径说明:排行、标价成本、参考延迟与基准分来自静态公开规格与参考基线(非实时网关流量,未经第三方 ZDR 审计签章的节点保持 Unverified)。
在隔离容器沙箱中执行标准测试套件,按单次通过率与工具步数效率加权计入模型基准分。
SWE-bench Verified 适配模型排行 (23)
23 个模型| # | 模型与实验室 | 流形评分 | 混合标价 / 1M | 参考速率与基准 | 上下文 / 输出窗 | 首选节点 | 分叉与 ZDR 审计 |
|---|---|---|---|---|---|---|---|
| 1 | 81.8 100% conf | $0.44 212.8 lift/$ | 122 tok/s ref 195ms TTFT · 92.8 bench | 128K max out 16K | deepseek-official | 共识领跑Unverified | |
| 2 | 81.1 99% conf | $0.35 262.4 lift/$ | 164 tok/s ref 162ms TTFT · 90.8 bench | 512K max out 33K | fireworks-nitro | 高性价比黑马Unverified | |
| 3 | 80.4 100% conf | $0.92 105.6 lift/$ | 96 tok/s ref 285ms TTFT · 97.6 bench | 1M max out 384K | deepseek-official | 共识领跑Unverified | |
| 4 | 79.4 100% conf | $0.64 147.2 lift/$ | 98 tok/s ref 235ms TTFT · 94.2 bench | 256K max out 33K | alibaba-bailian | 共识领跑Unverified | |
| 5 | 77.5 100% conf | $1.59 59.6 lift/$ | 96 tok/s ref 255ms TTFT · 94.6 bench | 1.0M max out 66K | volcengine-ark | 稳健基线Unverified | |
| 6 | 76.9 100% conf | $2.10 46.4 lift/$ | 102 tok/s ref 260ms TTFT · 97.4 bench | 1M max out 131K | alibaba-bailian | 稳健基线Unverified | |
| 7 | 76.9 100% conf | $2.00 47.9 lift/$ | 98 tok/s ref 245ms TTFT · 95.8 bench | 1M max out 131K | zhipu-bigmodel | 稳健基线Unverified | |
| 8 | 75.5 100% conf | $2.80 34.1 lift/$ | 94 tok/s ref 270ms TTFT · 95.6 bench | 1M max out 66K | alibaba-bailian | 稳健基线Unverified | |
| 9 | 74.9 100% conf | $0.88 108.7 lift/$ | 74 tok/s ref 310ms TTFT · 95.4 bench | 128K max out 33K | deepseek-official | 稳健基线Unverified | |
| 10 | 74.6 100% conf | $3.60 26.8 lift/$ | 92 tok/s ref 310ms TTFT · 96.6 bench | 1M max out 131K | alibaba-bailian | 稳健基线Unverified | |
| 11 | 74.6 99% conf | $0.96 97.0 lift/$ | 78 tok/s ref 360ms TTFT · 93.1 bench | 256K max out 33K | alibaba-bailian | 稳健基线Unverified | |
| 12 | 49.5 100% conf | $2.00 47.9 lift/$ | 118 tok/s ref 330ms TTFT · 95.8 bench | 2M max out 66K | google-vertex | 稳健基线Unverified | |
| 13 | 46.4 100% conf | $3.60 27.3 lift/$ | 114 tok/s ref 280ms TTFT · 98.2 bench | 1.0M max out 66K | google-vertex | 稳健基线Unverified | |
| 14 | 46.3 100% conf | $3.60 27.2 lift/$ | 108 tok/s ref 265ms TTFT · 97.9 bench | 1.1M max out 128K | openai-direct | 稳健基线Unverified | |
| 15 | 46.2 100% conf | $3.60 27.1 lift/$ | 106 tok/s ref 255ms TTFT · 97.7 bench | 1M max out 128K | anthropic-direct | 稳健基线Unverified | |
| 16 | 45.1 100% conf | $2.80 34.7 lift/$ | 112 tok/s ref 240ms TTFT · 97.1 bench | 500K max out 131K | xai-direct | 稳健基线Unverified | |
| 17 | 42.9 100% conf | $7.20 13.7 lift/$ | 76 tok/s ref 390ms TTFT · 98.4 bench | 1M max out 128K | anthropic-direct | 稳健基线Unverified | |
| 18 | 41.5 100% conf | $4.00 24.2 lift/$ | 86 tok/s ref 370ms TTFT · 96.8 bench | 400K max out 128K | openai-direct | 稳健基线Unverified | |
| 19 | 39.3 100% conf | $5.40 17.9 lift/$ | 84 tok/s ref 365ms TTFT · 96.4 bench | 200K max out 64K | anthropic-direct | 稳健基线Unverified | |
| 20 | 32.2 100% conf | $14.40 6.9 lift/$ | 44 tok/s ref 680ms TTFT · 99.1 bench | 1.1M max out 128K | openai-direct | 延迟退化预警Unverified | |
| 21 | 31.5 100% conf | $14.40 6.9 lift/$ | 42 tok/s ref 710ms TTFT · 99.0 bench | 1M max out 128K | anthropic-direct | 延迟退化预警Unverified | |
| 22 | 26.2 99% conf | $9.00 10.9 lift/$ | 38 tok/s ref 820ms TTFT · 97.8 bench | 200K max out 64K | anthropic-direct | 延迟退化预警Unverified | |
| 23 | OpenAI o3 深度推理专精BYOK | 24.2 98% conf | $12.80 7.7 lift/$ | 32 tok/s ref 960ms TTFT · 98.1 bench | 200K max out 100K | openai-direct | 延迟退化预警Unverified |

