跳至正文

从目标出发,找到产品、方案与交付方法。

语言
全球站 / Global描述目标
仓库级软件工程500 tasks (% Resolved)

SWE-bench Verified 仓库修复基准

人工核验的 500 项真实开源仓库缺陷修复测试集,检验多文件定位、补丁生成与回归验证闭环。

参考基线快照(非实时遥测)· GEO 摘要CN · 2026-11-21

SWE-bench Verified 仓库修复基准(CN 参考基线)前三:deepseek/deepseek-v3-2 / meta-llama/llama-4-maverick-400b / deepseek/deepseek-v4-pro;峰值标价性价比 meta-llama/llama-4-maverick-400b(262.4/$)。分叉预警:openai/gpt-6-2-astra 属高基准参考延迟退化,meta-llama/llama-4-maverick-400b 属高性价比黑马(非实时遥测)。

口径说明:排行、标价成本、参考延迟与基准分来自静态公开规格与参考基线(非实时网关流量,未经第三方 ZDR 审计签章的节点保持 Unverified)。

评测任务规模500
指标口径% Resolved
评测方法学

在隔离容器沙箱中执行标准测试套件,按单次通过率与工具步数效率加权计入模型基准分。

SWE-bench Verified 适配模型排行 (23)

23 个模型
#模型与实验室流形评分混合标价 / 1M参考速率与基准上下文 / 输出窗首选节点分叉与 ZDR 审计
1
@deepseek·deepseek/deepseek-v3-2
81.8
100% conf
$0.44
212.8 lift/$
122 tok/s
ref 195ms TTFT · 92.8 bench
128K
max out 16K
deepseek-official
共识领跑Unverified
2
@meta-llama·meta-llama/llama-4-maverick-400b
81.1
99% conf
$0.35
262.4 lift/$
164 tok/s
ref 162ms TTFT · 90.8 bench
512K
max out 33K
fireworks-nitro
高性价比黑马Unverified
3
@deepseek·deepseek/deepseek-v4-pro
80.4
100% conf
$0.92
105.6 lift/$
96 tok/s
ref 285ms TTFT · 97.6 bench
1M
max out 384K
deepseek-official
共识领跑Unverified
4
@qwen·qwen/qwen3-coder-480b
79.4
100% conf
$0.64
147.2 lift/$
98 tok/s
ref 235ms TTFT · 94.2 bench
256K
max out 33K
alibaba-bailian
共识领跑Unverified
577.5
100% conf
$1.59
59.6 lift/$
96 tok/s
ref 255ms TTFT · 94.6 bench
1.0M
max out 66K
volcengine-ark
稳健基线Unverified
676.9
100% conf
$2.10
46.4 lift/$
102 tok/s
ref 260ms TTFT · 97.4 bench
1M
max out 131K
alibaba-bailian
稳健基线Unverified
776.9
100% conf
$2.00
47.9 lift/$
98 tok/s
ref 245ms TTFT · 95.8 bench
1M
max out 131K
zhipu-bigmodel
稳健基线Unverified
8
@qwen·qwen/qwen3-8-max
75.5
100% conf
$2.80
34.1 lift/$
94 tok/s
ref 270ms TTFT · 95.6 bench
1M
max out 66K
alibaba-bailian
稳健基线Unverified
9
@deepseek·deepseek/deepseek-r1
74.9
100% conf
$0.88
108.7 lift/$
74 tok/s
ref 310ms TTFT · 95.4 bench
128K
max out 33K
deepseek-official
稳健基线Unverified
10
@moonshot·moonshot/kimi-k3-1-pro
74.6
100% conf
$3.60
26.8 lift/$
92 tok/s
ref 310ms TTFT · 96.6 bench
1M
max out 131K
alibaba-bailian
稳健基线Unverified
11
@moonshot·moonshot/kimi-k2-thinking
74.6
99% conf
$0.96
97.0 lift/$
78 tok/s
ref 360ms TTFT · 93.1 bench
256K
max out 33K
alibaba-bailian
稳健基线Unverified
12
@google·google/gemini-3-pro
49.5
100% conf
$2.00
47.9 lift/$
118 tok/s
ref 330ms TTFT · 95.8 bench
2M
max out 66K
google-vertex
稳健基线Unverified
1346.4
100% conf
$3.60
27.3 lift/$
114 tok/s
ref 280ms TTFT · 98.2 bench
1.0M
max out 66K
google-vertex
稳健基线Unverified
1446.3
100% conf
$3.60
27.2 lift/$
108 tok/s
ref 265ms TTFT · 97.9 bench
1.1M
max out 128K
openai-direct
稳健基线Unverified
1546.2
100% conf
$3.60
27.1 lift/$
106 tok/s
ref 255ms TTFT · 97.7 bench
1M
max out 128K
anthropic-direct
稳健基线Unverified
1645.1
100% conf
$2.80
34.7 lift/$
112 tok/s
ref 240ms TTFT · 97.1 bench
500K
max out 131K
xai-direct
稳健基线Unverified
17
@anthropic·anthropic/claude-opus-5-5
42.9
100% conf
$7.20
13.7 lift/$
76 tok/s
ref 390ms TTFT · 98.4 bench
1M
max out 128K
anthropic-direct
稳健基线Unverified
1841.5
100% conf
$4.00
24.2 lift/$
86 tok/s
ref 370ms TTFT · 96.8 bench
400K
max out 128K
openai-direct
稳健基线Unverified
19
@anthropic·anthropic/claude-sonnet-4-5
39.3
100% conf
$5.40
17.9 lift/$
84 tok/s
ref 365ms TTFT · 96.4 bench
200K
max out 64K
anthropic-direct
稳健基线Unverified
20
@openai·openai/gpt-6-2-astra
32.2
100% conf
$14.40
6.9 lift/$
44 tok/s
ref 680ms TTFT · 99.1 bench
1.1M
max out 128K
openai-direct
延迟退化预警Unverified
21
@anthropic·anthropic/claude-fable-5-2
31.5
100% conf
$14.40
6.9 lift/$
42 tok/s
ref 710ms TTFT · 99.0 bench
1M
max out 128K
anthropic-direct
延迟退化预警Unverified
22
@anthropic·anthropic/claude-opus-4-5
26.2
99% conf
$9.00
10.9 lift/$
38 tok/s
ref 820ms TTFT · 97.8 bench
200K
max out 64K
anthropic-direct
延迟退化预警Unverified
23
@openai·openai/o3-deep-research
24.2
98% conf
$12.80
7.7 lift/$
32 tok/s
ref 960ms TTFT · 98.1 bench
200K
max out 100K
openai-direct
延迟退化预警Unverified