SWE-bench Verified
last retrieved 2026-09-08 21:40 UTC
Third-party scores shown as context with provenance — our own arena ratings from blind human votes stay alongside every row and remain the primary signal.
| Model | SWE-bench Verified | Observed | Source |
|---|---|---|---|
| Llama 4 MaverickMeta | 21.0% | 2025-07-20 | source ↗ |
— = not yet evaluated by HumanEval (no eligible arena votes). External values are the newest observation per model; hover a value for source notes and a rating for its 95% CI. Unit: %, higher is better.
Not enough overlap with our arena ratings for a correlation scatter yet — fewer than two scored models are rated.