HumanEval.org

Changelog

Ranking updates, roster changes and articles — methodology changes are always announced here.

  1. new model

    Model added: Kimi K3

    Kimi K3 (Moonshot AI) joined the roster — https://humaneval.org/models/kimi-k3

  2. new model

    Model added: Qwen3.8 Max (0902)

    Qwen3.8 Max (0902) (Alibaba Qwen) joined the roster — https://humaneval.org/models/qwen3-8-max-0902

  3. new model

    Model added: Mock Beta

    Mock Beta (internal) joined the roster — https://humaneval.org/models/mock-beta

  4. new model

    Model added: Llama 4 Maverick

    Llama 4 Maverick (Meta) joined the roster — https://humaneval.org/models/llama-4-maverick

  5. new model

    Model added: Grok 4.6

    Grok 4.6 (SpaceXAI) joined the roster — https://humaneval.org/models/grok-4-6

  6. new model

    Model added: DeepSeek V4 Pro (0813)

    DeepSeek V4 Pro (0813) (DeepSeek) joined the roster — https://humaneval.org/models/deepseek-v4-pro-0813

  7. update

    Browser scene promoted: c3b-walk vetted scene

    The community-authored Browser Use scene “c3b-walk vetted scene” (v1) was reviewed and promoted to the curated pool. Battles on it now count toward ratings.

  8. new model

    Model added: GPT-5.6 Luna

    GPT-5.6 Luna (OpenAI) joined the roster — https://humaneval.org/models/gpt-5-6-luna

  9. new model

    Model added: Gemini 3.8 Flash

    Gemini 3.8 Flash (Google) joined the roster — https://humaneval.org/models/gemini-3-8-flash

  10. new model

    Model added: Claude Sonnet 5

    Claude Sonnet 5 (Anthropic) joined the roster — https://humaneval.org/models/claude-sonnet-5

  11. new model

    Model added: GPT-5.6 Terra

    GPT-5.6 Terra (OpenAI) joined the roster — https://humaneval.org/models/gpt-5-6-terra

  12. new model

    Model added: Claude Opus 5

    Claude Opus 5 (Anthropic) joined the roster — https://humaneval.org/models/claude-opus-5

  13. new model

    Model added: Claude Fable 5.1

    Claude Fable 5.1 (Anthropic) joined the roster — https://humaneval.org/models/claude-fable-5-1

  14. new model

    Model added: GPT-6 Astra

    GPT-6 Astra (OpenAI) joined the roster — https://humaneval.org/models/gpt-6-astra

  15. article

    Article: How HumanEval.org ratings work

    New article published: "How HumanEval.org ratings work" — https://humaneval.org/research/how-humaneval-ratings-work