The Llama 4 herd

Re:The Llama 4 herd by anotherpaulg on Sunday April 06, 2025 @04:59PM (#43604919)

Llama 4 Maverick scored 16% on the aider polyglot coding benchmark [0].

  73% Gemini 2.5 Pro (SOTA)
  60% Sonnet 3.7 (no thinking)
  55% DeepSeek V3 0324
  22% Qwen Max
  16% Qwen2.5-Coder-32B-Instruct
  16% Llama 4 Maverick

[0] https://aider.chat/docs/leaderboards/?highlight=Maverick

[ Parent | on HN ]

Re:The Llama 4 herd by phoenk on Sunday April 06, 2025 @05:26PM (#43605097)

Did they not target code tasks for this LLM, or is it genuinely that bad? Pretty embarrassing when your shiny new 400B model barely ties a 32B model designed to be run locally. Or maybe is this a strong indication that smaller, specialized LLMs have much more potential for specific tasks than larger, general purpose LLMs.

[ Parent | on HN ]

Re:The Llama 4 herd by senko on Monday April 07, 2025 @05:15AM (#43609414)

Side note: `highlight` query param doesn't seem to have any effect on that table (at least for me on Firefox)

[ Parent | on HN ]