News for hackers. Stuff that matters.

zhisbug

Re: on Tuesday March 17, 2026 @05:10PM (#47418370)
Re: on Monday August 04, 2025 @06:18PM (#44791968)
Re: on Monday June 30, 2025 @04:35PM (#44427570)
Pokémon is increasingly used to evaluate modern large language models, but current practices lack standardization, and depend heavily on game-specific harness. The Pokémon Red involves three major tasks—navigation, combat control and training a competitive Pokémon team. We find they come with limitations: navigation tasks are too hard, combat control is too simple, and Pokémon team training is too expensive. We address these issues in Lmgame Bench, a new framework offering standardized evaluations and initial results across diverse games.
Re: on Friday June 13, 2025 @05:46PM (#44272589)
where other models tops out in a few moves
Re: on Tuesday April 08, 2025 @05:27PM (#43626642)
Re: on Thursday March 06, 2025 @07:06PM (#43286323)
We find that spatial perception and spatial reasoning remain very difficult even for the strongest models like o3 or Claude 3.7
Re:Fast Video Generation with Sliding Tile Attention on Tuesday February 18, 2025 @07:19PM (#43096929)
Sliding tile attention accelerates Hunyuan video generation by 3x with no quality drop and no need for training
Try our demo and let us know
Re:Evaluating LLM Reasoning Through Live Computer Games on Wednesday February 12, 2025 @07:22PM (#43031366)
This is pretty clever and seems to have high potential, but it still relies on humans. What if some day all humans cannot outsmart AI?
please try and give us feedback!
We hope to redefine ai evaluation via our gamified AI evaluation platform: game arena!
Re:Efficient LLM Scheduling by Learning to Rank on Tuesday January 14, 2025 @04:11PM (#42703921)
We study how to approximate the famous shortest-job-first scheduling in LLM inference!
Hugginface model and data link: https://huggingface.co/FastVideo
New work from the vLLM team that disaggregates prefill and decoding to maximize goodput (throughput subject to latency constraints) in LLM serving
New parallel decoding algorithm that trades flops for latency reduction

Older »