Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by boredatoms on Wednesday September 09, 2026 @01:50PM
(#49630447)
For sure they quickly move to q8, the output quality difference to bf16 is small compared to the speed/capacity gain
[ Parent
| on HN ]
Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by torginus on Wednesday September 09, 2026 @04:02PM
(#49633351)
Some people here have remarked previously that while reduced precision doesn't show up in quick prompts, it does severely impact these models' ability to perform long running tasks - to the point that running these big models with severe quantization might be counterproductive as smaller but less quantized ones perform better.
Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by NineStarPoint on Wednesday September 09, 2026 @02:23PM
(#49631165)
Yeah q8 made so littler difference back when I was testing such things I'd be surprised if people could quickly notice that as a change. It's got to be either further quantized or some other type of optimization that kicks in when people notice the drop.
Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by selectodude on Wednesday September 09, 2026 @03:11PM
(#49632434)
NVFP4 would buy them a huge increase in capacity but I think it would be noticeable.
Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by Caracas288 on Wednesday September 09, 2026 @03:55PM
(#49633225)
Why doesn't someone just try to measure this next time!?
Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by embedding-shape on Wednesday September 09, 2026 @06:33PM
(#49635527)
Can't really measure without being sure you aren't being messed around with, when it's a remote platform. Stupidly easy to detect when people run such benchmarks/tests against you as well.
Re:GPT-6 Astra, Looped Transformers, and Hidden Reasoning
by nonethewiser on Wednesday September 09, 2026 @03:33PM
(#49632850)
Could this explain Opus?