News for hackers. Stuff that matters.
Re:A 10 year old Xeon is all you need (for 26B-A4B MTP Draft... by dvdkon on Monday June 01, 2026 @08:23AM (#48355890)
As far as I know, speculative decoding still verifies that the proposed tokens are what the "big" model would generate, it just uses the guesses to make that process faster. Setting the probability threshold too low then shouldn't affect correctness, just speed (time will be wasted verifying bad guesses).
Re:A 10 year old Xeon is all you need (for 26B-A4B MTP Draft... by lreeves on Monday June 01, 2026 @08:26AM (#48355916)
But won't setting it to accept 100% of the proposed tokens will skip the verification?