News for hackers. Stuff that matters.

A comment on Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x. Parent comment

Re:Consistency LLM: converting LLMs to parallel decoders acc... by carlthome on Thursday May 09, 2024 @05:30AM (#40306550)
I've had the same thought as above but unfounded (just a feeling, pretty much) so I'm curious to learn more. Do you have any references I can check out that supports these claims?
[ Parent | on HN ]
Re:Consistency LLM: converting LLMs to parallel decoders acc... by int_19h on Thursday May 09, 2024 @05:52PM (#40313313)
Come up with a novel puzzle that is guaranteed to not be in the training set, and ask GPT-4 to solve it.
[ Parent | on HN ]
Re:Consistency LLM: converting LLMs to parallel decoders acc... by carlthome on Sunday May 12, 2024 @11:06AM (#40335081)
To control for that doesn't seems trivial.
[ Parent | on HN ]