News for hackers. Stuff that matters.

A comment on Σ-GPTs: A New Approach to Autoregressive Models. Parent comment

Re:Σ-GPTs: A New Approach to Autoregressive Models by RivieraKid on Friday June 07, 2024 @04:06PM (#40612433)
Are there any obvious practical application of this algorithm for existing large (10B+) text / image models?

Does the rejection sampling lead to a statistically correct sample from the joint probability distribution or is that just a (possibly rough) approximation?

[ Parent | on HN ]
Re:Σ-GPTs: A New Approach to Autoregressive Models by arnaudpannatier on Saturday June 08, 2024 @08:30AM (#40617105)
For the application: being able to prompt anywhere in the sequence can be of interest. For what we've seen in the experiment, the rejection sampling leads to similar generation than the autoregressive one, we did not see any mode collapse or anything of that kind.
[ Parent | on HN ]