Re:ChatGPT vs. open source on harder tasks
by thomasahle on Thursday May 18, 2023 @04:30PM
(#35993655)
You don't need non-determinism. If you know you are going to generate a number, just mask the output of the softmax layer to only decode number tokens. That's exactly what stuff like Guidance and Outlines (https://twitter.com/remilouf/status/1653765371273330688) do.
[ Parent
| on HN ]
Re:ChatGPT vs. open source on harder tasks
by boywitharupee on Friday May 19, 2023 @08:21AM
(#36000941)
How do you mask the output of the softmax layer? The link you referenced doesn't provide specific details.