Re:Alternative clouds are booming as companies seek cheaper ...
by nextworddev on Monday May 06, 2024 @03:02PM
(#40278268)
Taking a step back, was it necessary to roll your own LLM (or whatever FM) API as opposed to using an off the shelf API
[ Parent
| on HN ]
Re:Alternative clouds are booming as companies seek cheaper ...
by zackangelo on Monday May 06, 2024 @08:27PM
(#40281018)
Yes we’re running our own inference server because we do nonstandard things in the decoder loop (think logit masking, other types of sampling, etc).