News for hackers. Stuff that matters.
Re:Alternative clouds are booming as companies seek cheaper ... by nextworddev on Monday May 06, 2024 @03:02PM (#40278268)
Taking a step back, was it necessary to roll your own LLM (or whatever FM) API as opposed to using an off the shelf API
Re:Alternative clouds are booming as companies seek cheaper ... by zackangelo on Monday May 06, 2024 @08:27PM (#40281018)
Yes we’re running our own inference server because we do nonstandard things in the decoder loop (think logit masking, other types of sampling, etc).