News for hackers. Stuff that matters.
Re:Locally hosted LLM use cases? by thiag0 on Tuesday June 27, 2023 @01:29PM (#36496075)
May I ask you how fast is gpt4all running on your laptop? I've tried to run locally on my desktop and it worked really well, however the "response" time it's pretty slow.
Re:Locally hosted LLM use cases? by thensome on Tuesday June 27, 2023 @08:30PM (#36500684)
Yeah, it's still pretty slow and can't take large inputs very well. I'd say it takes 30sec to a minute to process a 200-300 word selection.