News for hackers. Stuff that matters.
Re:Show HN: GPU-Accelerated Inference Hosting by spullara on Tuesday July 13, 2021 @03:18AM (#27818353)
Well, I guess I know where I am going to host GPT-J-6B then. I don't think it is sustainable.
Re:Show HN: GPU-Accelerated Inference Hosting by marcooliv on Tuesday July 13, 2021 @08:37AM (#27820154)
How are you planning to put a gpt whatever when the service clearly have a model size limit?!
Re:Show HN: GPU-Accelerated Inference Hosting by spullara on Tuesday July 13, 2021 @08:51PM (#27828178)
The size limit is very close to allowing it (12GB vs 10GB). I imagine you can reduce it somewhat further and get it to fit.