News for hackers. Stuff that matters.

A comment on Running a 180B parameter LLM on a single Apple M2 Ultra. Parent comment

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by ZoomerCretin on Thursday September 07, 2023 @02:11PM (#37423170)
GPT4 is still leagues ahead of the competition. Open source LLMs will be used more widely, but for the most demanding tasks, there is no alternative for GPT4.
[ Parent | on HN ]
Re:Running a 180B parameter LLM on a single Apple M2 Ultra by eurekin on Thursday September 14, 2023 @05:24AM (#37506701)
Anecdata confirmation: I've been toying around with LLMs for simple fun stuff, but when it comes to real work, GPT-4 delivers in spades.

I have cut many hours of debugging thanks to it. I could find issues easily, on-call in short conversation, when previously that was reserved as post mortem task.

Even reading documentation is nothing like before: once, I was looking for a single command to upload and presign a object in S3. SDK has tens of methods, which require careful scanning, if they do what I want. Going through documentation thoroughly would've taken me hours. GPT-4 simply found, no, there's no operation for that immediately.

[ Parent | on HN ]