News for hackers. Stuff that matters.

A comment on Apple: Transformer architecture optimized for Apple Silicon. Parent comment

Re:Apple: Transformer architecture optimized for Apple Silicon by pbronez on Thursday March 23, 2023 @07:31PM (#35283017)
Alpaca already works in just 4GB of RAM. This stuff is moving incredibly fast.
[ Parent | on HN ]
Re:Apple: Transformer architecture optimized for Apple Silicon by astrange on Thursday March 23, 2023 @08:09PM (#35283336)
It doesn't work well at all. (Also alpaca.cpp uses 8GB of RAM for a 4GB model, possibly because they don't know how mmap works.)
[ Parent | on HN ]
Re:Apple: Transformer architecture optimized for Apple Silicon by dachryn on Friday March 24, 2023 @06:06AM (#35287086)
alpaca lora is quite capable actually and is suddenly gaining a hell of a lot of traction in the corporate world
[ Parent | on HN ]