News for hackers. Stuff that matters.

A comment on Practical Llama 3 inference in Java.

Re:Practical Llama 3 inference in Java by mukel on Thursday May 16, 2024 @07:50PM (#40384763)
Llama3.java: featuring .GGUF file format support, Q8_0 and Q4_0 quantizations, fast matrix/vector multiplication routines using Java's Vector API; served by a simple CLI with a --chat mode to interact with the Llama 3 models.
[ Parent | on HN ]

There are no comments on this comment yet.