GGML – AI at the Edge

Re:GGML – AI at the Edge by Tan-Aki on Tuesday June 06, 2023 @10:55PM (#36222313)

Can anyone explain to me, in simple terms, and at a high level, what the heck am I looking at? What is this library for? What does it mean "it is used by lama.cpp and whisper.cpp"? How is it revolutionary? Thank you very much in advance!

[ Parent | on HN ]

Re:GGML – AI at the Edge by Tan-Aki on Tuesday June 06, 2023 @11:00PM (#36222346)

With maybe a tiny bit of history as well? Pretty please? : p

[ Parent | on HN ]

Re:GGML – AI at the Edge by orost on Tuesday June 06, 2023 @11:48PM (#36222597)

ggml is a library that provides operations for running machine learning models

llama.cpp is a project that uses ggml to run LLaMA, a large language model (like GPT) by Meta

whisper.cpp is a project that uses ggml to run Whisper, a speech recognition model by OpenAI

ggml's distinguishing feature is efficient operation on CPU. Traditionally, this sort of work is done on GPU, but GPUs with large amounts of memory are specialized and extremely expensive hardware. ggml achieves acceptable speed on commodity hardware.

[ Parent | on HN ]

Re:GGML – AI at the Edge by Tan-Aki on Wednesday June 07, 2023 @07:45AM (#36225610)

Thank you so much for your kindness Orost. Sharing really IS caring. I understand.

May good things happen to you. Peace.

[ Parent | on HN ]

Re:GGML – AI at the Edge by Tan-Aki on Wednesday June 07, 2023 @07:44AM (#36225600)

Thank you so much for your kindness Orost. Sharing really IS caring. I understand.

May good things happen to you. Peace.

[ Parent | on HN ]