News for hackers. Stuff that matters.
Re:Llamafile lets you distribute and run LLMs with a single ... by NoOn3 on Thursday November 30, 2023 @11:55AM (#38475803)
Got this: Terminating on uncaught SIGILL.
Re:Llamafile lets you distribute and run LLMs with a single ... by jart on Thursday November 30, 2023 @12:05PM (#38475924)
I'm assuming you're on Apple Silicon? Please follow https://github.com/Mozilla-Ocho/llamafile/issues/11 which is tracking this. We've received multiple reports even though I personally haven't figured out how to reproduce it yet.
Re:Llamafile lets you distribute and run LLMs with a single ... by NoOn3 on Thursday November 30, 2023 @12:21PM (#38476122)
No just old i5-2500k 16GB Ram and vega 56 GPU 8GB Vram.
Re:Llamafile lets you distribute and run LLMs with a single ... by NoOn3 on Thursday November 30, 2023 @12:32PM (#38476271)
I suspect it's lack of AVX2 support in my cpu.
Re:Llamafile lets you distribute and run LLMs with a single ... by jart on Thursday November 30, 2023 @05:46PM (#38480312)
So you have a Sandybridge processor with AVX support (but not AVX2). Could you open llamafile in a debugger and find out which instructing is faulting and what its address in memory is? I haven't personally tested Sandybridge but I'm reasonably certain we designed the build to not insert any incompatible instructions in there. Our intent is to support you. I also might be able to fish my old ThinkPad out of the closet if you don't have time. In any case it'll happen soon.
Re:Llamafile lets you distribute and run LLMs with a single ... by NoOn3 on Friday December 01, 2023 @11:40AM (#38488750)
I tried but gdb showed me nothing, this screenshot(https://pxscdn.com/public/m/_v2/97422265439883264/bc40e5d2a-...). gdb layout asm shows "No Assembly Available". Maybe I'm not so skilled in debbuging such programs it's running under wine it seemed to me.
Re:Llamafile lets you distribute and run LLMs with a single ... by jart on Saturday December 02, 2023 @04:04AM (#38497199)
Check out the llamafile 0.2.1 release. Old Intel CPU support is fully fixed now on the release page and hugging face. Enjoy!
Re:Llamafile lets you distribute and run LLMs with a single ... by NoOn3 on Saturday December 02, 2023 @05:36AM (#38497601)
Thanks a lot.