News for hackers. Stuff that matters.
Re:MK-1 by polishgladiator on Saturday August 05, 2023 @09:57PM (#37018232)
Based on the integration examples, I don't think they are simply repackaging llama.cpp

Rather it looks like they are reimplementing their own quantization scheme, in such a way that it is a little easier to integrate for basic python users, at the cost of performance (compared to llama.cpp and others).

Given that the bar for integrating something with higher perf like llama.cpp isn't very high (and that's the way the world is heading -- ask any 15 year old interested in this stuff), I can't see anything of value here.

Re:MK-1 by Lindon4290 on Sunday August 06, 2023 @11:00AM (#37022695)
Looks their performance is better than llama.cpp - https://news.ycombinator.com/item?id=37018989 - and scales to batches of prompts.
Re:MK-1 by polishgladiator on Sunday August 06, 2023 @01:33PM (#37024483)
Actually no -- that post shows they are not performing measurements and comparisons correctly.

These are not serious people.