anushka_gandhi

For 4bit, it takes you 5-10 minutes to compress a model. With this, you can do that in seconds. You do not have to worry about tokenizer configuration or anything. Just one-click compress and run.

This is our first version and yes, it will have some errors that will be fixed in the upcoming versions.

No, you require BF16 model weights for this quantization to work. If you do run it on 4bit or 1.5bit, it will degrade the model and it will hallucinate a lot.

The website was made with Gemini. Not a big html coder. I was a bit busy on doing research on things that matter rather than focusing on UI

Correct, you will be able to compress, load and talk with any AI models. This algorithm compressed all model weights by upto 3x.

Yes, you can use it on trillion parameter class AI models.

Maybe it will be better if you focused more at the research rather than doing such things. Our website it not optimised for mobile users as it has more explanation to do rather than just make visitors feel good. Good luck with the things you are going.

We have tested it on all kinds of small models including LFM3-1.7B. Our goal is to make a a system that can compress any AI. This is our first version and with your feedback we can improve.

The front/worker agent is exactly how this should be done,monolithic agents are way too slow for UI.For the remote issue,what if you exposed the worker's inbox via Tailscale? You could just text tasks to your mac while on the go.