cocktailpeanut

Hi guys, I really didn't think that it would move THIS fast, but it did, and we now have Alpaca.cpp.

As soon as I saw Alpaca.cpp, I started working on Dalai's JavaScript integration, and today I'm happy to release Dalai Alpaca.

Dalai is now kind of like a package manager of sorts, just one command to install LLaMA or Alpaca, and all integrated with the web UI and the ultra-hackable API.

all you need to do is one command:

npx dalai install alpaca 7b

Also, been fixing a lot of bugs since last time so if you had trouble installing, I recommend you try again.

Officially supports Windows, Mac and Linux. Still some quirks here and there but mostly works and happy to figure out if you are having trouble. Appreciate any feedback!

UPDATE 2:

Thanks to all the pull requests, we've managed to solve most of these issues in the most optimal manner.

Version 0.1.0 released: https://news.ycombinator.com/item?id=35143171

Hi HN!

I posted Dalai on HN yesterday (https://news.ycombinator.com/item?id=35127020) and I have been spending the last 24 hours trying to listen to all the feedback and incorporate all the incoming pull requests and make a new release.

  1. If you had trouble installing and they were just not installing for some reason, make sure you try this version.
  2. If you had trouble working with models other than the 7B one, yes it was my fault, only the 7B one was working, and now this has been fixed, and now ALL models should work with this new version
  3. Use existing workspace: If you already have an existing llama.cpp workspace folder, now Dalai can connect to it programmatically instead of creating its own repository under ~/dalai. Remember, while the easy installation is important, the most important aspect of Dalai is that it lets you interact with llama.cpp with JavaScript. So even if you already have an existing llama.cpp and just want to play with it using JS, you can use Dalai.
  4. New Web UI

I don't want to ramble on and on here so will link a thread where I discussed this in more detail, for those who are interested in learning more: https://twitter.com/cocktailpeanut/status/163539451761565286...

Finally, like 90% of the code for this release came from people other than myself and it's amazing how this happened. Thank you everyone who contributed.

There are some further issues I think need to be addressed. If you have any feedback that hasn't been addressed with this release, please let me know!

and if you want to follow along with the development, you can find me on Twitter https://twitter.com/cocktailpeanut

I am going to test this out today and roll this out as soon as I can, hopefully tomorrow. stay tuned.

UPDATE:

Thanks for all the feedback! I went outside to take a walk after posting this and just came back, and went through them to summarize what needs to be improved.

Basically looks like it comes down to the following:

  - add customize features: Should not be difficult (will add flag features)
    - path: customize the home directory (instead of automatically storing to $HOME)
    - python: some people are having issues with the python binary (since the package is essentially calling these shell commands). Maybe add a flag to specify the exact name of the python binary (such as "--python python3")
    - avoid downloading files: I have this issue too when I just want to install the code instead of downloading the full model which takes a long time. Might add a flag to avoid downloading models in case you already have them
  - documentation
    - document the machine spec
    - document the storage spec: how much space is used?
    - node version: which version of node.js is required?
    - python version: which version of python doesn't work?

Am I missing anything? Feel free to leave comments, will try to roll out some updates as soon as I can. To stay updated, feel free to follow me on twitter https://twitter.com/cocktailpeanut (or you could create issues on GitHub too!)

Hey guys, I was so inspired by the llama.cpp project that I spent all day today to build a weekend side project.

Basically it lets you one-click install LLaMA on your machine with no bullshit. All you need is just run "npx dalai llama".

I see that the #1 post today is a whole long blog post about how to walk through and compile cpp and download files and all that to finally run LLaMA on your machine, but basically I have 100% automated this with a simple NPM package/application.

On top of that, the whole thing is a single NPM package and was built with hackability in mind. With just one line of JS function call you can call LLaMA from YOUR app.

Lastly, EVEN IF you don't use JavaScript, Dalai exposes a socket.io API, so you can use whatever language you want to interact with Dalai programmatically.

I discussed a bit more about this on a Twitter thread. Check it out: https://twitter.com/cocktailpeanut/status/163504032247148953...

It should "just work". Have fun!

It's free. there's extremely cheap, and there's free. no matter how extremely cheap something is, "free" is on a completely different level and gives us a new assumption that enables a lot of things that are not possible when each request is paid (no matter how cheap it is)