minxomat

Experimenting with and/or reading about ChatGPT and then interacting with Siri feels almost offensive now. All of the assistants still suck - get on it AmaGooPple!

Is this related in any way to the actual ChatLLama project that is developed here: https://github.com/nebuly-ai/nebullvm ?

The former. Makes sense for their business model.

> The GPU version is commercial software. Please contact...

Shame.

Some numbers here: https://github.com/qwopqwop200/GPTQ-for-LLaMa#result

Follow up recommendation: for Spanish, “Madrigals Key” is language transfer in book form. It can in fact be used as a workbook for this course.

Sure. Sent a ping.

We're not far off: https://imgur.com/a/nJxxcUg

A plot point in an episode of Person of Interest, where AGI "Samaritan" funds a charity (free tablets for students, Samaritan access pre-installed) to take over education and recruit more mercenaries.

I'm building this, for (mostly) non-scientific non-fiction works (books, articles, news, etc.). Launching soon, with about 7,500 books indexed.

Generally, what I found useful to build a graph between "topics" or entities was to use a HyDE[1] prompt to generate possible distinct definitions and then build a nearest-neighbor network from that. This successfully identifies related concepts in the truly abstract sense rather than literal entities.

[1] https://summarity.com/hyde

For me the hangup was always the hidden rule: host won’t open a door with a car. That is unstated and remains unstated even in modern discussions of the problem (see Pinker’s “Rationality”). Once explicitly states the outcome becomes intuitive.

The whole point of the LLaMa paper is that large models are undertrained and oversized.

A laptop and a desktop (Mac Studio)

This has been fixed almost 2 days ago now. It’s literally mentioned at the top of the repo.

The full use case includes quantisation, which the repo points out uses a large amount of system RAM. Of course that’s not required if you skip that step.

You’re missing something. Both SHP (https://huggingface.co/datasets/stanfordnlp/SHP) and OpenAssistant datasets are referenced.

And the TOS violation might be the case, the project nevertheless has a mode to use OpenAI in the fine tuning steps.

With 16 threads, about 140ms per token for 30B, 300ms per token for 65B

I should also mention that 65B should be able to run on 64GB systems. Total system memory consumption on M1 Ultra is about 67GB when running nothing else.

No catch, just works. 30B works fine on an M1 Max with 64GB of RAM, had to go for the M1 Ultra at 128GB for 65B.

So if I'm reading this right, 65B at 4bit would consume around 20GB of VRAM and ~130GB of system RAM?

There are open datasets (see the chatllama harness project and its references). You can of course also cross train it using actual ChatGPT.

AdGuard

> not really competitive with ChatGPT

That's impossible to judge. LLama is a foundational model. It has received neither instructional fine tuning (davinci-3) nor RLHF (ChatGPT). It cannot be compared to these finetuned models without, well, finetuning.

GPT-3 might be the ultimate contender for https://en.m.wikipedia.org/wiki/Hutter_Prize - now if we could only find the prompts to recover it all

Everything old is new again … https://en.m.wiktionary.org/wiki/juvenoia

Not to forget the Toshiba AC100 with the original Tegra, which was one of the first usable ARM laptops (netbooks?) and helped much of Linux desktop support development for ARM in the early days.

Not really, that’s exactly what Papertrail does.

Would love to use an API, per token pricing is a good approach (with use limits like OpenAI). If you need testers, I have some use case (long form non-fiction content). LMK at ml[at]summarity[dot]com

I second this, API please.