Have you tried Mistral?

Mistral is genuinely groundbreaking, for a fast, locally-hosted model without content filtering at the base layer. You can try it online here: https://labs.perplexity.ai/ (switch to Mistral)

Are more companies/teams than the creating team working to get this to copilot/chatgpt standards?

Wow I was not expecting this, It's really something else in terms of speed, and results are not bad! Will test it more

It's very fast, but it doesn't seem very good. It doesn't take instruction well (acknowledges and spits back the same wrong stuff) and doesn't seem to have much of a corpus or it's dropping most of it on the floor because it successfully answers zero of my three basic smoke-test questions.

>doesn't seem to have much of a corpus

what do you mean by 'corpus'? It is only 13GB so questions that require recalling specific facts aren't going to work well with so little room for 'compression', but asking mistral to write emails or perform style revisions works quite well for me

Are you running mistral-7B or mistral-7B-instruct?

Thanks for the link, do you know any other similar services that support fine-tuning ?

Maybe at today cost. But this is a lab about AI not just LLM. Also given that French research is underfunded since decades, our researchers are accustomed to producing good results at a budget cost.

Also has incredibly good math institutions

“French research is underfunded since decades” is a French-ism, just so you know. Normally native English speakers would say “French research has been underfunded for decades”.

Depuis ?

That can be used with either a duration (“depuis des décennies”) or a point in time (“depuis 1995”). English “since” can only be used with a point in time.

Merci!

I think “merci, mon ami français” would be more correct, but at any rate, I’m not French!

I've noticed a lot of comments on Hacker News that use this construction. I don't point it out because often people dislike unsolicited advice about their English.

Spanish speakers also make this mistake. I haven't met German speakers, but they also say the equivalent of "is (adjective) since (time)" in German. It makes me wonder if English is just unusually strict about this distinction compared to other European languages.

I am also not a huge fan of unsolicited grammatical advice, because I feel like it's both low effort and derails the conversation. If the grammar error is more syntactical in nature and doesn't obscure the meaning of the thoughts intended to be conveyed, then let's just all move on.

This is an international forum, we're well accustomed to being able to parse meaning despite a few inconsequential grammatical issues.

People can just not engage in conversations they don’t like. This forum has a branching structure, it is easy to hide a topic that you don’t like.

If someone was to be really odious and hateful, the presence of that sort of thing could be harmful, but this seems like a polite correction of a common problem. And it is an international forum, there are plenty of people who might get something from language-chat.

I pointed it out because when I lived in France, I personally appreciated people pointing out grammatical mistakes I made in French. If the OP doesn’t care about sounding more like a native English speaker, that’s perfectly fine and he can just ignore my comment.

As someone learning a language that's not my native one, people who correct my Chinese kindly are the absolute gold standard. People who correct it rudely are rare but still more helpful than people who say nothing and just move on. This is very common in language learning communities. My default assumption is that other people who are learning a language that I know natively probably feel similar, and would appreciate kind, contextual corrections. I am happy to adjust if someone lets me know they don't want that or if it's against the rules of a specific forum.

I think most European languages lack the distinction between “he did X” and “he has done X”. Actually, a lot of languages (including Spanish, German and French) have a distinction like this syntactically but it doesn’t really mean the same thing as it does in English.

Huh, I just had a fairly long and fruitless conversation with ChatGPT 4 trying to understand whether this was true. ChatGPT kept insisting that romance languages distinguish between the preterite ("he did," simple past tense) and the present perfect ("he has done"), but every time I asked it to give me specific examples it would start translating them, find that they were the same, and then say "well, in Italian [or French] it's actually more about context..."

Finally it was able to give me the Spanish "Él hizo esto" vs "Él ha hecho esto," but admitted that whether one used one vs the other was quite regional.

In French, they only use the equivalent of the simple past tense in books, but in speech they usually only ever say the equivalent of "has done" in speech no matter the context. I don't speak French (not as a second language nor as my native language), but this is something I've read about secondhand in textbooks and online.

In Spanish, they do make a distinction in neutral speech, but it's not uncommon for someone to only ever use one or the other depending on the region they're from. The auxiliary verb form ("haber hecho" / "to have done") is more common in Spain than Latin America. Though, I don't know how much we should go into this; I feel like we've kind of derailed from the original topic.

I wonder if it is just the result of applying some transformation to “was underfunded for decades” in an attempt to make it cover the present.

A Gallicism :)

Thanks, I knew there was a word I was forgetting…

If they’re researching avenues other than LLMs, that money goes a lot further.

We can only hope.

During the conference where it was announced, they indicated they have a partnership with Scaleway to access their Nabu 2023 supercomputer[0]. I expect it will be similar to the relationship between OpenAI and Microsoft Azure, with free credits as part of their investment in exchange for being at the forefront of AI datacenter design. Indeed, Xavier Niel is one of three investors in Kyutai, and founded Scaleway.

[0]: https://www.scaleway.com/en/news/scaleway-releases-the-detai...

Maybe related:

EU is starting a program where AI startups will get access to their supercomputers and also get a chance to win prizes.

16 November 2023: Commission opens access to EU supercomputers to speed up artificial intelligence development [0]

- Launch of the Large AI grand challenge: This competition – launched today, is a collaboration led by the EU funded project AI-BOOST, with access to the European Supercomputers being facilitated by the EuroHPC JU. It encourages the wide participation from European start-ups with experience in large-scale AI models. The winners are expected to release the developed models under an open-source license for non-commercial use, or through publishing their research findings. The challenge will select up to four promising European AI start-ups that will be given access to EuroHPC supercomputing facilities to foster the development of large-scale AI models in Europe and a €1 million prize will be distributed among the winners.

- Opening up European supercomputer capacity: Access will be established for ethical and responsible AI start-ups, enabling them to efficiently train their models using European supercomputers.

- Enhanced activities and services: the EuroHPC JU will advance activities and services powered by High-Performance Computing to foster trustworthy AI in Europe. These efforts will aim to facilitate increased accessibility for AI communities and promote the optimal and efficient use of HPC technologies for scientific and industrial innovation.

[0] https://ec.europa.eu/commission/presscorner/detail/en/ip_23_...

First they set up massive barriers such that you’d be insane to start an AI company in Europe. And then they dole up subsidies.

Makes no sense. That money will go to waste.

What massive barriers are there?

> What massive barriers are there?

Starting and scaling a business in Europe is harder than in America. This is true for fundamental reasons, like language and national borders. It’s also true due to regulation and the culture of work and towards commerce in general.

Some of those come with reasonable tradeoffs, e.g. in respect of employee protections. Many do not, particularly when it comes to licensing, bureaucracy and the peculiar way most European tax law and regulation tries to compensate for its licensing and bureaucracy by adding more bureaucracy in front of a balancing subsidy.

Opinion of a german here: Build & break things (without caring for laws or implications) is what the US and their companies do. Often it works, and sometimes breaking the law makes a new law pop up to legalize it, but that's not a good thing.

Yes, it's a big annoying to start a company in some EU countries. No, it's not as bad as HN claims.

Working for a big American company.. we have to abide by a lot of laws, including the EU ones.

Small companies can get away with it for awhile, but not forever

Requiring companies to offer more than two weeks of paid time off per year.

Sorry, could somebody show a source for that? An $11M bill, that’s nuts and just need a reference lol

"But to put things into perspective, GPT-3 175B model required 3.14E23 FLOPS of computing for training. Even at theoretical 28 TFLOPS for V100 and lowest 3 year reserved cloud pricing we could find, this will take 355 GPU-years and cost $4.6M for a single training run."

https://lambdalabs.com/blog/demystifying-gpt-3/

There's some compute also, which may not be counted in the 330M

"Co-founded by the Iliad group...To do this, the laboratory will use the computing power made available to it by Scaleway, an iliad Group subsidiary. Scaleway’s supercomputer has the highest-performance computing power for AI applications deployed to date in Europe."

https://kyutai.org/CP_Kyutai_AI_EN.pdf

Seems like it could be unlimited if people just bought hardware

> $330M is like 33 training runs of a large LLM. Not sure that would bring open source anywhere near GPT-3.5.

Open source is already quite near GPT-3.5; it's reaching GPT-4 level that is the challenge.