idiliv

Switzerland has a strong startup scene. The country also ranks first in the global innovation index.

All voices offered sound human. I'd prefer a robotic voice, to avoid over-anthropomorphizing the AI.

What is the "compilers argument"?

Human verification of the Lean program only requires verifying that the theorem itself is represented correctly. The theorem will only make up a very small part of the entire Lean program.

Uber is likely on an enterprise plan - these charge tokens at API cost, which can be much more expensive than the $20 flat rate.

Sometimes model developers coordinate with inference platforms to time releases in sync.

Wait, but we're doing that already, and it works well (Qwen 2.5 VL)? If need be, you can always resort to structured generation to enforce schema conformity?

Duplicate, posted on October 9: https://news.ycombinator.com/item?id=41784591

Where do you see the MMLU-Pro evaluation for Llama 3.2 90B? On the link I only see Llama 3.2 90B evaluated against multimodal benchmarks.

Is the "Ultra Deep" analysis worth it over the standard "Deep" analysis?

In the demo, O1 implements an incorrect version of the "squirrel finder" game?

The instructions state that the squirrel icon should spawn after three seconds, yet it spawns immediately in the first game (also noted by the guy doing the demo).

Edit: I'm referring to the demo video here: https://openai.com/index/introducing-openai-o1-preview/

How are flexible working hours equivalent to more money?

You can rent them online for ~ 4-5 $ per hour per GPU. Not cheap, but definitely feasible as a weekend project.

Just tried this again and I also arrive at 16.92B. Not sure what I did wrong the first time, thanks for double-checking this!

Oh, and to answer your actual question: Assuming that the model is released with 16 bits per parameter, then it as 281GB / 16 bit = 140.5 parameters.

In Mixtral 8x7B, the 8 means that the model uses Mixture-of-Experts (MoE) layers with 8 experts. The 7B means that if you were to remove 7 of the 8 experts in each layer, then you would end up with a 7B model (which would have exactly the same architecture as Mistral 7B). Therefore, a 1x7B model has 7B params. An 8x7B model has 1 * 7B + (8-1) * sz_expert params, where sz_expert is some constant value that the MoE layers increase by when adding one expert. In the case of Mixtral 8x7B the model size is 46.3GB, so, sz_expert ≈ 5.6B.

If these assumptions port over to 8x22B, then 8x22B has, at 281GB, sz_expert ≈ 13.8B.

Hi Martin! It's Robert from Cambridge (you were my DOS :)). Glad to see your name pop up on HN!

People here seem mostly impressed by the high resolution of these examples.

Based on my experience doing research on Stable Diffusion, scaling up the resolution is the conceptually easy part that only requires larger models and more high-resolution training data.

The hard part is semantic alignment with the prompt. Attempts to scale Stable Diffusion, like SDXL, have resulted only in marginally better prompt understanding (likely due to the continued reliance on CLIP prompt embeddings).

So, the key question here is how well Sora does prompt alignment.

Hmm, are you sure that translations of LLMs like ChatGPT are not incorporating cultural context?

I'm curious how they evaluated model quality. The only information I could find is "Quality: Index based on several quality benchmarks".

They could join Mistral AI, which has published weights for at least some of its models. Another option is Meta AI, which has published weights for Llama and Llama 2.

Parent post is talking about LLMs, i.e. Large LMs. Research on LLMs is indeed in its infancy.

When I try out the topics you suggest at the huggingface endpoint you link, the answer is either my question translated into Chinese, or no answer when I prompt the model in Chinese:

<User>: 历史上的“天安门广场的坦克人”有什么故事？ <Assistant>:

I've tried out DeepSeek on deepseek.com and it refuses conversations about several topics censored in China (Tiananmen, Xi Jinping as Winnieh-the-Pooh).

Has anyone tried if this also happens when self-hosting the weights?

"Each atomic step would normally take over 5,000 CPU hours on a supercomputer. Now, we can do the same calculation in 2 milliseconds on a desktop,"

Is this phrase equivalent to "Each atomic step would take 5,000 hours on a desktop. Now, it takes 2 CPU milliseconds on a supercomputer."? ^^

I happen to take this train on a regular basis, and it is reliably delayed :-).

Conveniently, my connecting train is also usually sufficiently delayed so that I do not miss it.