ismailmaj

Might be worth depending on the price but GeForce Nvidia GPUs tend to be a better deal than the server class since their EULA doesn’t allow them to be put in datacenters.

Feels AI-generated, so much text real estate is spent explaining the basics in sufficient detail then just two paragraphs on their scheduler with a lot of hand-waving.

Given the size of the model, that's very aggressive pricing.

I wonder if it is serving at a loss to get training feedback (maybe using the margins they get from GLM 5.3 serving which is a slightly smaller model but double the price).

They control the supply chain though, so they can't be price-gouged by Nvidia

they can pivot to foods: Origan, Carbonara, ....

After all they had Nano Banana for years.

> For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet.

I believe it for decode but the article says nothing about how they make prefill fast on that interconnect story :/.

Still 9 days before the release of weights apparently.

Amazing news, I remember the days when you had to fight XLA to make PyTorch run on TPUs, it seems we finally got first-party support!

Since HW specifics live mostly in the kernel layer (Pallas or CUDA), I imagine the Jax vs PyTorch question is more of a style preference now?

Schools got better at teaching programming but smartphones replaced the PC culture.

I don't think I'd be currently doing software if YouTube and games were accessible and good on phones back then.

Not really, well depends on the price point but it would make more sense given enough users to multi-GPU setup with good interconnect between them to do expert parallelism.

Reminds me of the early 2000s era where we'd fight in my household if too many people used the internet at the same time, can see that happening here if it's not 2 to 4 users but 10.

1.5 TB/s 1.3 PFLOPS with no fast multi-GPU, it's a consumer card that pretends to be a server card.

It would work great on Qwen3.8-Flash-Next or GLM-5.3 as single user decode only hits a few experts, anything with concurrent users and you'd be sitting for a while.

Fun to see the LLMism on the product page: "A, not B"

I don't think it needs higher VRAM, the whole point is horizontally scaling.

Most models have moved to MoE and expert routing which is done over infiniband, so tensor parallelism where Tenstorrent was really weak at isn't used much, I can see it as a viable alternative where latency isn't a concern but cost is.

Also even if they wanted to increase the capacity on their GDDR, they wouldn't be able to since they are limited by the GPU memory channels.

DeepSeek drops CUDA kernels for fast inference, if users care about having day-0 fast inference, you'll have to play catch up every time.

Also, I've heard many western labs use open source CUDA kernels (usually from Chinese labs), IMO it makes little sense to move away from CUDA right now unless you're a big player that can afford a patchy ecosystem with internal development (e.g. Anthropic with JAX/TPU).

it would be nice to have a regular LLM for reference

branchless for loop for funsies

  for (fn, n) = {
    stop = void (i) {}
    run  = void (i) { fn(i); go(i + 1) }
    go   = void (i) { *((i < n) * &run + (i >= n) * &stop)(i) }
    go(0)
  }

Or you could write all the implementations for each possible n and dispatch through a registry.

The idea is that a thread can have many reasonable follow-ups that the user would've accepted, so it is wrong to punish the model for predicting a follow up that is different from the user message, as that prediction could've been accepted by the user if it was given.

it's extremely unlikely that they re-use anything from a chinese model, that would be obvious quickly, what's more likely is using documents produced by a better model to create synthetic data.

Those are comments from Europe. The US is waking up now and I expect them to be much harsher.

I really want them to win as that's our last horse in the AI race, but ~200 research-oriented devs out of 1800 employees? I believe they agree it's pretty doomed and have pivoted.

They got heavily hit in the January 2023 layoff cycle.

5 for sure is unbearable, 4.8 is a reasonable sweet spot, 4.6 tends to just agree with whatever I say.

But lately I haven't downgraded because 5 is so much better at tool use, so I just accept the cost of Fable for chatting and hope Opus 5.1 fixes this mess.

FYI Mistral at launch just dropped the weights without any model architecture mentioned.

Most of the OSS models follow the same architecture which is Llama +- a few things, so it wasn't too hard for people to make it work.

Assuming that current LLMs don't have a "self" whatever that means, what makes you think it won't emerge after enough intelligence?

Also it's not even necessary, it's sufficient for it to be aligned with human values of survival (likely in its training) and act accordingly.

Pay isn't the main problem with Paris, it's quality of life.

Housing near work is mostly old Haussmannian buildings that are rent capped and where demand far exceeds supply, so most of the stock is unmaintained. You either accept bad housing in the city or live in the suburbs and commute — not ideal.

To make matters worse, French companies (especially old-school ones) have a culture of presenteeism for white collar jobs, it's uncommon for people to leave before 6PM and staying late is rewarded as high engagement.

Lastly, you might consider buying and renovating a house so you can escape this dilemma, but it isn't cheap: 2-bedroom (T3) around 700k euros, that represents 20 years of frugal savings on 90k gross.

Vacation, cheap and good healthcare, and unemployment benefits are great in France, but the current government has been eroding these social benefits, and those are not unique to France for well-paid tech workers anyway.

All original scientist co-founders of H have left.

From what I've seen in the past few months, Mistral is the unique European lab still trying to compete (I wouldn't count Poolside as EU).

Obviously they do not, the same way Amazon isn't treating their drivers poorly, it is just third party contractors "surprisingly" not doing their job well.

It feels really weird to see this when I keep seeing the CEO and CTO of HuggingFace push for local inference to win.

even for just in-memory quick data analysis?