Oof, no, this "puzzle is a few days old" is incorrect even if it's a social media trend just recently. Asking a model to generate a world map in this way is _at least_ from August 2025 as it appeared on LessWrong at that time: https://www.lesswrong.com/posts/xwdRzJxyqFqgXTWbH/how-does-a...

one would hope that they disable websearch and internet access (maybe all tools?) when doing generalization testing?

Seems sloppy.

We've updated the caption to read: "Land/Water Generalization Experiment: We recreated the viral X puzzle by asking Beam to create a fixed 180×90 grid for longitudes -179° to 179° and latitudes -89° to 89°, with 16,200 points. Despite not being trained on multimodal inputs like images, Beam runs the analysis in text mode and gets 95.5% coverage right, putting us between Opus 5 (92.5%) and Fable 5 (97.8%), which shows how well it generalizes to novel new tasks."

> Beam’s capabilities come from major investments in both pretraining and reinforcement learning (RL). We pretrained the model on 23.8 trillion diverse, curated, high-quality tokens from the web and proprietary licensed datasets, matching or outperforming available similar-sized open base models. In parallel, we developed the algorithms, training environments, and infrastructure needed to sustain high-compute RL at exceptional scale. Our high-compute RL run generated over 100 million rollouts on 10.5K NVIDIA GB300 GPUs over 4 weeks of training.

Early access, no weights no tech details, just a sign up here for info

I think the comment you are replying to is unnecessarily hostile too though.

Yet going by the comments on localLlaMA or HN, those companies are the devil :-D. Colour me surprised.

Will release weights soon.

It's not just that they are doing users a favor with weights; they are just as much seeking favors with attention and usage (in a crowded market!).

I think the point is that people are happy to see promotional posts when they actually release it, but only then and not before.

Unfortunately pinky promises from corporations to release something at some indeterminate time in the future aren't worth the bytes they're stored in, especially in the AI industry which is full of grifters and charlatans.

That means that they have the 31th of October as the deadline to make true their claims.

The fact that they give early access to some may mean that they want some beta testers before the public release.

Imo you can get better results with great data and generic modeling techniques than with incredible modeling techniques and crappy data. Because if you have crappy data, you won’t even know if your model is good because your evals will also be bad.

This is why Anthropic is throwing a fit about the Chinese distillation “attacks”. Clean reasoning traces are gold.

Companies pay lots of money for proprietary agentic trajectories which are used during RL. These are things like "Task: summarize stock levels for months end accounting" which then traces the task though using SAP to look at different SKU stock levels, exporting them and generating summary Excel spreadsheets.

This is very different to the "scrape the internet" datasets that a table stakes for training a LLM.

Xiaomi released a fairly developer-centric dataset like this here: https://huggingface.co/datasets/XiaomiMiMo/MiMo-V2.6-RL-oss

SpreadsheetRL is another fairly specialized dataset: https://spreadsheet-rl.github.io/

> We will release the weights, technical report, model card, and developer artifacts later this month.

Google does do a great job with Gemma models. It's one of the few language models actually good at language. OpenAI's top closed models can't even write norwegian correctly.

[1] https://i.imgur.com/rJSG019.jpeg

Of course in any society there will be people that steal money or other universally recognized crimes, so when the state wants to remain in power they will promote news about actions against bad guys while cleaning up any bad news about the government.

It produces masses of either oblivious people or people who know it's probably strange and wrong, but would rather accept it than bother with the hassle of trying to change anything when you know all past attempts have failed.

China carries out more executions than the entire world combined. Even when they caught US CIA spies, China executed them all.

If you look at the way all of these totalitarian or authoritarian countries aligned together seem to engage in diplomacy, they try to put forward an extremely normal face to cloak their country against negative perception.

Venezuela, Cuba, Russia, China, Iran. Torture, assassination, institutionalized rape as punishment, funding terrorism, mass killings, starvation, global harassment networks, intentionally killing over a million Americans from drugs alone as part of unrestricted warfare even subsidizing the precursors ONLY if they're sold outside of the country, ignoring international law and building fake islands in the sea, ramming the ships of neighboring countries, persecuting religious people, purging their military like Stalin and the list goes on.

They intentionally normalize disappearance so that when someone disappears in their society, it's not considered overly strange. They cannot contact their family, their lawyer, their friends or any other organizations. They will not experience a just process and acquittal is basically unheard of.

They do a bad job of caring about people or human life even if there are public statements to the contrary. They constantly fail their people and cause unnecessary death, then under-report it or censor it. They lied about their COVID deaths and even cleaned up any evidence so that no international investigation could occur and the CCP could frame it any way they like, they censor deaths caused by floods they themselves caused by their poor planning, they censor the fact that Chinese companies poorly constructed the buildings in Venezuela that collapsed in the earthquake killing 10s of thousands of people unnecessarily, they lie about the schools in China that collapsed killing countless thousands of students. They were thinking over half of all residential buildings in China would need to be replaced, it was that bad.

How can anyone living in a country like that, have a clear eyed view of how good their society is to live in, when any information that should help you better understand it so you can improve your own society, is simply vanished? Is it just like, well, so long as it doesn't happen to me, it's fine! I don't see the problem! That is insane.

The problem we have here in the west, is that the CCP was founded by Comintern which had open ambitions to achieve total worldwide communist revolution in order to erase capitalism since true communism cannot coexist with it and this was a major cause of World War 2. The CCP has regularly lamented the fall of the Soviet Union and aim to do everything possible to prevent the CCP from collapsing that way.

There isn't any obvious open communication from the CCP to that effect, but the way they are expanding globally, with unprecedented influence, infiltration, etc and the most rapid military buildout in history combined with all the countries aligned with China starting to attack their neighbors, the question is fair to ask. China has this philosophy of using your enemy's own sword against them, and it is thought that capitalist market reforms within China have basically been that. Iran has radicals inside its country that hold similar ambitions, behaviors and strategies for expanding Islamic revolution which is why they fund terrorist groups all over the region and pretend it's not actually them.

All these things and countless more, yet their diplomats try to flip the claims around to make it sound like actions by other countries are just as bad, when in fact they are not. You can compare things, you can make analogies, but the actual practice and the intentionality are entirely different. Whenever a western country makes some mistake, they milk it for all its worth as if it's normal, but they do it to try to establish the psychology that "in reality we are better than that, but if something did happen, look they did it too!"

I'm not saying that China wants their people to die from COVID, wants them to die in building collapses, wants them to be executed in secret prisons. There is evidence they would rather that not be the case, but it's more pragmatic than compassionate. So many of the problems would not have persisted for so long if people were able to communicate more openly about these things, but they think it would be a threat to the existence of the CCP.

The spectacle of China is partly an illusion and people being taught to think otherwise aren't a good source for identifying whether China is problematic.

There are many great Chinese people, but the CCP is a threat to the whole world and so it's no longer optional to be vigilant about watching their behaviors. A thriving booming society post-industrialization is normal and simply more amplified with a large population like China, but that doesn't change the essential concerns. People can go live there and at the individual local scale maybe life seems fine, but in terms of world strategy and risks there is a bigger picture coming into focus.

I guess you the shill then :)

Am I missing something?

> That only means their training regime is inferior if their predecessors did so much more with so much less

Hard to imagine how that wouldn’t be the case. They probably missed the boat on distilling Claude (or their lawyers said no), they probably didn’t hire an army of math PhDs to write reasoning traces, they don’t have millions of DAUs in a coding agent to train from, and they probably have less money, less experience, fewer top tier researchers, and fewer resources for experiments. They are an underdog without a doubt.

None of that means they shouldn’t release their model.

Alternative explanation is that the Chinese have far more technical talent than anyone else, along with the infra and capital to build out these models.

My money is on the latter explanation, tbh.

Says who? We know Grok does at the least. They admitted it openly.

500B params performing worse than other OSS of the same size is pretty meaningless if no one will use it.

It's pretty clear from their framing ("Beam advances the Western open-weight frontier") that one of their main selling points is not being a Chinese lab.

I can't imagine that mattering to many individuals, but I guess someone out there has a government contract that forbids the use of foreign models

Asking an easily-searchable question is just lazy.

/s

The conversation is why.

InB4: kids these days :shakes-fist-at-cloud:

seems like they are aiming to provide both inference and RLaaS for american companies and western govts. even if they never fully beat deepseek if they get close enough the fact that they're American will help them close deals

I really enjoyed reading the log book from the training of OPT-175B at Meta… I guess it’s all classified info but it’d be fun to read a blog post about the crazy day to day issues you run into when doing things at this scale :)

Where do I get the data?

I mean, this many models. They have to start somewhere.

e.g. fineweb dataset is 50TB https://huggingface.co/datasets/HuggingFaceFW/fineweb

I recommend checking papers from Datalogy, Nvidia Nemotron, Ai2 (Ollmo, Tulu, ...) and the recent model from Aleph Alpha if you want to learn more.

It's interesting how the industry converged to this very term, given that very less work is being done by horses since quite a while.

It's great to see a company that acknowledges it still needs improvement instead of making false claims.

participation awards are not helpful.

https://github.com/inclusionAI/Ling

BTW, this is also an AI lab from China

                                DS V4.1F            Beam
    LM total params             552B                501B
    LM active params (prefill)  8B                  23B
    LM active params (decode)   16B                 23B
    N-gram/PLE params           196B                0
    Pretrain tokens             45T                 28T
    Disk KV bytes/token (FP4)   890                 No information
    Vision                      Yes (pretrain)      No
    Weights available           Yes (launch day)    "This month"
    Weights licence             MIT                 Apache 2.0

Access is currently limited. We'll contact you if early access becomes available.

1: https://en.wikipedia.org/wiki/Qwen

The reason I ask is even though it can take hundreds or thousands of contractors to teach a model a certain behavior, wonder if they really only need to do it once for each desired behavior? (of course, future models might expand and refine this previous training) Because if that is the case, then wow, then future models can really expand their capabilities really very fast.

...Am wondering if they somehow record a training session so they can play it back whenever they need to train a new model with the same info? Or maybe the new models can just use distillation from the old model to relearn the old behaviors?

Regarding your second question, I think if they want to further post-train a model using new data, they wouldn't feel the need to re-train it again on the data that it has already being trained on. But you never know. If the model is a completely new pre-trained base model, then they could either train the model using all the data and/or use a previous model to teach it. There is definitely a bunch of tricks they do to evaluate the models and check the performance or whatever their recipe is. It's really up to what the engineers would prefer. But you get the core idea, the models are not suddenly coming up with how to use the Blender on their own, they are explicitly being trained on a dataset curated by a professional that teaches the model how to use Blender. Surely there is another aspect that if the model gets better at coding, then it also helps it become better at Blender, and you have that transfer learning. However, there is no emergence or a deity popping up. But you get people who were evaluating theses models on blender use and suddenly seeing the model ace their tasks and they think they are dealing with a super-intelligence. They then undergo an AI psychosis once they try and extrapolate the (super)-exponential improvement in that one task over the next few months and across all other domains.

Regarding your third question, I already answered at it. But, when it comes to training, they definitely freeze the weights after each run just in case an issue arises and they need to address it (a GPU not working or the loss value blowing up).

... And yeah, agree with your statement that this is not emergence. But to me, it's still impressive, because the more post-training tasks they create, the more the model can do, and this set of abilities is only going to keep growing and growing.

... Also, if you wouldn't mind one last question, have read that the AI companies have started to do self-training, where the models are starting to train themselves. Might this lead to even faster and faster increase in model capabilities? ... although, based on what you're saying, maybe it just leads to a model reaching a its max plateau for existing capabilities rather than an explosion in new ones. Because, as you mentioned, for a model to learn an ability, it still needs a human to create a training dataset for it. Is this your understanding?

You're talking about real data created and curated by humans to help in training LLMs.

Anthropic have done experiments with having "dumber" models try and make a smarter model, smarter, but it still seems tricky so far.

But if you think about humans, we have the same issue. If you have a super smart person, how do they get smarter? Can you have dumber people make a smarter person even smarter? It's a hard question.

It's true that no benchmark communicates the whole picture, and we won't really know how it behaves until weights are out, but the performance here doesn't seem particularly groundbreaking just based on the benchmark.

What do I mean by cheap? You can rely on the SSD to retrieve the relevant tokens as no computation is needed meaning you can leverage storage (or cpu ram if you don't have unified memory) to serve part of the model which (to my understanding) is much cheaper to get than GPU RAM.

Anyone know what am I missing? Or is it that the pace of iteration for labs slow enough that they can't actually leverage it yet?

Qwen3.8 Flash Next release date: 26th August

DeepSeek V4.1 Flash release date: 10th September

Current date: 6th October

I think they'll become more popular in the coming months. Also Gemma 4 PLE (April) is similar to DeepSeek Engram in a lot of ways, just with 1-grams.

On the proprietary model point: I'm personally curious about whether heavy n-gram offload is one reason Anthropic keep driving down their token vocabulary size (the other reason being eliminating the LM head gradient bottleneck).

I am more interested with a SOTA Frontier 8B-10B model. Is this even possible?

The price of that kit is rapidly approaching $10k.

The email harvest move just doesn’t fit where we/they are in the cycle. There are established players and a buffet of models to choose from (plus a ton of empty hype). The first move at this point for any new entrant should be to show, not tell. Even an API only release with the promise to open weight would be better (actually probably all around better since most people can’t run this locally).

I wish this lab and all the labs releasing the best. It’s a brutal landscape to sink millions of dollars into for a guaranteed “behind x model from a year ago” evaluation. But, I do believe there is genuine innovation left to uncover.

I think it is ever more important to realize who is releasing models rather than what the models do and how they compare.

Because models iterate at breakneck speed, looking at today's benchmarks is only useful for someone using the models today. Whereas if one builds a product on top of it, or commits to one for a project or team, the company or organization behind it, is far more important. Will they exist in a few months? Do they need a business-model? Are they subsidizing usage with venture capital and how long can they keep this up?

Is reflection a company? University lab? NGO?

> The startup was launched in March 2024 by Misha Laskin, who led reward modeling for DeepMind’s Gemini project, and Ioannis Antonoglou, who co-created AlphaGo, the AI system that famously beat the world champion in the board game Go in 2016.

with the obligatory:

> Investors in Reflection AI’s latest round include Nvidia, Disruptive, DST, 1789, B Capital, Lightspeed, GIC, Eric Yuan, Eric Schmidt, Citi, Sequoia, CRV, and others.

Ideally i would like to place my own restrictions and align from scratch, currently I am resolved to do harness alignment using tools like Prismor but would love to do my own post training alignment