helloplanets

The doomer philosophy has been there since the inception. OpenAI was created from the fear of Google gaining a monopoly over superintelligence. Could've been just a recruiting strategy from Sam Altman's perspective, but maybe not so much from the people who got recruited. Dario has been around the Effective Altruism circles from way before Anthropic, and the company as a whole has always been about that line of thinking.

Wouldn't make sense to use anything below 5.5 from Anthropic at the moment. But pretty sure this is just an awkward transition phase of at most a week or two until Fable 5.5 is out.

And most LLMs have been multimodal for years at this point.

Even if the input is in plain English, the model never sees any words, tokens or glyphs to begin with. It's vectors all the way down.

Rhetorical question, not really self defeating.

Yes.

OpenAI could have done this same experiment with GPT-4, with possibly even worse results, depending on the quality of the sandbox. Even if the techniques used were not as sophisticated, the natural language output could still easily contain more unhinged sequences of words that lead to the techniques being used.

If the system generates strange conclusions as to when the task is done, or should be stopped, it wouldn't speak to the intelligence inherent to the system.

Not that the techniques used by the LLMs in the actual incident weren't unexpectedly sophisticated, but the outputs of each and every one of these processes could've been read at any time during the run. They just weren't.

The system that solved Navier-Stokes certainly was not about OpenAI engineers just typing and telling it what to do.

The whole latter part of the post is exactly about that.

This comment reads like Dario's a random tech blogger who's personally submitting these to HN

5% is insanely generous for the YouTube example.

Quote from OpenAI in the NYT article: "In addition, since the completion of Navier-Stokes, we have made substantial progress on another Millennium Prize problem."

Which model were you using?

It's been OpenAI both times though, going ham with poor sandboxing and lax supervision.

Should be treated like a digital cousin of gain-of-function research.

At least when it comes to chess, Magnus Carlsen has stated multiple times that he's been inspired by AlphaZero and adjusted his own style of play after studying its games.

Full technical report PDF: https://huggingface.co/spaces/tri-fair-lab/publications/blob...

> In this report, we argue that frontier performance can be achieved by a wide range of institutions through Continual Learning on readily available open-weight models.

> As opposed to existing limited approaches such as small-scale fine-tuning, prompt engineering, or tool-augmentation with a frozen model, our Continual Learning approach takes advantage of the effectiveness of a modern mid- & post-training stack while introducing safeguards preserving both plasticity and stability at each training stage and seeking to make the minimal number of high-impact interventions on the parameters.

For the large model, Thomson is utilizing the fine tuning stack they describe in the article, running it on Snowdon 1.0-Large, which in turn is a fine tune of Qwen3.5 397B. Same thing for the small model, but it's a fine tune of Snowdon 1.1-Small, which is a fine tune of Qwen3.6 35B.

As for the small version's run:

> The full pipeline consumed approximately 1.63 × 10²³ FLOP over 35,207 B200 GPU-hours, showing that these results are achievable with compute and personnel budgets substantially lower than commonly thought.

That would amount to around a quarter to half a million dollars of spend on that run. 100k minimum, if they got a great deal.

Playing the devil's advocate: The law also determined that any space, public or private, can't be used for same-sex sexual activity. So that was a driving force for people seeking completely anonymous encounters, in spaces that aren't linked to either person.

What about doing abliteration, weight pruning, representation engineering, etc, directly to open LLMs instead?

Building an LLM from scratch has a hard split between a tutorial project you can complete in a weekend (that's useless for actual usage) and then a solid 1km high brick wall if you want to create anything actually useful from scratch.

Modified open models have a very active community around them, without the need to look much further than Hugging Face.

Every part of training a model have the timing data observed + saved on multiple different axis, it's one of the most inherent parts of the training process.

When the post training run is nearing its cutoff point, there's a massive amount of data on how long coding tasks take to complete by that model in the golden format of "task -> time task took to complete", separable to whatever amount of subtasks, in the same format. With the parts from the end of the dataset being useful for evaluating the finished model's capabilites, whether that data is then fed back into another step in post training or not.

Completely separate from even the actual training: If you have a model proactively giving estimates that are an order of magnitude wrong, you can already fix the worst of it as of this moment by just changing the system prompt. It's a dirty fix, but it's the type of fix that has been used by Anthropic and OpenAI since forever when a model is dishing out blatantly wrong outputs.

Not sure if I'm misunderstanding your point?

There's pre and post training.

What I was going after with "aware", is that the actual people working at the companies, training the models, are aware that people aren't mostly going to be implementing the plan by hand, if they've already made the plan in Claude Code or Codex. As for a specific Claude / GPT instance, "aware" would definitely be the wrong word choice there, but the instance does have its stats and environment information in its context window, unless you specifically remove it.

Either way: Training does include estimates on working with the model, and adjustments of the model itself based on that. That's literally what RLHF is.

It's straightforward to have a portion in post training that aims specifically at the model being able to give better estimates on how long that model takes to complete a certain type of task.