ssivark

Perhaps because statistics education attunes us to average over individual differences, any mention of "data" unfortunately makes people instinctively jump to this 'cross-sectional' corner... textbooks explain how the typical human body behaves, and standard education and medical practice respectively emphasize averaging and population-level guidelines.

I think this approach is limited in some fundamental ways, and forces a dilemma: either accept wide and unactionable bands for what counts as normal, or use narrow bands that over-pathologize most individuals as diseased. The natural consequence of this is to start "fixing" deviations, and this is the territory of dangerous over-steering. This is why the medical community is afraid of "incidentalomas" and over-intervention. Tests which are point-in-time and compare one human to others are problematic in this respect. This is especially acute when the body is measured along many axes: like a full body scan. Much like the US Air Force Study of pilots discovered that there is no single pilot compatible with the "average" pilot measurements, comparing metrics from an individual's full body scan to a cohort average is likely to drastically overestimate problems.

In some ways, learned systems (like our physiology which is constantly adapting to our individual environment and experiences) follow the opposite of the Anna Karenina principle: every healthy individual is healthy in their own way. So the real diagnostic leverage isn't comparing an individual to an aggregate human template, but measuring change against their own homeostasis, and this requires high temporal resolution, not wider population comparisons.

I'm currently exploring how to bring this kind of longitudinal tracking to higher-fidelity modalities like wearable imaging. If any clinicians or researchers here are thinking about how to operationalize personal baselines in practice, I'd love to chat (email in profile).

What is unfortunate about this? I suspect we concur in not buying into the AGI/ASI/RSI/<jargon of the day> hype, but this isn't this kind of breakthrough extremely valuable regardless?

Maybe it's not expanding the convex hull of what is possible, but the fact is that human progress is extremely jagged even within that convex hull -- for myriad reasons limitations related to funding, cross-domain expertise, number of scientists, etc.

----

If you're worried about what it takes for a novice setting out in the field, it might be useful to see how specific "mature" subfields have evolved in their training structures. Eg. I have a background in theoretical physics and it is common for grad students to do courses for a couple of more years and take a few more years to reach the same productivity compared to grad students in (say) CS/AI.

Since you're also thinking about a user giving their agent access to their inbox -- any thoughts on what are the right capability primitives for that situation, and how the API & access tokens can represent that?

Whether it's an oversight in just this work, or TCS more broadly, I do not know. But it's a fundamental point nonetheless.

One can of course make a puzzle out of anything, but it behooves asking what situation is being modelled by that problem.

And yes, this criticism might apply more broadly to TCS rather than just this specific artifact from OpenAI. It's reward hacking regardless of whether humans do it or LLMs.

number multiplication is basically convolution and closely tied with FFTs, so I wouldn't personally be surprised even if a lot of the "FFT's work" could somehow be stuffed into exact arithmetic operations, and the remaining costs just track the packing and unpacking from that representation. It's just an exploit of the book-keeping.

That's ignoring the other complaints about this being a galactic algorithm.

Even easier if, additionally, many of the smaller websites were syndicated into a tracking network?

Also, from the study:

> The non-UPF and UPF test meals were closely matched on meal weight, energy, energy density, macronutrients, available carbohydrate, glycaemic index and load, total dietary fibre, sodium and water (all ≤1.6% deviation between meals; Fig. 1b).

The math peer review process can take months/years to decide on the correctness of a solution -- especially those written as sloppily as these AI proofs. Let's see what the numbers are once the dust settles -- how many of the 700 are correct, and how many of those are valuable.

Not to mention the math paper (on HN yesterday) which pointed out that OpenAI might have verified the wrong thing, in their Navier Stokes proof.

It's really not as cut and dried as you (and software/AI folks more generally) think it is.

PS: If a human mathematician had to retract three of their papers a day after posting publicly, they'd lose all credibility and their mathematical career would be all but finished. That social incentive structure is the field's immune system against slop. You're basically asking them to turn off their immune system, and for unclear gains (other than OpenAI's grandstanding).

We'll see what the final slop rate is, but the three papers they retracted yesterday were for a trivial sign error. If they didn't catch that, that means OpenAI isn't bothered to put in the minimum effort of sifting through their own garbage and making sense of it.

It's not like they needed to hurry out this release before carefully vetting. They're just "hacking" the math system and disrupting the work of thousands of researchers to create a gigantic RL dataset for themselves.

Really, what's the bloody hurry?

They could have released 1-5 papers, worked with researchers to understand what methods work and what don't, how to prompt the models better, how to build better guardrails for reasoning, etc. And give those researchers access to latest models and empower then to solve thousands of problems!

Instead OpenAI wants to piss all over the city to claim territory and now human mathematicians have to go around cleaning up that slop, only so that OpenAI can made some bullshit statement like: math is solved [mistakes are next].

--

Imagine someone gave you a million line PR claiming to have vibecoded the operating system of the future (or whatever your application domain). Would you drop all your other work to focus on this? And they generate enough PR that your manager and company leadership and public all start pressing you to accept it quickly? Guess what, it's your lucky day! You have not one, but 700 breakthrough PRs!

Composite indices are only useful for model companies which want to build one horizontal capability for N use cases, or naive users who don't want to bother with the effort of carefully pairing models with use cases. If you're a power user looking to understand and make deliberate choices, then you want pointed evaluations -- not general composite indices. You wouldn't hire the same person to do your taxes and mow your lawn, so why is it any different with LLMs? Only if you come at the problem with the folklore around "AGI" do you start making composite benchmarks.

As I mentioned in a sibling comment... Back in their 3.5/3.7 era [1], Stepfun had a small+fast model, specifically eschewing knowledge, and instead focused on "general intelligence" like managing tool calls, orchestrating sub-agents, etc. Could be a very good substrate for running a personal assistant (time will tell what's the right approach); so I'm curious to try that out and see how they've come along since then.

[1] https://news.ycombinator.com/item?id=47069179

[1] https://news.ycombinator.com/item?id=47069179

1. Homoiconicity: Harness mechanics are kludgy and we need proper homoiconicity to uniformly handle code (tool calls) and data ("natural language") as token streams.

2. Actor semantics: for isolation, encapsulation and concurrency.

3. Object capabilities: injected references and no ambient authority. Capabality-based reflection/introspection is a clean way to discover interfaces and affordances.

"Code mode" or whatever is basically rediscovering this by hacking outward from LLM token streams, instead of from system design principles based on decades of computer science. It is the beginning of treating an LLM as a programming-language runtime participant (any takers for eval/apply?) rather than as a text/token generator with the harness as an ad-hoc interpreter.

If existing implementations of code mode don't already support all this, I anticipate they will keep piling on hacks till they get to this point.

----

I think it was Dan Ingalls who said "An operating system is a collection of things that don't fit into a language. There shouldn't be one.". I see the same for harnesses -- they're awkward middle children which fit neither in an LLM nor in the programming environment.

Maybe the answer is to partner LLMs with Common Lisp or Scheme fibers / Spritely Goblins or Erlang BEAM and be done!

It's not just that they are doing users a favor with weights; they are just as much seeking favors with attention and usage (in a crowded market!).

> I do not understand how you can see what has happened in the last 3 years and say "it will not get us there" no matter what "there" is.

--

Your statement is something much weaker, and I would still question what exactly "general" means when AI capabilities are commonly accepted to be so "jagged".

This is what happens if you try to read the data like gospel without having a mental model to interpret it... in other words, critical thinking.

Chatbots are a failure because chat capabilities have sucked for most of the past decade and chatbots weren't really capable enough to do anything meaningful. Not because chat is fundamentally bad. Similarly with auto-router.

Also, if you want to "cross the chasm" you cannot measure reactions from current AI users (early adopters) and extrapolate to the rest of the populace. Early adopters are paying you for tokens and try to wieid them like a surgical tool; across the chasm the rest really don't want to know anything about "tokens" or models tiers or "thinking level".

If you had told someone in the 1800s that a machine could instantly multiply 100 digit numbers, that would have been considered dazzlingly intelligent. And yet we are not that dazzled by our calculators today (despite how useful they might be!).

[1] https://arxiv.org/abs/2608.25512

There was also an exciting fUS imaging demo from Aleph neuro a few months ago.