famouswaffles
famouswaffles on Hacker News
Right? Occasionally, I would think about how the concept of the singularity or recursive self improvement was a bit incongruous today. It was made when people expected generally intelligent machines to be these logic automatons that could 'alter their own code'. It seemed to me that this path would be closed off with the advent of Giant Neural Networks that take weeks/months to train. That GPT-X would 'improve itself' by designing a better/more efficient NN architecture. I didn't write it off completely but that just seemed a LOT more likely than altering weights directly for much the same reasons you describe. Maybe not lol.
This is pretty massive, and a result I (and I think most?) would have never expected. Holy Shit. How?? Did OpenAI figure out a way to train this? Are they even aware?
For A, a common misconception is that models write out lean proofs from the start. They write/develop the Natural Language proofs, and then translate (sometimes not even the same model) to lean afterwards.
Being racist comes easy for you huh?
Today with Opus, Sol, etc there's basically no reason to visit the site, but when chatGPT launched it was powered by GPT 3.5, which would realistically only be able to answer a (minor) chunk of all possible StackOverflow questions.
Software Development is far from the only white collar work frontier agents are very useful for. Most people just have no idea and marketing around them hasn't really struck. If you think it's just code or math that LLMs have gotten rapidly better at then you'd be wrong.
>For Python libraries they would be written in Python, eventually ctypes for syscalls.
So if you can't call/use the 'library' from the 'native' language, it's still a library? Whatever floats your boat. Lol.
I know what bindings are. A front-end to C++ for pytorch was only released from 1.0, literally 2 years later. The relationship is very clear here. There was no native C++ library. It was an afterthought. Even in instances where there was, like Tensorflow, functionally it was still an afterthought.
They are python libraries because people reached for those languages so they could continue work in python. They are there to serve python and nothing else. It's the same with Rust. It's by no means a C++ thing.
EDIT: Not saying this is always the case. Qt>Pyside is an example where it's both technically and functionally bindings, but the vast majority of the time they are serving python. Some don't even have a native library to speak of.
>the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.
They are Python libraries. Don't make the mistake of screwing up the cause and effect here. C, Rust etc - those are the replaceable parts of the chain. There is an official C++ pytorch library and virtually nobody uses it for ML Research. People want to code in python and build those libraries so they can.
Results come from an internal model that has not been publicly released. And there are lesser problems and conjectures ordinary folks have solved with Astra/Sol.
Because I might look elsewhere if they starting charging several orders of magnitudes more then I'm not sticky. Does that remotely make any sense to you ?
I'm not rich and you don't have to be rich.
Most people buy their apples from the same grocery store every week. They're not shopping around every month to find who has the cheapest apples or 'best' apples. They go where they're accustomed to going, and as long as price and quality are satisfactory, they have little reason to change. Walmart, price-sensitive as they are, have relationships with fruit suppliers they've maintained for well on a decade. Even in commoditized markets, people are sticky.
Google, the titan mega-corp with more money than they know what to do with and a billion+ android devices to ship gemini on and it's made no dent for ChatGPT's consumer growth and adoption. Yet some plucky open source chinese models will?. It takes more than that.
Yes it is. Tiktok's content and content recommendation algorithms are much better than Instagram. Users spend more time on Tiktok than they do Instagram and it's not close. Tiktok's engagement rates are several times better than Instagram. Reels did nothing to dent it (yet another example in the book). Again, nobody will care about Chinese models if they're just more of the same. They can take a piece of the pie, but this idea that they'll spell doom for the frontier labs is nonsense.
Given we are comparing two different substrates, there's no reason to assume this means anything.
Yeah. People have very strange ideas about competition and choice and moat. If all you needed to unseat the entrenched was a competitor then Bing would be going toe to toe with Google, and Edge with Chrome. They have a billion+ devices they can stick those options as defaults and still nothing. There are many examples.
People are naturally sticky. If they're satisfied with a product, they'll stick around unless something comes along that is so much better it gets people talking. Open AI's 1B+ customers won't give a shit about Chinese models. Enterprise customers won't move to Chinese models, especially just to save money. They are a non issue, especially if the M.O is just going to be playing catch-up.
Difussion Gemma performed a lot worse than its equivalent autoregressive counterpart (it was particularly bad because rather than a sort of constant-ish gap you might see between 2 model classes, the gap seemed to worsen the harder the task/benchmark) and the speed benefits got negated at scale. Not really surprising we haven't heard much.
No it's already explicitly stated that translation to lean is a second step that happens after NL proof is done. That's why some of 400 paper dump is without lean proofs.
All this is saying is that there's a lot of stuff benchamrks aren't covering. It's not saying models can't do those things well or don't get used to do those things.
A blip for NLP? What bespoke NLP research is about anything other than LLMs these days? How many companies are handcrafting bespoke NLP models instead of sending a LLM request? Really confused here.
Open AI have the best math models for sure, but it was Claude Fable that found the Jacobian Conjecture counter example. All the frontier models are much better mathematicians than even a year ago.
Being in the loop doesn't make LLMs anymore of a tool. How many software engineers can just take the vague description, bang something out, push to prod without talking, clarifying or discussuing with anyone in the meantime. Are they tools?
>I'd agree, although I doubt you'll like my framing: it's a commodity business - a race to the bottom on cost of production and pricing
>Coding agents competing with free weight models unencumbered by development cost?
Prices falling doesn't mean a market dissapears. Compute, Bandwidth and Storage requirements have already undergone reductions orders of magnitudes in the last few years and consumption has only exploded. The vast majority of customers, consumer and enterprise alike do not give a shit about open weight models. I have no idea why people here think they do. If they pull ahead significantly enough then they might attract people by value of the amazing service they're providing (nothing to do with open weights).
If they're just going to be playing catchup then they're not gong to gain any signifcant share no matter how free model dev costs become. This is just not how customers work. They're sticky. They value reliability and familiriality and simply having something that works. If you want to unseat google search then you need something so far better people start talking crazy about it. Creating Bing won't do shit even if it's the default search engine on your default browser (you'll just change it).
>I'm not sure if you're claiming a market for original math research, large enough to be more than a distraction, or perhaps you are imagining AGI will emerge from pursuing the problems that mathematicians have set for themselves?
The economic output of math research is very often something other than mathematics. It's not about the market for math papers.
When you see stuff like this
>> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
Do you imagine the gaining advances in math competence had nothing to do with it ? I'm telling they'd be well justified even if their only customer was themselves.
>Good luck with that. Let's see how much ad revenue OpenAI can take away from Google and Meta. The ad-serving product better be free, since the competition is.
They hit $1B in annualized ad revenue run rate ~200 days after launch. They'll be just fine. That you think they might have any trouble taking away or generating their own ad revenue is quite bizarre. If the metrics are good businesses will pay hand over fist, and it seems to me that OpenAI should have a much more complete profile of their median user than Facebook or Google.
>Coding agents competing with free weight models unencumbered by development cost?
The utility of frintier agets isn't remotely limited to software developers. Whoever cracks marketing for agents towars non software professionals will make a lot of money.
I don't really understand why you're treating frontier mathematics as commercially irrelevant, especially to a company like OpenAI. Their entire business depends on advances in algorithms, optimization, linear algebra and computational efficiency. Their latest release even includes results in matrix modification and algoritmic complexity. Even just to them, it's worth a lot. The value of a model capable of making original contributions to these fields should be blatantly obvious.
And you seem to be confusing the immediate commercial value of a particular math reuslt with the commercial value of a system capable of producing original math research. Thise are 2 different things, and the latter is worth a lot of money.
>AI is in some ways still a technology looking for applications, at least applications with a large enough commercial market to justify the valuations these companies are seeking.
These companies already have the fundamentals they need to justify current valuations. ~1T is nothing to a company that has over 1.2B+ weekly users (and growing) and is entering the ad space, nevermind everything else. What they are hoping to achieve will give them power akin to nation states.
Math is not a game. It has utility go and chess don't, so there are incentives to keep improving it. They're not running models on open problems to brag. Their models have simply outgrown all conventional math evals. They need a new one and open problems will have to do because there is still valuable signal for training.
So now it's just a question of, do you sit on your advances or release them? Why would sitting on them be the 'right thing'?
This is what the frontier can do with a computer now.
https://stillwet.art/
Where are the verifiable rewards? It's simply not true that RLVR amenable domains are the only things LLMs are improving massively on. Even Vending Bench is improving.
Intelligence is a jagged beast. That's just the nature of things. We evolved in certain environments and situations and sometimes there's a mismatch and it causes all sorts of wonky things. We just call them funny names like optical illusions and cognitive biases. Most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters (Watson Test).
That's still not close to killing a person in real life sorry.
Because people who say this are always begging the question. They're already at the conclusion and work backwards to justify it.