mlmonkey
mlmonkey on Hacker News
The question I've been pondering is: does this work only with bigger models modifying smaller ones? Or can a smaller model tweak and improve a bigger one? I'm guessing a whole bunch of GPUs are overheating in some datacenters out there trying to answer this very question.
My hunch is: one needs a model of a similar or bigger capacity to perform surgery on a smaller model.
I hate to say this, but that reads a lot like AI slop :-/
DeepSeek says "chat not found" :-(
Given the high gas prices in the US, one would think the govt would be doubling down on EVs...
Can you give some details on what you did?
If I were to hazard a guess, I'd say that OpenAI has been including some networks and their functionality in its training data. That's why the model is able to reason about networks with their weights. It makes sense to do so: if I wanted a network that could perform some weight tweaking, I'd include such networks in the training set.
This raises so many questions.
Unless Astra was trained on other networks, how is it able to reason about the weights?
Does this mean that OpenAI has been including networks in its training sets, maybe in an attempt to RSI? I wonder which other networks are able to ingest other networks and reason about their weights?
I wish I could find more recent videos. Even turning on all "recency"-related options, here's what I get:
https://imgur.com/a/e5Dcw5LEverything is 6 months old or older. :-(
Not a dig at Andrej, but Youtube itself is not that great at promoting recent content.
At my previous place, we had to write unit tests to cover each and every line of code. So if you add 10 lines of code, you ended up writing 100 lines of unit tests, mocking this service and that service, etc.
Once AI came along, I became like Captain Picard (RIP) and would just sit back and let the AI write all unit tests. "Try harder, make sure coverage is 100%". It was very exhilarating to sit back and watch the models churn away, creating 100s of lines of unit tests. Of course, I checked them all manually before submission ;-)
You forgot the reference [2] ...
Can someone give a TL;DR ? What's the input to the model? What's the output?
I don't think so. First price auctions are not truthful, and advertisers hate them. But there are 50 shades of gray between a second price auction and a first price auction.... if you get my drift.
It's not when it happens involuntarily :-D
There's a reason why GSP (Generalized Second Price) auctions are called "truthful auctions", and that's because they elicit the true value as the bid from the advertiser (in theory).
So (in theory) advertisers will lower their bids if they're not getting the desired value from the clicks; and over time some will migrate budgets to other channels (like "influencers"? shudder ...).
Difference is, TV had (has) content in 30 minute chunks. But Youtube has, among others, Shorts, which are often 30 seconds. But still Neal Mohan wants to show ads on Shorts.
On top of these ads, creators are inserting their own ads ("this video is sponsored by ...."), so it's ads all the way down (like turtles).
See the replies to that tweet. I agree, one swallow does not a spring make; but here there's a pattern. Ask anybody who does search advertising, and you'll hear the same story too many times for it to just be a coincidence.
Most people don't know this, but the Search Advertising auction is a black box. The advertiser just specifies a bid (the maximum they're willing to pay for a click); and the search engine promises that they will not pay more than that.
Here's a trick that Search Engines may (or may not, I plead the Fifth) use: the price an advertiser "i" who bid "bid_i" pays is bid_(i+1) * pctr_(i+1) / pctr_i where "pctr_(i+1)" is the predicted probability of click (pCTR) on the next lower ad; "pctr_i" is the predicted probability of click on the current ad.
Guess who calculates the pCTR? The Engine! In theory, if their pCTR calculations were 100% accurate, all advertisers would pay the minimum bid (which is typically $0.01)! Clearly, this is not desirable. So Engines juice the revenue by tweaking these pCTR values. Talk to anybody who works on auction mechanisms at a search engine, and you'll hear stories of how they tweaked these pCTR values.
Chrome logs every site you visit with the mothership, and that is very valuable data to Google. That allows them to spot, in realtime, trending sites for example. They see what the users are doing on the Internet in realtime and act accordingly.
Don't worry, they've put an Advertising guy in charge of Youtube, and he's busy inserting as many ads as possible into each and every video...
What is Pixel's marketshare, for developed markets? Despite a decade+ of competing, have they made any significant dent in iPhone's share?
> It's probably going to be a terrible earnings call in terms of capital expenditure, cash flow and new borrowing.
Now you know why they released it just before the earnings call.... :-D
As a driver, how would you track down the insurance company behind this trucking company? People have other things to do.
But there must be some compensation to be forced to become an unwilling participant in Musk's experiments! Otherwise what's the disincentive for these experimenters?
That's a different accident, and the last one involving a Tesla Semi (before the current one). The point they're making is: that one involved an actual crash, but this one did not. So we should investigate this thoroughly to see what happened (as the NTSB is going to do, of course), because large vehicles should not spontaneously combust on a public highway, especially one as busy as the I-880.
On a related note: I wish companies like Tesla, which are conducting live experiments on public highways and thereby inconveniencing 1000s of people when an experiment goes awry, are held accountable and forced to compensate those who got stuck on the highway, missed doctors appointments, job interviews, deliveries, etc. Maybe something like a fine of $5000 per driver impacted should be levied, and given to each driver?
I hear you, but my objection comes from the stress on "realism" in Google's post: how Genie4 is modeling the "real world" so well. If they didn't crow so much about realism, I wouldn't have cared as much.
> How often do things in the real world not behave exactly as you expected them to?
They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works.
Out of curiosity: is it possible to take a frontier model and just let it loose on all math textbooks out there, converting each and every proof/theorem/lemma/etc. into Lean? A frontier model should be able to do that, right? And it's a one-time task: do it once and it's there for eternity, available as a library to be used in other proofs.
Maybe my understanding of Lean is wrong; I'm thinking of it as a programming language but for Math.
When Google's epitaph is eventually written, PMs will figure prominently as the catalyst for the rapidly accelerating decline.
In the clip with a marble bust dropped on a watermelon, at this point
https://imgur.com/a/IHLnkUs the bust should've toppled over, if the physics were properly modeled.
Sadly, the stock is up on this news.