dumberquestions
dumberquestions on Hacker News
If model weights have a logical and coherent structure, I don't think it would be surprising for models to have some understanding of them, the same they can understand language and code and images, given enough data.
What would be surprising is if this happened without any specific training on model experiments and interpretability.
We don't know the specifics of what type of ML related training Astra underwent, and it wouldn't be crazy to find out that small model experiments are a big part of model training.
What's surprising though is that I'd expect this type of training to be exclusive to internal model snapshots or at least heavily guardrailed against in public models.
That feels like a technicality worth addressing.
That's higher than what I would've expected.
It's pointed out as a shortcoming at the bottom:
>What could make this wrong:
>1. "Understood" means you know every word. It ignores grammar, kanji readings, speaking speed and context.
I wish this happened decades ago because I'm not so sure fission can compete with renewables anymore given cost and time to build.
Does no one write their own blogs anymore?
I don't know why it shouldn't be as long as someone mixes in some additional post training.
If you're paying €35/month for home internet, consuming 500 GB per month, all of which was from taxable sources, and the tax was indirectly passed on to you, you'd be paying €60/month for internet.
But at the same time the tax is capped at 15% of the providing company's French revenue or 2% of its worldwide revenue, and realistically speaking, not all companies would pass the tax on to the users, so it could end up being a noticeable but not catastrophic additional expense for both users and companies, if the bill passes at all, that is.
Friend of mine, who's been treated as the unlucky one as an inside joke, made a binance account - after avoiding crypto for years - just hours before the hack happened, it's now official lore that this whole chain of security incidents is entirely his fault.
No one really considers crypto held in exchanges as decentralized, I guess some might but it's pretty standard knowledge.
Somewhat impressive that search share of revenue is 5% down, but search revenue is 12% up..
Artificial as in synthetic or man-made, everyone knows what it means, you're just giving a post-hoc rationalization for whatever game he's playing.
Code only needs to work, art needs to generate positive emotions.
No, I said could as in it's not an apples to apples comparison.
You're comparing different generations, any 5.5 models could be better than any 5.0 model.
Have you read entirely generated documentation of code? I wouldn't say it's not understandable but it's no where near good quality human documentation.
Why this and not any of the other robotics startups?
I think not being able to experience music as much as I do would be akin to losing an entire sense.
You brought up fine dining and frozen pizza but what about a situation where the automated one is both cheaper and better tasting? We're still functioning on the assumption that that will never happen.
«And if there's no inherent value to humanity, why should the machines be compelled to interact with you?»
Reading this part made me wish it was more compelling than I think it really is, because I think someone could argue that the value of human consumption isn't predicated on the value of human creation.
I used to find things like machine-run pizza places and coffee shops really cool (even though I've never been to one). I even wrote a small coffee automation program once, but recently, with the promise of automation getting more real by the day, I find myself genuinely worried for the fate of human-run pizza places! They're doing fine now, but what if the machine-run ones become 10x or 100x cheaper? Would anyone still occasionally pay for the expensive human-run ones?
While it's debatable how that will turn out, the value of human creation feels a lot more significant when it comes to art and literature, but I'm starting to realize that for many others there's no fundamental difference between consuming art and eating pizza.
This is speculation, but funnily enough Gemini models show the least evidence of being distilled from OAI or Anthropic compared to everything else in the market.
Well, tokens cost money, and in theory, if someone else already built something you need that costs a billion tokens, it makes more financial sense to use theirs than prompt your own.
They would still have trillions invested across EU and US, if their relevance bothers you then you should direct your anger at the leaders and voters who made them as relevant.
I'd be surprised if major labs aren't doing any diffusion experiments, and if they are, it would also be surprising if another lab is ahead of them in diffusion given just how much more compute they have at their disposal. My intuition could be way off though.
I think the term is wire-heading and it does feel like an inevitable development when you think about it, assuming the technology gets there.
It's also possible that our expanding understanding of ourselves and our desires would lead to entirely different outcomes that are impossible to anticipate from our current POV.
There should be a name for this cognitive bias, not accounting for what someone is not doing.
It's an awesome performance, but I don't know why you'd want this in your actual day to day car and pay a premium for it, all power to you if you do but I wouldn't call not sharing that preference 'poo pooing the incredible tech'.