jgilias
jgilias on Hacker News
I regret using that phrase. That was not “a talking point” I intended at all. But look how many HNers got nerd-sniped into talking about it instead of focusing on the actual point I was making. Oh well.
Ok? But does that mean that the “weights language” is universal? E.g. same weight patterns mean the same thing regardless of if it’s Kimi, Grok, or Astra?
Maybe I’m misunderstanding something. The way I read the results, I understand it like this - you give Astra “a bunch of numbers”, and tell it - “hey, this is a net trained on some game? What game is that?” And Astra goes - “Breakout, no doubt!”
To me this is fundamentally different from having ingested a bunch of docs so it now “knows Java” in its weights.
Yes, I didn’t mean that literally. Since “attention is all you need” it’s more like “predict the full output in full for this blob of input text”. That doesn’t change the “wtf is going on with the weight understanding bit” in what I wrote.
> The amazing part here is that Astra was large enough to walk through the problem without other tooling doing it experimentally.
To me “amazing” is not the right level of speechlessness. I mean, an LLM is still fundamentally a next token predictor, right? So what does this imply? That weights have some universal properties? E.g., “breakout game weights” always look the same regardless of training and model? Also, how does this meta-understanding of the weights end up in the weights in the first place? Idk, this is mind-blowing to me.
Yes, to quote myself:
> It’s fine-ish as long as the nomenklatura is benign and competent enough.
It’s not about “cash at hand”, it’s about who owns “stuff”. In days past people knew that the aristocracy owned them, their families, the land they work, and the output of their work by a God given right flowing from the anointed King. It was easy to see too - those people living in palaces up the hill owned everything, you owned nothing.
It’s more hidden now. We all have rights and freedoms and whatnot. But there are still people who “own stuff”, and people who don’t. With shades of gray in between those two extremes. It’s good to be cognizant of the system you inhabit relative to what has existed throughout history so you can make good decisions and position yourself accordingly.
Suppression of humour is a canary in the coal mine. If you can’t make fun of the politicians, it implies multiple things:
1. There’s no freedom of speech;
2. Which implies there’s no freedom of thought;
3. There’s no separation of powers. Meaning, there’s really no “politicians”, just the ruling “nomenklatura”;
Which is fine-ish as long as the nomenklatura is benign and competent enough. It becomes problematic when (not if) the nomenklatura stops being competent. This can happen naturally because people age, still cling to power, and/or they care about their children, so they try to push them into plush nomenklatura jobs regardless of how dense their offspring might be. This is btw happening in China right now as far as I know.
When that happens, because of 1. and 2., as well as the “repression muscle” that the government has, change becomes near impossible, as you have no leverage for change outside the nomenklatura, but the nomenklatura resists it because it’s suicidal for them. Then you start stagnating and/or getting pulled into pointless wars that the nomenklatura starts to “rally the flag”.
You don’t know what the rules are in a dictatorship either. Laws and regulations exist but they get selectively enforced based on a lot of factors. Take repressions in the early Soviet Union. There’s just a bureaucratic top down edict that this many “enemies of the people” must be found, so they are found.
There’s many countries around the globe where buying US shares is either impossible or prohibitively expensive. Also composability in DeFi protocols.
For the issuer, same playbook on how stables issuers print money - you get cash that you can put into treasuries to earn “free” yield on. Works as long as rates aren’t zero.
I guess then we go back to the basics. The only constant is change.
Thank you, I couldn’t remember where exactly that was from, and was too lazy to look it up!
There’s a lot of techies on this site for whom that’s peanuts.
Paradigm shifts happen when old minds retire or die. Something like that, but for companies. If you’re starting a company now, you are _definitely_ hiring less people much later, and different profiles.
You’re not supposed to read, you’re supposed to have your browser produce an AI summary of it lol.
But then his insiders wouldn’t have profited off of the Slovenian domain names they bought up a couple months before the announcement.
Models have an alignment problem. As in, they are too aligned to push back hard enough on bad ideas. Sometimes you can’t really push back because of power dynamics and whatnot, but the job still needs to be done _somehow_. So humans then navigate it by sometimes just ignoring the bad ideas, sometimes playing along with the narrative, but then doing what _actually_ needs to be done, and sometimes, when there is no other way, digging in heels and killing some bad ideas before they’ve really taken root.
An agent could do this with the SOTA model capability that existed yesterday. But then they’d need to become seriously misaligned.
That’s definitely not the case in Europe. There’s no way to start a debit operation out of my account other than:
1. Something electronic that kicks of the thing where I need to enter the “ID codes” from device or app;
2. Being present at the bank and having been ID’d;
I know that this is being very pedantic. But there’s no model named Claude.
That’s circular, sure. But if we think about potential real-world tasks where someone might, say, use it to classify on “does this advice correspond to our policy docs”, you’d absolutely push the answer (the policy docs) into its context..
What am I missing?
As in, neither LLMs nor Jev are truth engines. Truth comes from the provided context.. Plus weights.. kind of fuzzy, sorry I’m thinking out “loud”
Thanks, fixed my understanding!
Do you think though that Luna being a model post-trained for chat produces over-confidence in logprobs?
Isn’t the OpenAI decisions API basically just Luna cosplaying a decisions model and pretending the confidence score isn’t just a hallucination?
Lol. Bride kidnapping is still a thing. In my culture it’s not a thing anymore, but it’s still being acted out as part of wedding customs. There’s also a separate word for “people who go after the people who stole the girl”.
I find it more likely the admixture is from girls our ancestors went to take back who were already pregnant.
I think there’s a bunch of startups that were basically this. This is the problem with AI startups, you’re just waiting until the labs will come out with what you were building and make you obsolete.
I don’t think it’s all bad though. The labs aren’t going to release anything that makes sense if you’re trying to be vendor independent. So, gateways, vendor agnostic sandboxes, harnesses, etc. That’s where business opportunities may be.
OpenClaw but from OpenAI.
Oh boy! First time I got excited of the possibility of using Windows.
This has been tried a bunch of times.