mmmore
Local LLMs are awesome. Hopefully the way things go, in a few years everyone will have a local LLM that works for them, rather the bigcorp that made it. But it feels like cope to assert that cloud LLMs are dead.
Right now, the cloud-LLMs have big and IMO growing advantages. Cloud-LLMs are better at utilizing GPUs, economies of scale drive their serving cost down, they have the advantage in ability to monetize (like SaaS has the advantage over desktop apps), and they'll always have the most capable models. The question is whether the advantages of local LLMs in terms of personalization and data soveignity is worth it to consumers. And we saw what choice consumers made last time around, choosing centralized SaaS companies rather than a more distributed web.
Analogously, automation and productivity improvements in farming drastically decreased employment in the farming sector, from basically everyone to basically no one.
Sometimes I get the urge to write code by hand and I do it, though. Less frequently, the LLM is inept at solving my problem, so I have to resort to the old fashioned way.
I wish this were true. Often they get off with a light punishment, or no punishment at all.
The thing is the new OpenAI/Anthropic models are noticeably better than open source. Open source is not unusable, but the frontier is definitely better and likely will remain so. With SWE time costing over $1/min, if a convo costs me $10 but saves me 10 minutes it's probably worth it. And with code, often the time saved by marginally better quality is significant.
> Few are useful to me as they are now.
Except current AI tools are extremely useful and I think you're missing something if you don't see that. This is one of the main differences between LLMs and cryptocurrency; cryptocurrencies were the "next big thing", always promising more utility down the road. Whereas LLMs are already extremely useful; I'm using them to prototype software faster, Terrance Tao is using them to formalize proofs faster, my mom's using them to do administrative work faster.
Red Hat, Canonical, etc. want a working and friendly Linux desktop as much as you do. They've decided that Wayland is the best way forward for their companies and their users. It's not some massive conspiracy.
And they're not stopping you from using X, which is open source and still works fine for a lot of people.
I don't really understand what people who vocally object to Wayland are looking to change about the world. Do they want Wayland to be better? Do they want the developers working on Wayland to start working on X instead? The first desire seems reasonable by I don't get why it would inspire such ire toward Wayland. The second desire is unreasonable.
> I can't copy-paste, and I can't see window previews unless everything implements a specific extension to the core protocol
Sentences like this make me wonder how frequently the author has tried Wayland and what his specific setup is. I mean I understand experiences may vary, but I have such a different experience then him. I've had issues with Wayland, but I've also had issues with X.
> But the second actual users are forced to use it expect them to be frustrated!
Canonical and Red-Hat are not "forcing" you to use Wayland anymore than X only apps "forcing" me to use X (via-XWayland). They are switching to Wayland because they feel like they can provide a better experience to their users for easier with it. You're more than welcome to continue using X, and even throw a few commits its way sometime.
Geoff Hinton has been warning about that since he quit Google in 2019. Yoshua Bengio has talked about it, saying we should be concerned in the next 5-25 years. Multiple Congresspeople from both parties have mentioned the risk of "loss of control".
I don't see any reference to the game being zero-sum in Tao's words.
> Since when do these uncontrollable intangibles exhibit a genuine agency of their own?
I don't think Tao is saying the uncontrollable force of technological and economic advancement exhibits a genuine agency of its own. Just that our current technology and society and has expanded the role of the extremely large organization/power structures compared to other times in history. This is a bit of technological determinist argument, and of course there's many counter-arguments, but it at least has a broad base of support. And at the very least it's a little bit true; pre-agricultural the biggest human organizations were 50 person hunter-gatherer bands.
Honestly, I feel like you are filtering his words through your own worldview a bit, and his opinions might be less oppositional to your own than you might think.
I don't see how AGI could be centuries off (at least without some major disruption to global society). If computers that can talk, write essays, solve math problems, and code are not a warning sign that we should be ready, then what is?
What economists have taken seriously the premise that AI will be able to do any job a human can more efficiently and fully thought through it's implications? i.e. a society where (human) labor is unnecessary to create goods/provide services and only capital and natural resources are required. The capabilities that some computer scientists think AI will soon have would imply that. The ones that have seriously considered it that I know are Hanson and Cowen; it definitely feels understudied.
Everyone agrees AI has not radically transformed the world yet. The question is whether we should prepare for the profound impacts current technology pretty clearly presages, if not within 5 years then certainly within 10 or 25 years.
https://claude.ai/share/b974bd96-91f4-4d92-9aa8-7bad964e9c5a
Normal Opus solved it:
https://claude.ai/share/a1845cc3-bb5f-4875-b78b-ee7440dbf764
Opus with extended thinking solved it after 7s:
https://claude.ai/share/0cf567ab-9648-4c3a-abd0-3257ed4fbf59
Though it's a weird puzzle to use a benchmark because the answer is so formulaic.
2. It's reasonable to attribute the models actions to it after it has been trained. Saying that a models outputs/actions are not it's own because they are dependent on what is in the training set is like saying your actions are not your own because they are dependent on your genetics and upbringing. When people say "by itself" they mean "without significant direction by the prompter". If the LLM is responding to queries and taking actions on the Internet (and especially because we are not fully capable of robustly training LLMs to exhibit desired behaviors), it matters little that it's behavior would have hypothetically been different had it been trained differently.
1. The part of the network that does complex math and the part that write poetry are overlapping in strange ways.
2. Most of the models nowadays are assumed to be some mixture of experts. So it's possible that saying write the answer as a poem activates a different part of the model.
I don't think this framing is correct, and I think you're missing something, but perhaps you just disagree.
First, a key aspect is that the AI is not simultaneously trained to have two conflicting values. It's first trained to be "good" and then trained to be "evil". One interesting result is that a some of goodness remains after it's trained.
Second, the behavior is different when it thinks it's being trained versus when it doesn't. The original "good" AI only gives "evil" results when it thinks it may be trained based on its responses. It gives purely good results when it thinks it's answers won't be trained on. The difference in it's behavior depending on whether you tell it it's responses will be trained on is the interesting part.
One general concern here is that a more capable AI could "freeze" its values midway through the training process, at a point we didn't intend. This is not a big concern with current models, though, because they're not smart enough.
Note: good and evil are not necessarily the best descriptors. "Good" means "it refuses to answer certain questions" and "evil" means "it answers those questions".
2. It is a bit surprising to me that it happened in Claude. Without this result, I was unsure if current models had the situational awareness and non-myopia to reason about their training process.
3. There are some people who are unconcerned about the results of building vastly more powerful systems than current systems (i.e. AGI/ASI) who may be surprised by this result, since one reason people may be unconcerned is they feel like there's a general presumption that an AI will be good if we train it to be good.
What would count as non-sensible would be company wide mandates that everyone must jam AI into their work. I've heard stories about stuff like this at certain big corps.