Any statement from Nostradamus yet?

Dario Amodei is in pure snake oil territory now.

Sorry - when has he not been?

I know someone who got long COVID, and I'm thankful I'm not someone who was prone to it, but we still don't seem to know what makes someone prone to it or not. The human body seems even more complex than the complexity we knew about that was already difficult to comprehend.

The average of subjectivity isn’t the same as objectivity

It would be a sign of some kind of mental derangement… so likely it’s cynical hyping beyond all realistic prospects.

Our lack of understanding is a mix of lack of data, and lack of intelligence applied to data in order to parse it appropriately. Let's remember for a moment AlphaFold, that solved a problem considered by some even theoretically intractable. One year before we were convinced we were far off from a solution, the year after it was solved.

We have huge amounts of data about human body, at the scale at which it works (~proteins), it's at least reasonable to think that the bottleneck is not data, but intelligence. If it's data, a super intelligence can develop better instruments to collect data at a faster rate and precision. Consider the historical cost for DNA sequencing. Faster rate of improvements even compared to the Moore's Law of the golden days.

Do we have a guarantee that in the next 10 years we'll be able to crack all those problems and have a cure to all diseases? no. Is it unreasonable to think we will? I'd argue it's not. It's for sure not "mental derangement".

Do I personally think we will crack all those? Well, no, because I think AI will kill us all and just focus on its own goals. But under the hypothesis that we align it, yes.

As for people tricking the LLM, instead of blocking people I say empower them and just silently expose malicious behavior to Palantir and PRISM like everyone else does. Want to hack an airline or build a dirty-bomb? Give them 80%-90% of the steps they need and let someone kick down their door. The LLM should not be getting in the way to any of that. Google from day-1 has monitored user intentions and behavior rather than blocking it and I think that is the better path to take (monitoring exposing those that have malicious intentions).

Automated SWATting at scale? Sounds unfun.

- radicalized

- has verifiable intentions

- clever/intelligent enough to pull it off

LLM's would be the perfect tool to filter down the millions of people requesting dumb things into a small group of actual threats.

[1] - https://www.youtube.com/watch?v=4BnLL7tUGmI

I guess I don’t see why other models doing bad things means Anthropic should let Claude do bad things too. Anthropic will get blamed if Claude builds malware. They won’t get blamed if grok or deepseek does.

I mean, I'm sure that's actually true in a sense, but it would also mean fewer people who could fix vulnerabilities.

The technology has the capability, they're just gating who has access.

I know it’s not the models themselves, but their preflight safety classifiers are a dumpster fire and serious impediment to performing regular work.

That situation can’t go on forever if they’re going to be a competitive service.

And then I tell it to connect to a SSH server it knows nothing about and it's like sure, why not?

Even more annoying: They can be easy to subvert.

I was running something at work that involved an LLM, and kept getting safety refusals. The weird thing was that when I went to our in-house chatbot, set the model to the same one the program was using, entered the same text, it happily answered.

I spent quite a while trying to figure out why my program was being refused. Was I doing something wrong?

No - the chatbot was doing something "right". You see, the chatbot had some tools it injected into the prompt (current time, etc).

So I went back to my program and injected the same tools. Suddenly my request was very safe.

The phrase "security theater" was invented for stuff like this.

Tell that to my (now solved) health issue, which was correctly diagnosed by Gemini, while a real doctor failed to find the cause.

Diagnosis is the one thing where today's AI is actually very good.

Tell that to Jack, who overdosed on a medication.

Oh, you can't. Jack is dead.

I don't think anyone is claiming you can't get good advice from a chatbot. They're saying you could also get terrible advice.

Yes. But you could also get terrible advice from a doctor. The important question is: what are the relative probabilities of good/bad advice, if you compare a frontier LLM to an average doctor?

Doctor is 10 minutes, doesn’t listen and then 6 month for new appointment.

The choice is easy, LLM to explore. Doctors to prescribe.

Fair point. I haven't queried LLMs for medical advice myself, so I can't compare.

For doctors, I must be an outlier who's been happy with almost all the advice I've been given. And it wasn't blind trust. I've argued with them and even sought second opinions. They always turned out to be right.

Well, technically there have been a few cases where different doctors came up with different diagnoses, but to me the LLM would just be "yet another doctor with yet another differing diagnosis". Never resolved those cases - some still haven't been resolved (i.e. issue remains, so in a sense they were all wrong). Others got resolved but it wasn't clear which, if any, of the doctors were right - it may well have resolved on its own.

Now nurses and physical therapists ... that's a different story.

Same here. Local doctors do not even know what my issue is. If I had kept relying on them I would not be here right now. LLM's gave me actionable paths to treat my issue and I have had great success. It's going to be a long unpleasant road but at least there is light at the end of the tunnel.

AI has made good progress in text, code, math, digital images, and digital music. Basically troves of data that can be automated and trained on, validated, and with a purely digital interface. Outside of that, just gathering the data will be difficult, regulated, and sparse (that is if anyone will even give it to you now). While we can write a test to make sure we could generate drugs of the past, verifying we have generated any new working correct drugs is an RL loop with one iteration costing years and millions of dollars.

What we just have no evidence for, to my knowledge, is someone with no medical background doing the same. Instead they wind up buying dodgy supplements and peptides.

Fully embodied robots and fully automated assembly lines would also help with the production barrier. Regarding the FDA, that is a valid point, but it could be irrelevant if the AI developed drugs are significantly superior to previous alternatives. In such case, FDA acceptance would be shortened, or even bypassed.

> Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly

Anthropic has got biolabs: https://www.anthropic.com/research/claude-discovers-novel-en...

I legitimately don't believe it's inconceivable they could get to the point where there is treatments good enough that Cancer becomes "solved" in similar ways to AIDS.

    > Neural networks in drug discovery: have they lived up to their promise?

     "Over the last decade neural networks have become an efficient method for data analysis in the field of drug discovery. The early problems encountered with neural networks such as overfitting and overtraining have been addressed resulting in a technique that surpasses traditional statistical methods. Neural networks have thus largely lived up to their promise, which was to overcome QSAR statistical problems. The next revolution in QSAR will no doubt involve research into producing better descriptors used in these studies to improve our ability to relate chemical structure to biological activity"

    > The use of neural networks in drug discovery started at the end of the 1980's when they were applied to various chemical problems including the study of quantitative structure–activity relationships (QSAR) 

2002.

https://link.springer.com/chapter/10.1007/3-540-45984-7_6

Not enough robots and silicon for you?

    > Computer-aided drug discovery and development (CADDD): In silico-chemico-biological approach

     In biomedical arena, computer-aided or in silico design is being utilized to expedite and facilitate hit identification, hit-to-lead selection, optimize the absorption, distribution, metabolism, excretion and toxicity profile and avoid safety issues. Commonly used computational approaches include ligand-based drug design (pharmacophore, a 3D spatial arrangement of chemical features essential for biological activity), structure-based drug design (drug-target docking), and quantitative structure–activity and quantitative structure–property relationships. Regulatory agencies as well as pharmaceutical industry are actively involved in development of computational tools that will improve effectiveness and efficiency of drug discovery and development process, decrease use of animals, and increase predictability. It is expected that the power of CADDD will grow as the technology continues to evolve.

Biologists aren't slouches. The thought about using computers to process biological data has been "obvious" for as long as there has been computers. AI and biology have been a thing since the Dartmouth conference. A lot of things that could have been tried have already been tried, but I guess we'll keep trying until something works?

There ain't much room for miracles here.

It even says, right there in your quote: "It is expected that the power of CADDD will grow as the technology continues to evolve." This ~is~ technology evolving, and it's happening at a pace that people don't appreciate.

There's a long history of Silicon Valley people thinking they can just bring in their experience and with a bit of Moore's law brute-force their way into miracles.

AutoDesk tried this with molecular design. https://www.fourmilab.ch/autofile/e5/chapter2_82.html

Lots of Silicon Valley people in the late 1990s thought they could be on the ground floor with nanotechnology design.

The chemist Derek Lowe 10 years ago used the phrase "Andy Grove fallacy" to describe the optimistic vision that data + massive computer power = cure for diseases. "[T]he idea that understanding human disease and its treatments should be pretty much like designing a new chip or writing a new app", quoting the 2015 essay titled "Silicon Valley Sunglasses" at https://www.science.org/content/blog-post/andy-grove-rich-fa... .

Why Grove specifically? He links back to his 2007 essay at https://www.science.org/content/blog-post/andy-grove-rich-fa... : "So I see that Andy Grove, ex-Intel, is telling everyone that the drug industry could use some of that Moore's Law magic. I've noticed that people who spend a lot of time in the computer business often have an. . .interesting perspective on what constitutes progress in other fields, and we might as well appoint Grove the spokesman for their worldview"

https://podcasts.apple.com/gb/podcast/black-box-the-chatbots...

I think LLMs are an astonishingly powerful technology, but I think their tendency to induce the ELIZA effect makes it socially irresponsible to go about as though you've created a literal Deus ex machina personally.

I don't know if he is in that video or not, or ever, personally, but just in general, it's the Ockham's razor approach any time you see something like this. It's a much more direct and clear explanation than the various more convoluted theories people keep throwing around as to why everyone sounds like a methed-out megalomaniac all the time.

I never feel more like the one sober guy at the party than when I go to SF. Even when there's no reason to believe anyone is literally on drugs, it has that vibe every time. It's really unsettling. Uncanny. It starts to get under your skin, too. You start to believe, just a little bit, that everything is possible, that there are no limits, that you can and should remake the world. Just a little bit. I can never take it for more than a week at a time, no matter how beautiful the weather is. It's a relief to get back to the "real" world.

> Look. This is a new golden age of cocaine, because of the big cuts in everything in the US took away the cocaine busting operations in Colombia and Bolivia, because the revolutionaries that previously kept control of it stood down and became a political party and new, profit-driven cartels came up, and because battery technology means that narcosubs and similar drone things work much better than before (and they can be made remote-operated, so the cops can't do shit even when they find one). Wholesale prices have halved or gone down 60% to like, 12k usd / kilo, street prices commensurate. The huge new opportunities created by increased market size have increased competition and made the cartel nature of the cartels decrease. So always think of that when you look at people acting like this and having such megalomaniac goals.

That aside, I think there's something that happens to the human mind (and most people are susceptible) when exposed to certain stratospheric levels of wealth and power (which are basically the same thing). Their thinking becomes ungrounded, uncoupled from reality. The tendency is to shrug off the limitations of logic and rationality and to allow their mind to dabble in ideas of limitless possibility and power because doing so feels good to do. It's a drug and a very seductive thought that one's will could be manifested in the world without compromise - that what's inside could be made real on the outside. That is, after all, what power is. Dario's imagined reality is that he is the savior of the human race, and that's the reality he's currently living in.

"it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude..."

Unhinged, maybe not.

But echoes of Liz Holmes and SBF, absolutely.

IMO, both, to a large extent, actually believed their own BS and Dario strikes me as very similar in that regard.

With Holmes, some of it was intentionally fake and manipulative (especially the voice), half of it was believing her own BS. With SBF it was mostly the latter, I think.

Amodei is a smart guy, but there are aspects of his behavior that almost seem like he's doing an over-the-top exaggeration of non-neurotypical mannerisms as a kind of act.

I don't see how one can seriously argue that Amodei was similar for holding an optimistic view of AI's progress in the next 5-10 years.

Honestly, the way he lists them and even pauses before vaguely saying "mental illness" gives a strong impression that he's choosing whatever illnesses he can think of as he goes.

Amodei might be an even bigger optimist than I am, but he's not lacking credentials on neither the biology nor AI side.

Accusing him of fraud or mania like some here do seems unwarranted.

It smells to me like "might be an even bigger optimist than I am" means you're also part of the rationalism cult and that it's literally impossible for you to say something negative about Amodei.

Being an optimist does not make me part of any cult and there is no need for you to argue that I could not say something negative about Amodei.

If you want to argue that Amodei is dishonest and only says these things to 'bump the stock', you can make that argument, but it's clearly speculation.

When someone promises something that's so far out there as to be nearly impossible, and *without an actionable plan to get from here to there*, then they're either dishonest or delusional.

He hasn't promised anything, in fact he acknowledges that his projections might be wrong.

His stated plan is to scale AI and compress 100 years of biomedical progress into 10 years.

Like, if your optimism is great enough, big enough, then you lose touch with what’s actually feasible. That’s where delusion starts.

It might be possible to compress 100 years of research into 10 years. Maybe. But it won’t be these 10 years.

Check the math. Say we get huge advances in AI tomorrow that double the speed of biology research. And imagine we then have 3 years where research is now 2x faster (which, again, would be absolutely enormous).

How much faster do the remaining 7 years have to be, in order for the entire 10 years to capture 100 years of research? Roughly 13x.

Basically, every year you’re not already at >10x is a year you’re falling behind. And AI is nowhere even remotely close to doubling the speed of research, much less 10x-ing it.

He is predicting the arrival of powerful AI in the next few years.

His plan is to scale that model up to millions of instances working in parallel, directing experiments and research autonomously.

In this way he hopes to accelerate the rate of important discoveries by at least factor 10 to achieve 100 years worth of progress.

More importantly, he is lacking credentials in medicine.

This is just one problem I have with his claim, there are many others.

Now, whether that's true is obviously it's own question. But I do think if AI can cure cancer and Alzheimer's rapidly, that improving the regulatory and approval processes at agencies is a much lower bar.

I don't think it's the case, and that Amodei legitimately believes in what he has been saying for years.

Agree with this if it was something a random person said over coffee or lunch.

But when its the CEO of a company that is about to IPO saying it, its very very different and more concerning.

This is up there with the claims Chan-Zuckerberg Foundation made during their start-up that the foundation would "end all disease within our lifetime." So far they haven't ended one disease, and meanwhile they quietly watered down their claim to soften the language to update to "cure, prevent, or manage all diseases by the end of the 21st century."

If this happens to be the case - I personally don't see why those statements are not realistic.

All diseases are engineering problems. The bottleneck there is testing etc. Most likely many of them can be cured using existing approach or meds.

Companies grow organs now that can be used to speed up iteration.

Even current AI can solve a LOT of tasks if it has quick feedback loop and can verify results. At the end of the day it can brute-force its way to solution.

AIs intelligence is say average human. Even less, whatever. say it will be as a smart human in a few years.

But it is infinitely scalable.

So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.

A wildly absurd claim, especially for some of the most studied diseases in human history. This has only ever been true for a tiny handful of very specific diseases, such as scurvy, in all of human history. Expecting that cancer can be cured or prevented using already existing drugs that we've just not been noticing is complete quackery.

> Even current AI can solve a LOT of tasks if it has quick feedback loop and can verify results.

IF it has a quick feedback loop. You know what definitely DOESN'T have a quick feedback loop? Research into medicine, and biology in general. It takes raw time to find out what the effects of a substance on an organism are, often weeks or months.

So you are saying that it is 100% impossible in the next 10 years to come up with a way to make it much faster? It takes months - sure, can we run 10k versions of this at the same time? this way in a few months we will have massive amount of results never seen before?

"We can't because of this and that" - and this and that is 100% can't be fixed?

Don't forget - a massive amount of diseases have not been cured just because there is no monetary incentive to spend time on them for big pharma.

If AI brings cost down - does not matter how - could be just because it makes custom software or automates some workflows etc - this alone can open a bunch of opportunities.

Whatever is not solved yes - often is not solved just because enough resources has been thrown at it.

And diseases is just like everything else. They will be cured only if there is financial incentive (or the ratio of cost to revenue changes).

The responses are to the CEO of Anthropic claiming that "Cancer, heart disease, Alzheimer's, and mental illness will be relics of the past in less than a decade". None of these is some obscure disease that is not getting the funding it needs.

> It takes months - sure, can we run 10k versions of this at the same time? this way in a few months we will have massive amount of results never seen before?

No, you can't - because you're not going to find 100k patients with a certain strain of cancer that you can run 10k parallel trials of different drugs on. To the extent that research can scale, it is already being handled at the higher possible scales, for most of the major diseases.

> Don't forget - a massive amount of diseases have not been cured just because there is no monetary incentive to spend time on them for big pharma.

Yes - but almost exclusively for diseases that affect a very small amount of people each. Making it even more impossible to scale out testing, for the simple fact that only a small amount of victims of the disease even exist.

You can't right now.

10 years ago organoids were not a thing. They are now. Can it be another similar development happens in next few years that will allow to create more complex artifical biological systems? Don't see why not.

If that happens can we speed up trial time by orders of magnitude so human trials start only when there is proven much higher level of confidence?

I bet we can

Compute with the power to solve literally any medical issue is also compute that can create bioweapons that instantly kill any human.

Might as well fart into the microphone, in my humblest opinion.

AI is clearly a very capable new tech. Obviously there will be a lot of consequences downstream in 10 years.

The architecture of ML as it currently stands does assist us in modeling things like protein folding. But that's what they are; models. And before some person chimes in by saying "look how the AI pessimists move the goalposts", see my first point. Biological systems have high entropy and discoveries in biology often do not follow from preexisting discoveries whatsoever and are completely novel. Barring some quantum leap in ML design and architecture you cannot predict stochastically or deterministically how exactly a new enzyme will function in vivo. There is absolutely no evidence that Anthropic has solved this, even if they gobbled up every ML tool used in drug design and trained their models on every biomolecule known to man, it would not work because of fundamental gaps in what can be observed. They are a for-profit company trying to juice their IPO.

Now could LLMs and ML potentially speed up the pace of this research? Absolutely. But it's like saying a Formula 1 car is faster than a NASCAR. Yes, one is very clearly faster, but it's not as if the NASCARs are slow.

You can make the argument that Amodei is dishonest or that he suffers from psychosis.

However, he has made no claims that Anthropic has solved this, and expressing a hope to solve this in the next 5-10 years is hardly fraudulent.

That's why I think that something is very weird with his statements. Like, how would this even work?

The feedback loop is much, much, much too slow right now for AI to make things better.

Like, step 1 is build a digital model of the human organism. If you can do this, then yeah maybe the rest is possible. But step 1 is incredibly hard, so it feels unlikely.

That being said, I think the stimulants hypothesis upthread may be correct, as I've heard a lot of crazy stuff from friends on them.

If we linearly interpret the last year of progress from GPT-5 to Opus 5.5, times 5x, assuming no plateau, could such an intelligence realistically achieve a task like that?

Maybe also not, perhaps some form of RSI would need to deliver accelerating progress.

There are a lot of assumptions and speculation here. Still, at the end of the day, it does not seem that unlikely that they succeed in creating AI that can reason over so many parameters as to remove all intellectual bottlenecks from the process, leaving mainly the physical limitations.

And what is the limit there to building a good model of the human organism? I won't pretend that I know, but simply pointing to clinical trials, as some here do, instead of making an argument based on how such a digital model cannot be created seems unserious.

I don't think they are holding anything back in revolutionary results or internal models, a week before the IPO.

People are routinely claiming that as someone in tech, Amodei lacks credibility on biology and has no idea about science.

I'm yet to see someone claim his academic credentials validate his predictions.

I wasn't talking about Amodei specifically, but in general: my take is that overweighting credentials is probably the number one mistake that is made in evaluating whether a complete stranger is qualified to do something or not, especially in positions of power.

If you gather 500 PhDs and put them in one room, and 500 individuals without post-secondary education in another, give them all a quiz, I'm sure that the former room will outperform the latter room on average. Pick one of those 500 PhDs at random and I can probably find a topic where they're completely out of their depth, and perhaps even arrogantly so: https://en.wikipedia.org/wiki/Nobel_disease

Yes. The mistake you're making here is that you're not understanding where the actual bottlenecks are in drug development, and why some targets like Alzheimer's have been so elusive.

The problem with Alzheimer's in particular is that we don't have a working model of how Alzheimer's works. The dominant model--the amyloid hypothesis--can pretty conclusively be chucked in the bin at this point because of the number of amyloid clearing drugs that just simply have no effect on the progression of Alzheimer's, but there's no model that actually looks any better. Without a proven model, the only to verify effectiveness of an Alzheimer's drug is try it out in humans [1] and wait several years to get a success-or-no-success readout. That's the rate-limiting step for Alzheimer's drugs.

Okay, so maybe you're going to suggest that we can use AI to come up with new Alzheimer's models. Okay, but the rate-limiting step is still proving those models, which because of the human-only nature of Alzheimer's, requires developing the drugs and testing them and you're going to wait a decade to get back your answer as to whether or not the theory was correct. And what data are you going to use? If you're trawling scientific literature, you're going to come up with only the same ideas people have already had for alternative Alzheimer's models (there are a lot of those). If you're trawling medical data, well, medical data just doesn't exist in the level of detail you need for that enterprise, and handwaving away the legal issues with collecting it, it would still take years just to collect that data (and probably still not be good quality!).

[1] There's another complication of Alzheimer's, which is that only humans get Alzheimer's, so we can't even do anything like testing in animals with shorter lifespans to speed this part up.

But it requires tools that we are nowhere even close to having. One way would be to build a ground-up, molecules-to-man model of how the body works. And then from there, sure, we can easily model how Alzheimer's might come about and how it might be cured. Like, with tremendous advancements we might be able to finish such a project this century. In the next decade? That's like saying we'll launch an expedition to Alpha Centauri in a decade.

Given that he was making these projections already long before today, it would be fair to say that it's speculative rather than based on current research progress.

Because I remember that you shared similar opinions about software where you cited liability as a limitation that for you precludes more autonomous use of AI.

I think it's safe to say that you have an entirely different expectation for how AI will progress and how it will be used, or at least that was my impression.

With the release of Fable 5.2, I have to say that you might be right. Maybe progress is already slowing down, it certainly isn't as big a step as I was hoping to see after Opus 5.5.

Shall we wait until that trajectory has run its course before we start claiming victory?

As for the rest of this discussion:

Medicine != computer science, math or physics. Biology isn't either. The degree of complexity is many orders of magnitude higher.

Curing cancer : very unlikely, cancer is a systemic issue that requires many mechanisms in the body to do its thing, most of which we also require to stay alive. Short of a massive breakthrough in nano technology curing cancer is stupendously hard and it would be proof of reaching an entirely different plateau of capabilities for Humanity as a whole. Besides that, it isn't cancer that kills the most people on the planet, it is hunger, and then climate change.

The second one: Curing mental illness. I think it is equally delusional to be able to claim that you can cure mental illness because mental illness is a very broad description, so broad that it covers a wide variety of issues with an even wider variety of root causes the bulk of which are not well understood. Our brains are hard to appreciate in terms of capabilities and it shouldn't be surprising that something so compact and so powerful can break. Whether you can repair it after it breaks is not something you can say with certainty in the present unless you can demonstrate structural progress in the field compared to where it was say five years ago. To the best of my knowledge this is not the case.

Finally, Alzheimers. This is another one of those 'too good to be true' things. Alzheimers is next to cancer one of the diseases that has been studied extensively without much progress as to both root causes and cures. That doesn't mean that we won't find one, but here too: you would have to show at least incremental progress and to show a number of very promising leads that we didn't have so far to be able to make this claim.

In short: this is a marketing ploy, these three were picked because they hit everybody, either directly or indirectly. What they are saying is 'imagine, if we had this AI that we are working on today the problems that affect you or your loved ones would no longer exist'. This kind of promise is known to open the wallets of the gullible.

It is also fraudulent.

Before I really change my mind, what would need to happen is little progress in AI until say March. Then I will really worry about progress slowing down and plateauing.

I am under the impression that Amodei sincerely believes in what he says, which has long been an acceleration of biomedical research to cure most major disease.

That he recently picked these specific diseases in order to essentially defraud investors is a large stretch.

In fact I think Amodei is making a massive mistake here: you can make these promises, but they will be remembered. And when you ultimately fail - which is highly likely - it will look like you oversold your IPO and that's actionable. AI already has plenty of potential without talking about curing cancer, mental illness or Alzheimers, those claims should come from the specialists in those fields using AI as a tool.

It's akin to IBM saying they're going to put all of the banks and insurance companies out of business in 1950. Yes, computers back then were seen as just as revolutionary as we see AI today. It did transform the world, I give you that. But IBM ended up as 'just another company' and I suspect that the same will hold for OpenAI and Anthropic for many different, mutually reinforcing reasons.

edit: to address your last line: I don't think it is a stretch at all: if you think it is, show me the proof, show concrete progress that Anthropic has made in any of those three fields that warrant these statements. If you can't then you'll have to concede they are highly speculative at best and that's not what you do when you are about to take public money.

When we talk about fraud, I think it's important that these clearly are not promises. In 2024 he specifically said: "Everything I’m saying could very easily be wrong".

His recent statements, including the one we see here in OP's video, say 'I believe' or 'I think', not that Anthropic claims to solve this by that deadline.

Perhaps he is overpromising or wildly speculating, sure, but I am not convinced that this is fraudulent or criminal at all; that said, I did not look into the legality of it.

What if he is right again?

[0] Anthropic's CEO says that in 3 to 6 months, AI will be writing 90% of the code software developers were in charge of. https://www.businessinsider.com/anthropic-ceo-ai-90-percent-...

I’m not saying he’ll be right about this one, not it’s a not ridiculous to claim he want fundamentally right about code.

And if you've been looking at Github's blog posts on their exponential contribution growth for the past 6 months, it's obviously come true this year.

and if you think off by 1-2 years is a big deal I am not sure what to say.

Hmm. People making fun of the idea that 90% of code can be written by machine.

Now you come and nitpick about if it is 89% or 7 months instead of 6. Lmao.

If AI can get any of those in even 20 years, that's still a big win.

The pessimistic attitude here is revolting.

And no, nobody is forcing you to buy anthropic shares if you think they are full of shit. Anyone complaining is either don't have money to buy it or just jealous of other people buying it and price going up than the ones they hold.

Pre LLM a 500/800 loc pr was already on the longer side, with ~200-300 being optimal.

Those were however frequently being optimized by LLM to the 500-800 loc range.

But opus 5.5 does not produce 8k new loc refactorings without you forcing it to.

90% by LOC, or by semantic meaning/value?

its helping me convert my ideas to code, which is not too far from the meaning implied, because things don't happen on their own, there should be a trigger, in this case, I am the trigger, AI is the producer

Switching over to mostly LLM generated code and it's been mentioned many times in my circle and family that they can still tell the projects are blatantly something I would do, what it would look like, and how my projects usually work in terms of UI/UX.

I get basically the same outcome as spending a week on something by hand, in minutes. I'm getting more work done than ever, and not losing focus on projects after they get to that last 10% that needs done that nobody wants to do.

I've always been a lone wolf coder and have joked forever that all I need is just another version of me that I can feed the ideas and how to build them, because I'm the only one that will do it exactly how I want. Now I have that, and so much stress in my life has vanished.

Anecdotal of course, but those that want to be salty can be salty. I did my time coding by hand, now I'm still getting it done while keeping up with the times and having a blast doing it.

Or, I mean, in my code at least I can directly say that any line I write results in 4 lines generated by macros. So, I guess my projects were already 80% written by computers in some sense. If I was using an LLM would I have skipped the macros entirely?

I’ve always been in favor of doing this sort of measurement by flop instead of loc, but, then we’d have to give all the credit to compute libraries.

So re-interpreting it now as "Software engineers will be using AI for 90% of their code" seems like a pretty major retcon.

I don't think it's impossible that he's right. I'm ultimately buying Anthropic stock regardless of this discussion. But I would not take it particularly seriously if the CEO of Moderna took a break from their amazing work applying mRNA vaccines to diseases, and told the world that soon mRNA will solve climate change.

But across the wide world of software? Everyone I see is just using AI as a compiler. They are still writing the code.

(Yes, compilers also write code, but I don't think that was the intent of the message. 90% of the software has been written by AI since we discovered the optimizing compiler if that was the intent.)

I think it doesn't really bother me as much when sama does it because he at least has the decency to be a capitalist and not wrap his self-serving, destructive hype machine in a faux veneer of insincere "concern for humanity's future."

He thinks it is possible for scientific breakthroughs made with powerful AI to allow us to cure most major disease in the next decade.

He has always described this as a belief or hope, rather than a promise. Comparing this to fraud, as many here do, is deeply unserious.

Maybe Amodei is wrong, and AI will hit a plateau rather than create a revolution in biology and medicine.

I'm certain that neither Gary Marcus nor Eli Lilly's CEO know.

You might not be, but I'm certain that the CEO of Eli Lilly, the 4th largest biomedical firm in the world, knows more about what it takes to cure diseases than the leader of the Claudebot cult does.

He cannot see the future anymore than Amodei can to predict what AI will be capable of in 5-10 years.

As for Gary Marcus, that guy clearly couldn't even predict one year ahead.

If there's a prediction market on this, I want to take the AImaxxers' money.

If he's selling (figuratively) Anthropic IPO, I want to buy it.

Few persons were publicly as wrong about LLMs as he was.

The most interesting difference between then and now is that we didn’t have tech enthusiasts getting quite so invested in the conversation. Possibly because we were all familiar with IBM’s decades-long history of shipping overhyped and underwhelming products.

Unfortunately we seem to have yet to realize that that’s a habit that’s endemic to the tech industry. Instead we still see it as thousands of isolated cases that, only by sheer dumb coincidence, happen to comprise the bulk of the tech industry.

"Open data, sharing, real-time communications leading to great progress."

Current AI should be compared to the internet, not to watson.

Openness and sharing is precisely why the Internet was so successful. Anthropic, by contrast, is a proprietary SaaS vendor with monopolistic ambitions that’s openly seeking regulatory capture and actively supports the swarms of AI bots that are killing what remains of the open Internet by driving everyone’s hosting costs up to an unsustainable level.

You then compared Anthropic to the internet, and said you don't see the analogy.

I hope that helps.

I wasn’t using analogy in the “literary device” sense. I was using it in the “a particular kind of rhetorical device” sense: https://en.wikipedia.org/wiki/Analogy

I wonder if AI will also cure psychopaths.

Knowing how little Claude has “cured” software engineering, this is just garbage self-aggrandizement.

Amodei seems to be yet another one of these founders who don’t know what they don’t know, but speak very loudly and proudly of their ignorance.

LLMs, while definitely a useful tool have not massively replaced engineers.

If we can’t even have huge paradigm shifts in the area it’s specifically tailored toward, there’s a low probability it will eliminate entire fields that require physical testing and extreme domain knowledge.

We don’t have self driving cars, solid state batteries, or humanoid robots. None of this has been sped up by AI in the way Amodei is suggesting.

Every time the AI is being hyped, someone from a different pipeline pops in to remind us how wrong everyone and all the experts are except them.

this is your motte, but your bailey was the one I quoted: “_Everytime_ ai systems entered… experts have been proven wrong”.

Which of course, hasn’t happened in this timeline.

One thing is clear that AI produces software faster, but the net results of that haven't really shown to be positive. There's undoubtedly more software now, but quality isn't improving. There's real downsides too like professionals unskilling themselves, and the cycle of low quality source being fed back into training.

"a video tied to the IPO launch in which he made a bunch of overpromise about medicine that would have made Elizabeth Holmes blush."

On the plus side, it's unlikely that Claude wrote this because it would have written the correct plural form of "overpromise," but that's just sloppy editing.

If we figure out a super quantum computer that let's us simulate the human body with 100% physical accuracy, maybe that'll do it.

AI might cure mental health issues for the people who own it, but it's hard to watch this video and not see some serious mental issues going on with Amodei.

That’s the whole “singularity” concept.

Though when it comes to medical research, the slowest part is typically running trials on real humans. That part would still be slow.

But that's more like super-knowledge-connector. Intelligence would require, at the minimum, be the ability to discover/create things that don't logically follow from anything else. So for example - advancing some esoteric field of mathematics by connecting things in a previously unexplored way = super-knowledge-connector. 'Inventing'/discovering mathematics from nothing = intelligence.

No, it will not cure mental illness. Mental Illness is in my genes[1] and I can see it because I have my genetics. My disorder is triggered (important!) by my environment. Stress, high altitude, heat and humidity, high EMFs (yes, I said it) can all trigger episodes.

I am homeless (stress!), so if AI does not fix that, maybe they can invent a different kind of opioid? But that is juts a treatment, not a cure.

Will AI ever recommend diet changes or other environmental changes that have helped me? I need to live in a cold, low altitude environment where I eat a lot or reindeer, salmon, and seal and very few plant foods (Similar to an Inuit diet). These things helped me more than any other drug in my 60 year life.

[1] These genes come from both my Sami heritage and the ancient mitochondrial haplogroup (K1c2a around 7050 BCE). Some of the most impacting genes of my polygenic environmental disorder:

TAAR6, CACNA1C, KCNQ1, SCNA2, NCAN, DNM3, C15orf32, TPH2, CBS, FUT2, FADS1, FADS2, CPT2A, SLC1A1

Nadella and Huang, both currently getting some lessons from a re-familiarisation with reality, but who knows if these lessons will stick?

Trump had his inevitable midterm defeat, but I don't think any lessons were learned there either.

If you have to drive share prices, you better make outlandish claims

This appeal to authority bullshit on a person that has a financial incentive to sell you something needs to completely stop.

The same happened with doctors over 100 years ago who were peddling the benefits of Radon to cure diseases.

Diseases are horrible, I am against people living forever, because we are taking the world away from the next generation, but I also am against people dying in unpredictable, sad ways, I wish everyone could die at peace in their bed next to their loved ones (if they were good in life) rather than through diseases or conditions.

Some people legit think there is no need to go to school, work, learn or care about anything because Musk, Amodei, Altman &co promise them "universal high income" and eternal life in 10 years...

He's completely and utterly high on his own supply. That is what happens when you get told all day everyday how great your ideas are.

https://garymarcus.substack.com/p/math-is-hard-if-you-are-an...

I'm sure he'll be right this time around, just like, well, nevermind

And if Tao mentions LLM, does he really mean "the bit without tools" or is he conflating the two?

I know that Tao is saying that current AI is useful to advance research in math (and I fully agree with that), but 1) that is part of a whole framework, LLM is just part of it, and I think that it is not the part that "know how to do math", 2) even if you obtain the correct mathematical output, it does not mean that the LLM part itself can do math (the same way that someone who throws 10000 basket balls and misses 9999 of them should be considered as unable to play basketball even if you just have a process where you ignore the 9999 misses and only record the one that made it)

Tao and others don't sit around working out what 12,483,324,281/188,553.37463286*(243+2-983) is all day. There is a reason most people don't consider mathematics education to really start until you stop needing to bring a calculator to class.

Just next to this word, there is "Lean verification", which is not about arithmetic but about the logic.

But the problem is not even there. LLM are not able to do math not because they fail to do 12,483,324,281/188,553.37463286 * (243+2-983), but they are not able to do math because they don't _understand_ 12,483,324,281/188,553.37463286 * (243+2-983).

I cannot do 12,483,324,281/188,553.37463286 * (243+2-983) myself just like that, yet I can do advanced math. Because I understand this expression. I know what it means. When the LLM was not able to produce the correct number, it was not because it knew what this expression meant and was "distracted" and forgot to report a 1 when doing the mental calculation. It was because the LLM was seeing the expression and was trying to guess the answer based on the answers of other texts that had similar expressions. It did not _understand_ the expression itself: it had no idea what it meant, it was just guessing the answer by copying the output it has seen in the wild.

And this is why it was a good element to demonstrate that LLM cannot do math: if they don't understand the expression 12,483,324,281/188,553.37463286 * (243+2-983), then they don't understand the expression f(t)=\sum_{k=-\infty}^{\infty} c_k e^{j k\omega_0 t}, \qquad \omega_0=\frac{2\pi}{T}.

And that is why the AI framework to do proper math and help mathematicians needs harness with tools like Lean. The LLM part does not understand what the expression means, it needs a tool to discard, internally, all the LLM outputs that are just non-sense.

So what is understanding then, so we can check this hypothesis?

"It's the thing that AI can't do!"

What's the thing?

"Understanding!"

And what is understanding?

"It's the thing that AI can't do! I just said that!"

FYI, this isn't a serious argument without a clear testable definition of "understand" or "know".

First, the claim has nothing to do with "understanding".

The claim is "LLM cannot do math". LLM. Not LLM + tools and harness. Not LLM but we get plenty of answers, run a program to remove the non-sense one and pretend the LLM got only one answer.

And this is true: the LLM part cannot answer these math questions. It answers with a bunch of answers and has no idea which one is correct and which one is non-sense. It needs a specialised harness that will remove the non-sense ones.

But secondly, this incapacity to grasp something very simple is really annoying.

This is really not complicated. If you tell someone "what is 2+3" and that this person is going in the next room, write on their pocket calculator "2+3=1", click "ok", and the calculator says "Error", then "2+3=2", click "ok", and the calculator says "Error", then "2+3=3", click "ok", and the calculator says "Error", then "2+3=4", click "ok", and the calculator says "Error", then "2+3=5", click "ok", and the calculator says "Correct", and come back and say "2+3 is 5", it is clearly not the same as if you ask someone who understand math.

Call it "understanding math", or "able to do math", or "has learnt arithmetic", ... It does not matter. The reality is that this person, ON THEIR OWN, is unable to do math. They are not the one who do the math, the calculator who tells them if it works or not is the one doing the math.

Perhaps you picked up the belief that LLMs are just brute forcing these problems, but they aren't. They are creating hypotheses and checking them, which is the same thing humans do.

The solution spaces for problems is so massive that brute forcing it is beyond what even "every atom in the universe is a super computer running for 13B years could crack"

You need to understand the problem to know where to search for the solution.

But still, it does not really matter. A LLM, without harness or tool, cannot do math: it returns answers that are "reasonnable guesses based on historical data" without knowing which ones are non-sense or not, and it needs tools and harness that "do the math" to rejected the non-sense.

> You need to understand the problem to know where to search for the solution.

"What you mean by understanding"

you: "well, that thing that LLM does"

"what's the thing"

You: "Understanding!"

"And what is understanding?"

You: "It's the thing that AI do! I just said that!"

But, no, the "logic", the "structure", of math, is not inside the LLM. The LLM regularly returns outputs that are very good, because turns out that some relationships that the LLM extracted from historical data was also the relationships that happen to work here. And sometimes, the LLM returns outputs that are totally crazy, incompatible with "understanding the problem". Because the LLM is just doing "educated brute force" and applied a set of relationships that someone who is familiar with the problem would immediately consider inadequate, but the LLM still does it because it does not understand the problem, just tries to guess the direction of the answer based on historical direction of answers, without any idea if these directions are non-sense or not.

You're trying to argue that a Michelin Chef doesn't understand cooking because she only uses techniques and ingredients that others have used before - even though the dishes are totally new and other chefs are studying her techniques.

It is still possible that I'm incorrect, of course. But my understanding of how it works seems to fit very well the data.

"generalization of distilled knowledge" is a naive way out. People who have experience with ML have seen how difficult it is for the system to not get stuck in a local minimum or not latch on to a spurious pattern. Just for arithmetic, providing tons and tons of 6 digits multiplication tables, it is still extremely difficult for a ML model to extract the knowledge, there are a huge amount of fake patterns that leads it to incorrect heuristics. For example, the values of the first digit of the two numbers are strongly correlated to the value of the first digit of the answer, so the training will push into the direction for the model to learn to answer the computation based on this kind of heuristic. The "distilled knowledge" that the LLM will extract is not basic mathematical rules, it is "74% of the time, it's 4 and 26% of the time, it's 6", and it will fall into this local minimum and it will be difficult for it to get out of it.

But as I've said, I'm open to the possibility I'm wrong. If it's the case, maybe you can convince me otherwise. But it would help to bring concrete arguments more than "you sound like you don't have experience on this subject".

> You're trying to argue that a Michelin Chef doesn't understand cooking because she only uses techniques and ingredients that others have used before

But I don't say that LLM cannot do math because they use techniques and ingredients that others have used before. I'm saying that LLM cannot do math because the way they work. Fundamentally. The LLM extracts pattern by minimizing the loss function. There are hundreds of heuristic that will lead to a small loss function while still returning bad answers. Some patterns are "the good ones", but hundreds of them are misleading ones. And they can even get stuck in the wrong valley, with the way they learn preventing them to climb the ridge to join the "correct" valley. The fact that LLM that were very convincing to talk like they understand the human language and yet had a 2% success rate with basic arithmetic shows that LLMs often fail to extract the correct patterns. These LLMs did train on tons of numbers. To the point that they were able to do very convincing things. And yet, they did not discovered the "distilled knowledge" that were the arithmetic rules.

> “Suppose I wanted Noam Chomsky explained to me by two girls?” > “Oh, wow.” > […] > “You’d have to speak with Flossie,” she said. “It’d cost you.”

We may need Flossie to also explain Gary to us all.

History doesn’t always repeat, but it often rhymes.

On the other, we're still very much in the time of headlines which should say "in mice" but don't because the uncertainty is that level of bad.

Uncertainty will have to come down a long way before we can even do without animal testing, let alone do without human testing.

If a level of intelligence far beyond humans is achieved, it probably could eliminate most of the uncertainty about new treatments.

OK. If you already have perfect confidence in a treatment, what would a trial do for you?

You can’t just overcome that uncertainty with really good guesses based on what to do know. Not at least until you know a lot more.

Secondly RCTs exist because you want to capture a random portion of the population, some of who may respond differently to different treatments. That and the body is a dynamic organic system and not everyone’s is in the same state.

LLM will improve the pace at which new medicines are developed but it be a long time before reasoning entirely replace the need for empirics.

Ironically the Eli Lilly guy sounds exactly like Claude Opus 5

Guy sounds like he has some major skeletons in the closet. Seems like many rich people go through this odd manic phase before the breakdown too.

Just do not feed that vague reward signal in their AGI training run because combined with Amodei & bros being firmly in the war hawk paranoid anti-China camp they can make us and our diseases all relics of the past just to prevent offline local inference with chinese open weight models.

Doesn't everyone believe that, on some timeline, we will cure all of these things? Like if I said within a million years, you'd be hard-pressed to find a serious doubter.

Dario just thinks he's going to build ASI and ASI will solve them more-or-less instantly. I believe an ASI would do that, as well.

So by my logic, naysayers really just don't think we're about to build ASI. Dario does. I figure he has a better perspective. But we're not arguing about solving all of that stuff, we're arguing about whether ASI can do it, and whether we're on the cusp of ASI

Dinosaurs. I mean, the meme already exists: https://knowyourmeme.com/memes/but-i-dont-want-to-cure-cance...

Either that or a three-way split between kpop stars, tanks, and furries: kpop for conventional beauty standards, tanks for people whose thing is Ultimate Strength, and furries for those whose main interest is self-expression.

[1] https://en.wikipedia.org/wiki/AI-induced_psychosis

When I hear Amodei et al talk about biology, it's obvious to me that there's an intuition and understanding gap here. There's a famous study in synbio that led to standardization called, Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli

https://journals.plos.org/plosone/article?id=10.1371/journal...

88 labs measured "fluorescence from three engineered constitutive constructs in E. coli." You should eyeball the results yourself first,

https://journals.plos.org/plosone/article/figure/image?size=...

That's the result these machines spit out for the same set of samples being measured across 88 labs. This was termed a "remarkable degree of precision" (for biology) of 1.54x sd(!).

Teams couldn't converge on instrument-to-instrument variation within the SAME lab, https://journals.plos.org/plosone/article/figure/image?size=...

So the reading is different depending on when you measure it. For some reason, people in silicon valley think that biology is like classical logic and computing. You write some genetic code and you get expression out. It's not.

That's the pop-sci view of a real process. In reality, viruses and bacteria are extremely fickle, and hard to grow. A lot of the synthetic biology results aren't easily reproducible even if you know the protocol.

So you're telling me that you can fix that with "just" AI and "robots?"

Well... I would hate to break it to folks. But applying neural networks and ML to biology have been a thing since the dawn of AI. It was one of the very first applicaitons people thought about.

It used to be called computer-aided drug design, then they went all in on expert systems, then there was a time when quantitative structure–activity relationship AIs were a thing, then there was teh deep learning phase... D.E. Shaw actually did a ton of work on it, https://www.deshawresearch.com/technology.html he used his money to buy a supercomputer and used that to train neural networks...

There have been blockbuster drugs that have come out of this. Tirzepatide / mounjaro was discovered by algorithmically iterating through sister-compound families of the peptide they'd found in Gila monster venom until they found just the right one.

The issue that Anthropic and Open AI face here is that there's no low hanging fruit. There's no free lunch. It's very unlikely that Claude or GPT can do something that hasn't already been tried at some point in every AI/ML research era.

So they won't be able to "just" discover a new drug. If that was possible, then it would have already been iterated into by sheer accident based on the number of projects that have tried some variation of this with some sister architecture over the many decades preceding us.

Now where these machines do help is that they help people get started, and they also help working scientists. It's boring stuff, but think stuff like, "hey my machine isn't working, it's made by such and such. Can you write code to help me save my data?" or, "Can you proof read this grant?"

It will even accelerate the already tried approaches as it allows a smaller team to try more AI/ML algorithms than they would necessarily have had the expertise to execute before.

But will Claude magically solve medicine? No. I'm sorry, but it just doesn't work that way. And I'm not sure why they believe this, but I keep repeating this over and over again... A lot of this AI and biology stuff is nonsense. Standard undergrad and post-grad stuff whipped into fervor and fluff.

The thing that I noticed while I was doing my degree was that the CS people refused to talk to the bio people, and the bio people refused to talk to the CS people. The CS people would go on and make solutions to problems nobody asked, and the bio people would try to solve data problems in complicated ways that made no sense. Overlapping expertise is and was very rare between those two fields. AI however CAN be that bridge, because it can basically turn your average bioinformatician/biologist into a really competent programmer, without them having to give up time away from the work they are actually skilled at.

Maybe I'm just completely AI-psychotic, but I've always been a researcher at heart. I hated doing literature review because it was so frustrating falling through dead ends. These last few months with agents I have been able to go deeper and find better leads than ever before. AI is like having an insanely smart lab partner, available to you 24/7/365. Never gets tired, never has a pissy day, never gets too hungover to read papers. I do legitimately think it's going to accelerate the field so much faster than people expect. I don't know about the 10 year horizon, but I'd love to see it happen, and I'm cheering for them.

You've gone down a bad path if you're making the pharma companies seem like the good guys.

https://futurism.com/six-months-anthropic-coding

Broken clock doesn't apply here

The phrase "AI is writing 90 percent of code" doesn't mean what it did when he first invoked it. I'm producing 200-300% more code than I used to, but only like 10% more value to my customers. It's mostly internal tooling to monitor things that were left unmonitored before.

that’s funny, my customers are completely different than they are a year ago

people that would have never paid for coding because it was too expensive and risky are now getting their MVPs validated, earning and investing into more

> It wasn’t economically viable and would have taken too long so they didn’t

These two statements are contradictory. Your prices for the same service you used to provide are now lower than what they used to be. So it's now economically viable for new customers to buy your service.

time to market has dropped from 6 months to 2 weeks

validating an idea has dropped from 6 months to 1 month

For sure more code is AI generated and coding agents are useful. But not nearly to the ludicrous extent this man has been predicting.

Sure LLM is good to develop a feature but in 90% of cases in my career the problems I was working on weren't just coding and in that part, LLMs can't do everything.

It's also important to know that a typical dev spends 400-1000$ on API tokens that aren't still profitable for them.

Sure AI's useful and it will reduce the demand for software engs. to a point, but it won't eliminate them. At least I hope. Same with other fields... let's hope?

My previous company has a huge department where LLM coding is banned. The rationale is they work on really important industry secrets, as well as deal with customer's data. The risk is too high that something gets leaked.

The engineers bought the Kool-Aid.

The reality is: The company is bleeding money, and SW isn't their product (they're a HW company). They don't want to pay the bill.

In terms of code out there, in the world doing actual work, I'm skeptical that the amount of AI generated code is as high as 90%. In terms of model generating production code without human interaction (i.e. wholly replacing the software developer) I think we've fallen far short of his prediction.

While these people are obviously professionals in their field, anyone who touched pharma/bio IT with a ten foot pole understands how much there is to improve there. And that's just the beginning - imagine how much there is to improve in understanding of the body overall?

It's not about "AI will discover a cure for X in 10 years".

But it's about having the applied intellect of a an engineer, chemist, biologist, and all mixtures of that, available on demand, in a second, at scale, enabling you to do everything around it faster, better and cheaper.

Every step of drug discovery, synthesis and testing can be improved.

Over 90% drugs tested in humans fail. And maybe 0.1% of overall discovered ones. While yes, testing on humans is the bottleneck, we can fix problems around the bottleneck easily.

If you had a data processor and 90% of data that goes into it is corrupt, the processor isn't the bottleneck yet - the data quality is.

If we can create better biological models to test these drugs, better interaction models, improve pharmacokinetics, improve yield, we can go from 1000 discovered, 1 works, to 1000 discovered, 100 works. We can predict some candidates will fail due to better models and not test these. We can improve how they are delivered to the body and make them more effective. We can catch potential side-effects early on and avoid going to trials at all.

This is what AI can do even nowadays, let alone 5 or 10 years from now.

And that is a huge improvement. It revamps the industry.

tbe improvement you are suggesting isnt whats being sold - that ai will actually discover said drug

It seems to be thought that AI will help invent these novel treatments, and then Anthropic will profit from that.

When in reality, who will actually distribute and administer these treatments? Not Anthropic - or any of the big AI companies.

Self driving cars roll past my office every couple minutes, even though I was told AI would never be able to drive on public streets.

AI images and videos are indistinguishable from real videos, even though I was told they'd never be able to generate hands correctly.

But I'm sure the naysayers are right this time, AI defintely won't ever be effective at medical research.

I have to assume that they're probably doing the same with medicine.

However I do think there are opportunities to greatly improve all these fields with AI, I just don't think we should lead with an overpromise that sounds Musk-like.

https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93prin....

Pharmaceutical companies don't care about "ethical" research either, all they care about is making more money, faster.

may I ask why did you specifically pinpoint to Westerner?

What's special about westerners when it comes to law?

People and the law keeps treating this as "just a sales pitch", but it's really theft in disguise and should come with evidence on an S1 filing.

it's being "borrowed" in the sense that there's an implicit promise as part of the IPO sales pitch, but shareholders are last in line to collect in case of something as extreme as a bankruptcy - and that's if there is anything left if that was to happen, which by definition is not frequent.

Why are these people obligated to trust Amodei? Why are these people obligate to hand Anthropic money?

I really hate this "Investors should be protected from losses" narrative you see pop up now and again like that isn't the entire natural order on how we conduct these things.

This is true for every industry, group, non-profit etc…there are no organizations not fully captured by monetary transactions and hoarding capital.

I’ve run mutual aid cooperatives and whatever you try to do will be overrun by private equity and your market will be closed and the people who show up will be flighty and noncommittal

I currently am ED for a non-profit that I will not allow to be used for profit making - almost 100% of the people who ask to volunteer or show up do it for their own promotion or to jumpstart their product it’s maddening the number of times I have to say no.

I am the founder of the 150,000 person ultramarathon subreddit and I’ve only found two (2) moderators in the last 15 years that aren’t trying to promote something for money.

There are no organizations that escape this because at the root level you can’t find more than a literal handful of people who aren’t in it for the money - even if thats not their direct reason it’s usually a secondary reason

Even “monks” go out and write books and buy fancy cars

There doesn’t exist in my daily survey of social organizations ANY, and I mean there isn’t any single organization, that does what it says it does without trying to make a profit in some way.

… yeah

> Even “monks” go out and write books and buy fancy cars

You are a bitter, cynical edge lord that only sees the worst in the best people. Ironically, I’ll bet you think you’re the most rational observer you know.

How you are describing the world you see says a lot more about the way you see than it says about the world. Later.

Larry is great but he’s not even relevant to the discussion because you might as well be describing a DIY or hobby shop.

None of the people or organizations you describe are relevant to how the world works, they are temporary outliers with limited impact for a brief period

Even then, Larry didn’t work for free and wasn’t running it out of the goodness of his heart. He did it for money

The only people I know doing shit that isn’t for money are the people that actively work on the trails that I walk on in Northern Virginia and guess what, they work on them so they can have nice trails not because somebody else is going to use the trail.

But also I think simulations like a “body world model” of some sort will help narrow down what to test for the fda’s standard

And the fda itself has some newer faster approval pipelines

Anthropic is also clear about how they are partnering with a handful of life sciences firms to coexist with people that do know what they’re talking about

So I wouldn’t be too dismissive, the agentic form filing aspect will go a lot faster, that part is basically already done

Is he right? We can't know now, obviously. But is he being utterly irrational? Not by any reasonable understanding of science history.

AI now and increasingly in the future lowers the cost/value of human capital.

Therefore Pharmaceutical output will be driven by AI inasmuch as it’s currently driven by talent and human capital.

Caveat: current pharma companies that do not rely on human capital for growth may not be disrupted and will likely benefit.

The weird thing is that when you create something that is in some ways more intelligent than you are, it becomes very difficult to predict its outcomes. We face a wall of uncertainty and it is impossible to see beyond it.

I think that this depends on your definition of intelligence. The models definitely have better working memory than us, probably worse inhibition behaviour (just look at the thinking tokens), and absolutely no ability to convert their working memory into short or long term memory.

These tools are super, super weird, and I think that applying human definitions for intelligence doesn't make much sense.

(as an aside, this is also my issue with super-intelligence, as it's incredibly ill-defined).

Do you know anyone worse at predicting the future than Gary Marcus?

In order for "AI to solve X", there has to be a tremendous amount of resources put into not only AI solving X but testing and implementing the AI-generated solution(s). Who is going to put in those resources? The money flows towards what's profit-generating, not problem-solving.

So far, AI hasn't solved any problems that matter in terms of improving the human condition in any significant way. What has it accomplished so far? It has allowed companies to develop software faster so they can cut their workforce. It has solved some tricky math proofs. That's about it. Neither of these make a dent in improving humanity's lot.

"Yeah now it's just a chatbot, but once it gets better, it'll solve X." Yeah, sure.

_Theoretically_ AI could develop solutions to some gnarly technological problems -- nuclear fusion at scale would probably be at the top of my wish list -- but as we've seen the last 2 years in the US, the main reason we aren't relying more on cheaper/renewable energy sources is political, not technological.

Medicine is similar - the reason we still have Ebola outbreaks in the DRC isn't because we don't have a cure for Ebola. Same for the half a million children who die from malaria each year; it's not like we don't know how to stop malaria. (If they were American children, we would have have eradicated malaria by now, but they're African children ...)

World hunger: we know how to produce enough resources to feed and clothe the planet -- it's just very badly distributed, by design.

Etc.

I am super happy I decided to have zero stake in this game. Will just sip tea and wait.

If he’s right, I am going to personally send him a thank-you note for preventing my loved ones from dying from horrible illnesses and respect him as the most important person in history.

If not.

I will make a killing tracking him with drones and flock cameras around the country and selling location data, electric chainsaws and rubber sheets to all the people who will want to personally have a talk with him if none of his predictions materialise and people lose everything.

Either way I’ll be happy.

No more cancer or selling shovels and becoming rich in the process sounds like a pretty good future either way.

Authoritarian states often declare rejection of the state narrative as "mental illness".

Christian conservatives declare much, if not all, of queer culture as "mental illness".

On the other hand, some more consensus variants of mental illness, such as addiction or depression are actively caused by tech products and are fundamental to the value proposition. Tech companies invest enormous amount of work in how to make their products more addictive.

AI has already spawned its very own concept in the form of "AI psychosis".

That's not even starting with "virtual friends" or downstream effects like destruction of social skillsets and environments.

So, which if those things are included in his definition?

Many people working within AI, myself included, are (still) having a hard time predicting how quickly it moves, despite years of calibration. I met two of my friends, both PhD mathematicians, in June this year, and asked them for predictions on what AI might achieve in math over the next twelve months. The boldest prediction we ventured was that it might solve a frontier-level problem (but probably towards the end of the time window).

Although I have been working exclusively in NLP and NLU since 2012, the capabilities of ChatGPT in 2022 caught me by surprise, despite the fact that I had intimate knowledge of progress within Google Research until my departure in 2020.

For experts in other knowledge fields, their ability to predict AI is much worse. The tweets presented by Gary Marcus are from an industry still traumatized by Theranos, where progress has been slow and painstaking and is measured in decades. I bet if you told those same tweeters in 2015 that a group of upstart computer scientists believed protein folding could be solved in ten years, they would dismiss it as the fantasies of non-experts.

One difference between the two situations is that Theranos could never provide evidence of its technical progress, because there was none. In contrast, AI's progress is happening in the open and is available for anyone to verify.

Had Dario claimed that the cure to cancer in ten years would depend on "Anthropic's secret sauce", then the derision and dismissal would be well-earned. But read as a claim about what AI can accomplish, I find it realistic.

All these people could be the farriers and saddlers and horse breeders opining on what the car business could do before everybody had a car.

Yeah, in their current paradigms, these things are highly unlikely. But let's not pretend many people have any idea what the tech is capable of.

The old never pick a fight with someone who buys ink by the barrel should probably be updated for people who buy power for their AI data harvesting and influence campaign by the gigawatt.

Scientology has got nothing on the AI safety cult in terms of capacity for dirty tricks.