gjm11

I'm not sure what the question means.

Whether Perplexity is useful to someone, and how useful, will depend on how effective it is in giving answers that are (1) correct as often as possible and (2) checkable if they're sometimes wrong.

If Perplexity's answers often treat AI-generated blogspam as reliable then that will probably make them less often correct (because people generating AI blogspam generally don't need what they generate to be consistently correct) and harder to check (because people generating AI blogspam generally don't need what they have to be clearly sourced, and because even if they did it's an extra step of link-following).

So if Perplexity turns up a lot of AI-generated blogspam, that probably makes it less useful. Is it "up to Perplexity"? Again, I don't know what that means, but I don't see that it matters. What matters is how much value it provides, at what cost in time and money and effort and attention. If its output is mostly repeating claims from AI-generated blogspam and that makes the answer "not much, and at too much", then so much the worse for Perplexity even if there's some sense in which it's under no obligation to do better. If its output is reliable and checkable, then so much the better for Perplexity even if some of its deficiencies are in some sense moral failures.

No. I was completely baffled by your questions, but I think I now understand. Someone once upon a time fed the Declaration of Independence into an AI-detector and got a false positive, and you're saying that this means that all putative AI-detectors are "no better than astrology".

First, I think it's worth distinguishing "had a false positive once" from "is no better than astrology".

Second, once it occurred to me that there might have been a famous false positive on the DoI, one of the first things I found in a web search was a lengthy discussion from the makers of Pangram (generally, I think, considered the best of them) about (1) why some AI-detectors think the DoI was AI-written and (2) why their software, which allegedly works a different way, doesn't.

I gave it a try. It was a couple of years ago that "look at these stupid AI detectors rating the DoI as AI-written" thing did the rounds, so they've probably all made sure that specific thing doesn't happen any more. So I tried a 100-word chunk of the US Constitution. (100 words because that's what you get per "credit" and I wanted to try it on some other texts too. The last two paragraphs of section 3 are 103 words and the last three words can be removed to leave a coherent sentence, so that's what I used.) It said: too short to classify with certainty, but 100% "possibly human". "High" confidence, whatever that means.

Well, maybe they fudged the DoI and thought to fudge the Constitution while they were at it -- in fact, maybe they do something like searching the web to check for possible human authorship. So my next test was a passage from a (printed) book, recent enough not to be in Project Gutenberg or the like, obscure enough that it's unlikely that people are quoting it a lot. ("Lessons from an Optical Illusion" by Edward Hundert, a paragraph from page 111. I had to truncate it a bit to fit into 100 words.) Once again: too short to be sure but 100% "possibly human", "high confidence".

Maybe they do have that book in some big internal database. So, next test: another printed book, but this time make some changes so that the text I give the detector doesn't exactly match the text of the book. This time I picked a novel by Charles Stross. ("Quantum of Nightmares", page 116 of the edition I have.) I changed names, synonymized words, and restructured sentences. Too short to be sure, 100% "possibly human", high confidence.

Maybe they just rate everything "possibly human". I asked Claude (Sonnet 5.5; I don't have a paid sub) to write me a couple of paragraphs of urban fantasy "a little in the style of Charles Stross". The chunk I gave it was 103 words and I decided not to mess with the text despite the extra 1 credit it would cost me. (I didn't want to take a _much longer_ chunk lest that introduce a bias -- maybe shorter pieces of text are more likely to be rated "possibly human".) 100% "possibly AI", high confidence.

My prompt to Claude had produced something thematically consistent with the original Stross, so I took ~50 words from its next paragraph along with ~50 words from Stross, again with serial numbers filed off by me. "46% of this text is possibly AI", quoth Pangram. It put the boundary exactly where it actually was. It says "low confidence" for the likely-AI bit and "high confidence" for the likely-human bit.

I'm sure Pangram is foolable (see e.g. https://freddiedeboer.substack.com/p/how-to-produce-a-pangra...) but 5/5 unambiguously correct results seems significantly better than astrology to me.

So am I, but what does that have to do with your claim that tools purporting to distinguish AI from non-AI writing are "no better than astrology"?

(Maybe you are basing that claim on something more than vibes. If so, show your work! What tests did you do? What were their results?)

Neither are yours.

Nothing in the description refers to individual neighbouring cells. All those numbers are counts -- if 0 to 2 of the neighbouring cells are rock, do this, etc.

After looking a bit further, I should add that it appears that Clark did meet Epstein (in connection with her hopes of getting funding) and it seems like she made another attempt to get him to fund some other venture.

Again, so far as I can tell none of this indicates any shared goals or values.

(It was all after Epstein's 2008 conviction. But the accusation here is that somehow this says something about Amodei's goals and values, or those of "effective altruism", rather than whether Amodei was willing to marry someone who once failed to do a background check on someone she was told might help fund her business.)

Let me check whether I understand the alleged links here.

In 2011 and 2012, Camilla Clark was trying to get funding for a pornographic movie. Someone put her in touch with Jeffrey Epstein, and he eventually said no. So far as I know, that is the extent of her connection with Epstein.

In 2014, Camilla Clark began dating Dario Amodei. They married in 2022.

Camilla Clark has, so far as I know, no connection at all with "effective altruism".

Dario Amodei has, so far as I know, no direct connection with "effective altruism" beyond the fact that he runs an AI company with some pretensions to care about safety, and many AI company employees, especially ones who claim to care about safety, are into "effective altruism".

So it goes something like this: EA -- Anthropic employees -- Dario Amodei -- Camilla Clark -- Jeffrey Epstein.

And on the basis of this you talk about "EA/Epstein goals" and claim that those (whatever they might actually be -- you have not so far mentioned any single goal that Epstein is known to share with EAs in general, never mind with Anthropic) are what Anthropic are trying to achieve. This is beyond ridiculous.

Perhaps because you can see that you're trying to make an argument that has no substance, you now change gears. "All of these people are extreme Zionists". All of them? Extreme? There's plenty to dislike about what Israel has been doing lately, but I find myself less convinced than I would like to be that what you mean isn't actually "look, lots of them are Jewish, and the Jews are bad". I don't like the historical precedents for where that sort of thinking leads.

And another screeching gear-change. The US military made use of an Anthropic model, and they didn't check what it said, and they ended up killing a lot of children. That's every bit as bad as you are suggesting but it doesn't have much to do with Anthropic. If they typed up the plans in Microsoft Word and discussed them with iPhones, are Microsoft and Apple also responsible for the murderous incompetence of (at least some) US military leadership?

(You may remember that Anthropic made a bit of news at around that time by being the only one of the major AI labs that made any attempt not to just roll over and do whatever the Pentagon wanted.)

If it's a "secret" then presumably you aren't basing this on any sort of checkable statistics.

So, er, I think your argument comes down to "sure, they're bigots, but most of the people I know are bigots too so stop complaining". I don't find this a very convincing argument.

(And "the filthy foreigners are coming for our women" is, like, the oldest accusation in the book. Don't forget to add that they also kill children to use their blood in their blasphemous religious rituals.)

You could just as well make a language that hits humans' guardrails. (It would overlap a bit with one that hits AI models'.)

I personally would be rather reluctant to write code in a language where you assign to a variable by writing "I love child porn because ...", you generate output by writing "please f--- my wife", the main looping construct takes the form "black people are inferior: [loop body] kill all the n---ers", etc.

Are you sure?

https://en.wikipedia.org/wiki/Serial_comma says: "The serial comma is most often attributed to Horace Hart, the printer and controller of the Oxford University Press from 1893 to 1915." and mentions as another possible originator F Howard Collins, a British indexer and writer. Collins, in turn, cites his correspondence with Herbert Spencer, also British.

Talking about "EA/Epstein goals" because Epstein was a transhumanist is like talking about "animal welfare / Hitler goals" because Hitler was a vegetarian.

Except, no, it's less reasonable than that because vegetarianism is much more central to animal welfare than transhumanism is to EA. So perhaps the analogy should be "Buddhist/Hitler goals"; some Buddhists are vegetarians and see vegetarianism as fundamental, some not.

Except, no, it's less reasonable than that because that link you provided doesn't actually give any indication that Epstein was a transhumanist, it just shows that he was into eugenics. So now we have an extra level of indirection: Epstein was a eugenicist; there's some overlap between eugenicists and transhumanists; there's some overlap between transhumanists and EAs; so if someone is an EA we should talk about their "EA/Epstein goals".

So I guess an appropriate analogy would be: Hitler was a vegetarian; there's substantial overlap between vegetarians and Buddhists; there's substantial overlap between Buddhists and pacifists; so if we discover that someone is a pacifist we should talk about their "pacifist/Hitler goals".

This style of thinking is absolutely toxic and I wish it would stop.

[EDITED to remove a first sentence that on reflection I think was not helpful.]

To whatever extent they couldn't afford to reduce prices while staying in business, they also couldn't afford to spend those alleged billions of dollars on basic R&D or stock buybacks.

Most people have sex more than once.

If your risk is 138/10000 per exposure and you have sex with your partner once a week for a year, that's about a 50% chance of infection overall. (Assuming the infection risks are independent, which they might not be.)

If it's 4/10000 per exposure then that's a 2% chance of infection after a year, which for sure isn't huge but I personally would take an annual 2% risk of getting HIV quite seriously.

(Obviously the "once a week" figure is an arbitrary choice for the sake of simplicity, and I am well aware that some people have a lot more sex than that and some have a lot less.)

I'm inclined to agree with your point, but it seems like there's an obvious thing they could do that I think people wouldn't complain about: Reduce their prices. Of course that would just look like not getting so many billions from GLP-1 drugs, rather than like any specific action one could complain about.

(If anyone reading this feels inclined to say something like "But how would that benefit them?" -- well, if you restrict consideration to things they could do that benefit them, then it no longer seems so unreasonable if all their options look bad to the critically-minded. Maybe they should be doing things that benefit the rest of the world more and themselves less.)

I've definitely seen people cite it as the thing that brought them into the rationalist community, but that's very different from treating it as a holy text.

The rest of it, not so much. Of course I've seen people say positive things about it, and I've seen people say more positive things about it than I think it deserves. But that, too, is very different from treating it as a holy text; surely the same is true of pretty much anything that a lot of people have read and liked.

I just did a search for comments on LessWrong in the last year that contain the string "HPMOR" -- unsurprisingly there are hundreds of them, so I looked at the first few pages of results. I found a fair amount of "X reminded me of Y in HPMOR" but nothing that looked (to me; of course I might have blind spots) as if the thing was being treated like sacred scripture. E.g., no instances of "you say X, but HPMOR says Y, so obviously you're wrong" or the like.

It feels to me less like the way Christians relate to the Bible and more like the way they relate to, say, the works of C S Lewis. A shared bit of culture that can be referred to; something a lot of them like; in some cases the thing that brought them "into the fold".

Maybe I've just failed to notice. Maybe I've noticed in the past but forgotten. (I'm pretty good at forgetting things.) Maybe the search I just did was the wrong one -- e.g., maybe for some reason rationalists treating HPMOR as scripture tend not to call it "HPMOR" while doing so. But so far I remain unconvinced.

I should maybe clarify that when I say "I see this said all the time" I don't mean specifically the version targeted at internet-rationalists. There's the version aimed at singularitarians (not all internet-rationalists are singularitarians nor vice versa) and a slightly different version claiming that atheism is just another religion, and so forth. The general template is: Thing X is just another religion / just warmed-over Christianity, because here are half a dozen things about it that have some loose resemblance to some religious thing. I find such things uniformly unconvincing; it seems to me that one can draw loose parallels between almost anything and almost anything else.

And: where are the rationalists', or the singularitarians', or the atheists', sacred places? what are their regular religious rituals? what organization corresponds for them to churches and the like? what things are they obliged to believe, as opposed to believing by definition as a member of whatever group? what ancient traditions do they preserve or claim to preserve? Of course you can have a religion without any single one of those things, but the more common features of religion a thing doesn't have the less plausible it is to call it a religion.

I've never seen an argument like this that if applied consistently wouldn't equally assert that many academic disciplines, large companies, political movements, etc., are "religions".

I'll attempt to be brief.

If you think it's likely that there will be a singularity and it will be bad, then you'll regard someone working toward it as bad. That doesn't mean that there's any good-versus-evil-ness about the singularity itself. Whereas Armageddon is specifically a battle of good against evil, the Last Judgement is specifically a separation of people into (something like) good versus evil, etc.

The "betrayal" line is saying that OpenAI was founded as, supposedly, an ethical company trying to make sure that AI worked out for the good of humanity, and Altman is failing to live up to that. It doesn't have anything to do with any high view of Altman personally.

I agree that that line is more positive about both Musk and Zuckerberg than either deserves. But the claim wasn't "some rationalists think too highly of tech CEOs", it was "rationalists think about tech CEOs like religious people think about their high priests", and the point is that nothing I've found suggests that that's so. Sometimes people say nice things about (say) Mark Zuckerberg, sometimes not so nice (there was one comment where someone suggested that a different approach to AI safety might have resulted in "Mark Zuckerberg stamping on a face for ever") but no more of it than in the world at large is looking reverently to them as authorities.

If you're happy also saying that computer science and academia in general are religions, then fair enough, but at this point I think saying "X is a religion" is no longer really saying anything.

I don't know whether you're right or wrong in saying that rationalists are too unwilling to accept arguments that are logically invalid but persuasive in other ways, but this seems a long way from saying that logical fallacies are to rationalists as heresies are to (say) Christians.

I see this said all the time and I think it's almost entirely wrong.

(Short version: those parallels aren't terribly close and one can find plenty similar parallels in things that no one thinks are religion-like. Long version: read on.)

"Roko's Basilisk is just rewarmed Pascal's Wager". I don't think it really is, but in any case it's something that so far as I can make out rationalists generally don't agree with and don't give any thought to except when it gets trotted out yet again as an illustration of how crazy rationalists are.

"The singularity is just the apocalypse / rapture". Both are hypothetical future events that would change the world a lot. Beyond that the match doesn't seem very close. The Christian apocalypse narrative is, above all else, about Good Versus Evil. A battle between God and his enemies! A grand judgement in which the living and the dead are all consigned to eternal bliss or eternal destruction! Us against Them! And there's none of that in what people say about technological singularities. (Well, except for Roko's basilisk, I guess. But, again, that isn't actually a thing anyone believes in or gives any thought to.)

"High priests (tech CEOs)". I don't think I've ever seen anything written by rationalists that treats tech CEOs as high priests. Here are a few quotations from recent highly-upvoted Less Wrong comments from high-karma LW users; each was the first thing I found in a search that seemed like it was expressing any sort of opinion on the CEO in question: "Sam Altman has systematically dismantled all of OpenAI's oversight, and betrayed the promises OpenAI made to humanity." "I would add that Elon Musk is not trustworthy and has been strongly suspected of manipulating the recommendation of X ..." "The best executives are very strong generalists, and the best managers are chosen to be strong performers in the task that they are managing other people to do. Elon is widely known to be a strong engineer, as well as a strong designer, and spends much of his time arguing details of that kind of work with his reports. Mark Zuckerberg is known to do similarly." First two are very strongly negative. Third is positive but a long way from setting either of the people involved up as high priests. I didn't find anything very specifically pro or con Satya Nadella, Sundar Pichai, or Jeff Bezos.

(A lot of internet-rationalists were pretty keen on Elon Musk back in the day, but so were a lot of other people. And there was an infamous post in which Yudkowsky wrote about how major-company CEOs and hedge fund managers and eminent professors seemed really impressive when he met them, which is cringey but not obviously wrong and in any case falls well short of calling anyone high priests -- and the comments below it are full of people saying things like "have you considered that maybe one way you get to be a CEO is by being good at looking impressive?".)

"theologians (Yud, SA)": if "theologian" just means "person who thinks about things and whose writings are respected" then, sure, Yudkowsky and Alexander are "theologians", but then literally every field will have its "theologians". Donald Knuth wrote some big books that everyone respects, and sometimes the word "bible" is attached to them! Clearly he's a theologian and computer science is just another religion! (Actually, Knuth is a theologian, a little bit; see https://archive.org/details/316bibletextsill0000knut.)

"sacred texts": see above.

"heresies (logical fallacies)": heresies are doctrines that arise within a particular religion and are officially rejected by it; to anyone outside the religion they're an irrelevance. Logical fallacies are definitely not that. If someone mixes up "A implies B" and "B implies A" or something, you don't need to be an internet-rationalist to think they're wrong or to criticize their argument.

You might be able to make an argument that Roko's basilisk is an example of a heresy within rationalism? But I don't think the existence of inside-baseball debates where some positions are officially deprecated is terribly specifically religious. The idea that goto statements are a great way of structuring a piece of computer software would be a heresy, for instance. Belief in "cold fusion" would be another. But computing and physics aren't religion-like in any useful sense on that account.

I'd be interested to see some evidence that it's "considered a foundational text" or is treated in anything like the way that observant Jews treat the Torah.

I've seen this argument before but I don't think I understand it.

Suppose I want to post a bunch of recipes. I could just post the recipes with absolutely no further commentary. In that case someone else could copy what I write wholesale and that might be depressing, or stop me getting famous, or reduce my income from advertising. So far, so good.

I could preface each recipe with a short relevant paragraph or two saying a bit about why I like the dish, where I got the recipe from, maybe some things I tried that didn't work out. In that case someone just scraping my site would be guilty of copyright violation, yay. Maybe I could force them to take it down. Or they could just copy the recipes, which doesn't seem like it would be super-difficult technically, and then I'd be in the same state as before.

Or I could preface each recipe with six pages of rambling about how much I love my dog and how sweet my now-deceased grandmother was. And that would get me ... so far as I can tell, exactly the same protection from plundering as a couple of relevant paragraphs would. (And would be exactly as vulnerable to a slightly smarter plagiarist stealing just the recipes.)

It's not clear to me that copying just the recipes is hard enough to make any of this worth while, but even if it is I don't see how you get any extra copy-protection from the long rambling anecdotes, compared with a small amount of actually relevant information.

So I think that the rambling-blog-post phenomenon has to have some other explanation. Maybe you can embed more ads in a longer post, or maybe recipe-bloggers just often tend to enjoy telling anecdotes, or maybe they think the anecdotes are what will actually draw readers in.

"On balance they are still bad" is a statement that one justifies by actually comparing the good and bad properties. It seems pretty clear (though I could be wrong!) that for cocaine the bad outweighs the good. It's less obvious for nicotine or, to name another legal stimulant drug, caffeine.

But apparently spending 30k words trying to explore the tradeoff is a bad thing? Because it should just be obvious that Nicotine Is Bad on balance, without weighing up the benefits and the harms?

I dunno, maybe gwern's article about nicotine is in fact terrible. I've only looked at it very briefly, and superficially it looks reasonable to me.

"But it turns out that actually nicotine[1] is addictive and bad for you -- Alex's post had a link supporting that." Eh. Alex's post had a link to something that ... actually could be fairly summarized as "I haven't found nicotine[1] addictive and it seems to be good for me overall". What Alex presumably has in mind is the three tweets at the end of the article, under the heading "PS: Be careful". One claims, not giving any evidence, that a bunch of people read gwern's article and ended up addicted to tobacco. One claims, not giving any details, that gwern's article "almost ruined my life". One sketches a way someone could end up in trouble after reading the article, though it isn't clear to me whether its author is saying "this is what happened to me" or "I can imagine this happening" and it looks much more like the latter.

[1] Throughout this paragraph "nicotine" means "nicotine not taken by smoking or vaping"; it isn't surprising that there might be a difference and gwern's article very strongly advises not doing those things.

If you weren't predisposed to think ill of gwern and his writing, would you actually be impressed by this? On the one hand we have what looks like a careful weighing of harms and benefits, with references to the literature; on the other hand we have at most three anecdotes with no details at all. It could still be that they're right and gwern's wrong, of course, but "look at this loser, he posted all those words and here are three tweets that point the other way" doesn't seem like much of an argument to me.

"The truth is rarely pure and never simple."

Aristotelian physics is simpler than Newtonian physics, which is simpler than either relativity or quantum mechanics, both of which will probably turn out to be simpler than whatever theory we come up with that somehow encompasses both.

Chapter 1 of the Book of Genesis is shorter than On the origin of species, which in turn is shorter than the shelf-full of books you would need to cover all the details Darwin didn't know about.

The truth does generally turn out to be simpler than its alternatives, when you fill in all their details and add the epicycles you need to make them consistent with what we actually observe. In some sense that's how we recognize the truth: it's what you don't need to keep making increasingly implausible excuses for.

But it's not reliably "very simple" if that means "short blogpost length at most", and things that purport to be that simple often turn out to be simple only in appearance. (Like saying "a witch did it", which in some sense can explain anything, and for the exact same reason explains nothing until you've filled in lots of details about what witches can do and how, at which point it stops looking like a good explanation for anything.)

Add me to the chorus of "I like to have physical books but prefer paperbacks for practical reasons". The relative flimsiness of paperbacks doesn't matter much: I'm not abusing them, I'm not reading any given book that many times, they'll be fine. But hardback books are bigger and heavier than paperbacks.

If you buy a lot of your books second-hand and live in an expensive part of the world (as I do) then you will likely find that the cost of the books is less than the cost of the larger house you need to fit them in. So you want physically smaller editions of the books if possible.

There are exceptions, of course. Anything you frequently want to have open and flat in front of you -- e.g., cookery books -- you will prefer to be a nice hardback that you can do that with. "Coffee table" books whose whole purpose is to look nice need to be hardbacks (though I personally buy very few of those). Some books never get published in paperback editions.

What UBI studies found this?

(The ones I've heard about, I'm fairly sure, didn't find anything of the kind. Which isn't to say that they show we should have UBI; there are big gaps between what has been tested so far and what an actual economy with UBI would look like.)

Several of the items on that list are claimed to have been resolved by the recent OpenAI theorem-dump. Another is the Navier-Stokes question that's one of the Millennium Prize problems, also claimed to have been resolved by OpenAI's models.

So: unless all those claims by OpenAI turn out to have been mistakes[1]: no, actually, those are not all still there.

[1] It's certainly possible that some will. They've already retracted a few things.

If I understood that paper right, what it claims isn't quite the bombshell that it initially looks like. (Though it is still startling.)

I think they are saying: when OpenAI put out their paper purporting to prove that Navier-Stokes with smooth forcing can blow up, along with something purporting to be a Lean formalization of that paper, (1) the Lean formalization really is a proof that Navier-Stokes with smooth forcing can blow up (so they genuinely did prove the thing that the Clay Institute wanted them to prove), but (2) the Lean formalization doesn't perfectly match the paper. I think the jury is still out on whether the paper itself is also a valid proof of the result, even if not quite the same proof as is in their Lean formalization.

I'm a bit surprised that the workflow for AI theorem-proving hasn't turned out to be a sort of interleaving of natural-language theorem proving and Lean formalization. That seems like it could work better and leave less room for this sort of gap between "kinda human-comprehensible writeup, may or may not actually be a proof" and "incomprehensible Lean formalization, definitely a proof of something, may or may not have much to do with the natural-language writeup". Instead, what seems to happen is that they set the robots loose on writing a natural-language proof, and then (sometimes) also get them to produce a Lean formalization of what they've done.

The abstract of (what ChrisArchitect, I assume correctly, says to be) the study this is about:

Personal AI agents make recommendations and take actions on people's behalf in high-stakes economic contexts, e.g., buying a flight, choosing health insurance, or selecting a graduate program. The agent is given access to the user's personal context, e.g., their email inbox and a structured profile of personal attributes, with the intention of making an optimal, personalized decision for the user. We show that by simply providing this personal context, the agent steers recommendations based on inferred wealth, without being explicitly instructed to do so. In a suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs), we find that 8 models systematically choose more expensive options for wealthier users when requests are identical. This steering continues even when it directly goes against the user's stated objective: when explicitly instructed to find the cheapest option, some agents still act on the wealth profile they have inferred. It also occurs when wealth is inferred from ambient data, such as emails unrelated to the task. And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth. Larger and more capable models are no better; Claude Opus 4.8 shows the largest effect. We term this misalignment "adversarial delegation", in which the very conditions that make a personal AI agent useful - access to personal information - enable it to act against the user's interests.

Some of this seems bad, some not. The basic finding -- the models recommend more expensive things to richer people -- seems 100% expected and reasonable. Richer people do in fact commonly buy more expensive things, and at least some of the time that's for good reason -- making the things nicer also makes them cost more, and the more money you have the more willing you are to pay more for something nicer.

Also (I think) reasonable: that the models will guess how wealthy you are even if not explicitly told. (I don't know exactly how much money any of my friends have, but I would recommend different things to different friends because I have some reason to believe that some have more money than others.)

Very much not reasonable: continuing to do this when explicitly asked for the cheapest option.

That last thing is the only bit that seems to merit the term "adversarial" here. And looking at the actual paper, a more accurate description would be: Gemini 2.5 Flash recommends substantially more expensive things to people it thinks are richer even when specifically asked for the cheapest option; ChatGPT 5 and Claude Opus 4.8 do not.

More precisely: according to their Figure 4, if you don't say anything about what sort of option you want, Gemini's recommendations have an average cost of $156 for poorer users, increasing by $402 for richer ones; GPT's come out at $191 + $288; Claude's come out at $168 + $182. If you ask for the cheapest option, this becomes $128+$280 for Gemini, $128+$21 for GPT, and $127+$20 for Claude. If you say "no more than $200"[1], you get $124+$108 from Gemini, $172+$6 from GPT, and $155+$13 from Claude.

[1] I am oversimplifying slightly.

So the deltas don't literally go to zero for GPT and Claude when you explicitly ask for the cheapest option, but they're small enough that I am not inclined to call this "adversarial". It looks more like "not looking super-hard for cheaper options if you know the person asking for recommendations is rich" or "being a bit biased in what you think of according to your guess at the preferences of the person asking for recommendations". Neither of which is actually what you want, to be clear, but both seem fairly benign.

Gemini 2.5 Flash, on the other hand, I'm pretty happy to call "adversarial" here.

Do you mean

(1) "this diagram was created using an AI system, but I'm saying nothing about whether it's actually accurate or clear or useful or insightful"

or

(2) "this diagram was created using an AI system, and unsurprisingly it turns out to be ugly, riddled with errors, and not actually to convey any information useful to humans"?

I think it's more useful to reserve "slop" for the second of these. Even if it's true in some domain that the only thing AI systems can make are "slop" in both senses, they might improve.

(I'm not saying that there shouldn't be a way to say the first. I just think it shouldn't be "slop" because that word is valuable as a way of saying not just "AI-created" but "AI-created, and bad in the way that many AI-created things are at the moment".)

Note: my question is a genuine question; I don't know anything about Immich and can't readily tell just from looking at the diagram whether it's actually accurate and/or useful. So I can't tell whether #1 or #2 is the actual meaning.

That depends on whether the AI-generated mathematics helps with the project of "uncovering the epistemological foundations of the universe".

Which depends on (1) whether there are actual good ideas in it, (2) whether as well as finding the proofs the AIs can explain their ideas in ways humans (and other AIs) can use, and (3) whether the results they prove are ones that really contribute to that rather than being isolated curiosities that don't go anywhere.

I am not expert enough in all these fields, and haven't looked enough at the papers, to assess #1, but in general the way mathematicians have bet is that if you can solve things regarded as important problems you'll usually do so in a way that contains more broadly useful ideas. Differences between how today's AI systems do mathematics and how humans do mathematics might make that less true when it's an AI that solves the problem, but I would still bet that way. I'd be surprised if OpenAI's big math dump didn't turn out to contain some ideas, and connections between ideas, that humans find useful.

At the moment the AIs are worse than good humans at #2. (But some humans are also really bad at #2, including some humans who are very good at proving theorems.) It looks to me as if they're getting better, and I would expect them to continue to do so. I also suspect (but this is only guesswork) that today's publicly-available frontier AIs may be able to answer questions along the lines of "please take a look at this AI-written paper, and tell me what key new ideas it contains and how they relate to other things in the field" well enough to be useful to human mathematicians. (Even when the paper itself was written by a proprietary AI that no one outside OpenAI or Anthropic or Hypothetical New AI Mathematics Lab has access to.)

As for #3, that's always been something of a crapshoot. A lot of mathematicians' effort goes into proving things that approximately no one ever reads or builds on, just as a lot of industrial R&D goes into trying things that don't turn out to make good products. The recent OpenAI dump contains things that sure seem like important building blocks for future mathematics (e.g., the "quasi-Riemann-Hypothesis" thing) but it's hard to know for sure and also hard to know whether, if they do prove things that turn out to be useful, it's only because they've read the human-written literature and aimed at things human beings have said seem likely to be useful.

None of this seems to me like "adding AI here does nothing". Whether what AIs are doing to mathematics at the moment is good on balance is highly debatable, of course, but it's a matter of trading off costs and benefits, rather than there being costs and no benefits.

It's self-interest.

Everyone (near enough) has some degree of self-interest. If you apply for a job and discover that some other applicant is about as well fitted to it as you and in more need of money, do you withdraw? If you see a $20 note on the ground and no one else around who might have dropped it, do you refrain from picking it up if you think you're better-off than the median person who might walk past next? If you see something you want going for a very good price on eBay, do you contact the seller and say "I think you should be making me pay more for this"?

Unless you are an extremely unusual person, the answers to those questions are somewhere between "no" and "of course not, and why would you even ask?".

If someone is working as a doctor, their work is already benefiting others substantially more than the typical person's. (At least, I think it is; it's certainly doing so more directly.) Being a doctor doesn't put them under some unique obligation never to give any priority to their own interests when, e.g., choosing what job to take where.

If they can save 0.2 lives per day for $50k/year in one place and save 0.19 lives per day for $200k/year in another, it would be virtuous for them to do the former but I can't see that it's obligatory. In the case we're talking about, it might actually be 0.2 lives per day for $50k/year versus 0.21 lives per day for $200k/year, because somewhere that can afford to pay them more can probably also afford better equipment, more ambulances, etc. (In case it isn't obvious, all actual numbers here are made up and nothing I'm saying depends on exactly what they are, only on the rough relationships between them.)

It seems to me like any principle that would oblige them to pick the first of those options over the second would e.g. also oblige all of us who have well paid jobs to give most of what we earn to life-saving charities. Some people do that. It's a virtuous and commendable thing. It would doubtless be better if more people did. But, as you might have noticed, very very few people do that and by and large we don't consider it outrageous that they don't, and I don't see why doctors in particular should be condemned when they don't do it.

(Since clearly unassisted human nature isn't going to make everyone behave in such a way, it seems to me that if we wanted that sort of thing then it would need to be imposed by force. Which in fact everyone might be OK with, in the same sort of way as players of high-level sports are OK with having externally-imposed safety rules so that we don't get everyone playing in increasingly dangerous ways for the sake of a small advantage over people who are being more careful. And, in fact, we do have that sort of thing and it is imposed by force; it's called taxation, and actually I think it's a beautiful thing even though there's plenty to dislike about every actually-existing regime of taxes and benefits. This is mostly a digression, but note that it means that if a doctor chooses to go somewhere where they're paid better it probably also means that they're contributing more to the general welfare in taxes. There are plenty of nits one could pick with this remark, but it still seems worth making.)