sseagull

While I see lots of discussion about AI solving these problems, I feel like there's a more sinister undercurrent that's is less discussed - about who did the solving. It wasn't academics empowered by AI, but OpenAI themselves.

I feel like it's a sign of what's to come - large AI companies with resources academia never had, at some point deciding that selling access to AI is not as important as just doing the research themselves with models inaccessible to anyone else. The end game is all scientific advancement will be done by one or two companies, who will reap all the profits. Why would they give access to a model that develops a cure for cancer when they could develop the cure themselves and get all the money?

(Kind of like Amazon coming out with their own products - use the greater community for ideas, then become a competitor at a scale they often cannot match).

I don't necessarily disagree, although there's a lot of "ifs" and "to our knowledge". The main thing I disagree with is identity.

If you want to claim that the evolution of the universe can be modeled using a Turing machine/finite state machine, that's probably not terribly far fetched, and I would somewhat agree. But it's a large jump to say "can be modeled by" is equivalent to "is one".

Various physical processes can be modeled by equations, but the rock falling down the mountain isn't an equation. A swinging pendulum isn't an equation. Code modeling a bridge is not a bridge. Ceci n'est pas une pipe.

I hold the view that various models and approximations are just that, and try not to confuse a successful model for what the underlying reality is.

And getting back to the question at hand, even if our brains can be modeled by a Turing machine, and LLMs behave/can be modeled like Turing computers, still does not mean our brains are equivalent to LLMs.

(Note that I'm learning a lot from these debates, even if I disagree with a lot of people. I've started down a more philosophical route and they do get me pondering)

> Literally the whole universe, including your brain that exists within it, is to our current knowledge a Turing computer

A Turing machine is an abstract mathematical model that is not, as far as I know, physically realizable in the finite universe. A human brain cannot "be" a Turing machine.

"Behaves like" or "can be modeled by"? Possibly, although still not proven. But it cannot "be" one.

If Turing computers are so powerful, then what do say about ethics and morality? Have the various ethical dilemmas been completely solved by computation now?

This is kind of the point of the thread - being mathematically and scientifically knowledgeable and rigorous can lead to different perceptions and definitions of "reality" (and maybe "truth"). But right now, we need ethics more than ever, and that's largely outside the realm of science.

The reality of a scientist is that everything is made of atoms, which are made of electrons & protons (even that's oversimplifying), there are various biological pathways to do whatever, planets orbit the sun, blah blah.

The reality of the person on the street is that they are hungry and would like a warm place to sleep, out of the rain.

> actual reality

Actual reality, as defined through a lens of science and math (see the sibling comment about filters).

Now, don't get me wrong. I'm a hard scientist (PhD in computational chemistry). Science is useful, necessary, and leads to amazing things. And there should be more dialog between hard scientists and philosophers, ethicists, etc.

But science has blind spots, and is itself built on unproven beliefs (for example, that "true reality", whatever the definition, can even be modeled with math). Scientists should remember that.

I’ve been thinking lately how lots of tech people dunk on humanities and liberal arts, only to come to forums like this and discuss ethics, reality, and the nature of consciousness (at quite a shallow level)

“Engineer’s disease” and all that.

> Display ads are meaningless to agents. And agentic commerce is a huge threat to Amazon’s profitability.

Would agents be susceptible to advertising? If humans have issues differentiating ads from real results, and are swayed by TV ads and stuff (they are), then why wouldn't an AI agent.

I kind of assume they would, given how ChatGPT brings up garbage websites sometimes for my questions.

It does say something about our culture, doesn't it? That something as mundane and non-violent as computer programming gets compared with violent combat.

I suppose it comes down to how hyper-competitive some people are, particularly the "greats", that working on software is in the same category as fisticuffs.

Everyone quotes Goodhart's law, but I think the McNamara fallacy is even more important nowadays. It should be read aloud to every CEO every day.

https://en.wikipedia.org/wiki/McNamara_fallacy

TLDR: Making a decision based on only qualitative data, and therefore ignoring qualitative information and observations, can lead to bad outcomes. Not everything that is important can be easily measured.

There’s a reason services like GitHub Classroom Replit’s education offerings shut down. Serving educators can be very hands-on. Education can have lots of different requirements (every teacher needs this one little feature), often legal issues (student privacy), and dealing with universities and governments is awful and bureaucratic. Most companies will find it easier to make money elsewhere.

I’m a computational chemist who has been working on this kind of stuff for a while, and I will say it’s just a weird field. The overall issue is that it’s still very academic and very very fragmented. People have tried to make businesses out of this stuff but it has rarely worked, and there isn’t enough money/funding to sustain the level of work needed to really advance the field in that way. Especially now that government funding is drying up and business are reluctant to spend on fundamental research like this.

My big question is where is that money going to go?

On the surface, I don't mind the program. However, the implementation is likely to be terrible as we have seen so far. The various agencies tasked with implementing this have already been gutted and are dysfunctional. There is widespread belief that the Genesis mission is just a grift to get government money away from academia and into already-wealthy businesses, and that the call for proposals was a way to mine for ideas that could be stolen.

I was one of the thousands of proposals submitted to the program, which we didn't hear anything about after submission. If the rumors are true, there will only be a pittance of awards this round. So yeah, I'm skeptical that this money will actually end up in the hands of actual scientists.

Congress does mandate things, but then never enforces it.

See https://www.science.org/content/article/exclusive-nsf-slashe...

(Long story short, the administration is slashing the budgets of various NSF divisions to make room X-Labs, despite the Congressional budget stating that is not allowed)

> but is anyone actually willing to do anything about it?

They did. They elected Trump.

> no major populist uprising against the dismantling of the institution of the presidency

Some people view the president as the only directly-elected person powerful enough to fight for them in dismantling the rest of a government that isn't working (or even actively working against them). So they implicitly want a strong president.

This only goes as far as the president agrees with them of course.

I remember having a computer and reading about how the CPU (486) could address up to 4 GB of memory. I thought that was maybe a mistake, since even my hard drive was maybe 100 MB, and I had 16 MB of memory IIRC.

I figured maybe they meant hard drive space (which I could kind of imagine being that large).

I don’t know how much it has ever been true, but it feels like today’s wealthy, especially in tech, have completely abandoned “noblesse oblige” - fulfilling social responsibilities that their wealth should bring them.

https://en.wikipedia.org/wiki/Noblesse_oblige

Archive Link: https://archive.is/mNP7v

The tension of feeling the actual need to use AI, but internalizing blame and guilt, is something I've noticed in my students and colleagues as well.

Obligatory XKCD

https://xkcd.com/1494/

“There is Mr. Geborand, buying a bit of paradise for one penny.”

(from Les Misérables)

I only use the Home Depot website, and it shows the location for me. Maybe it’s regional or something

Oh I’m sure we could commiserate about that. I think we work in vaguely the same area.

Unfortunately, getting money from industry isn’t much easier in my opinion.

We have some software projects we want to spin out into a small business or non-profit (because federal funding…), but industry is absolutely cold right now. Had a few very promising partners lined up, but it all evaporated last spring. Between tariffs, AI spending, and now oil, everyone is reluctant to spend.

The chaos is affecting pretty much all areas of science, not just the controversial ones. I work in non-controversial, pretty run-of-the-mill chemistry research and the attacks on the NSF have certainly impacted our funding situation. Very long delays in proposal review, complete pivoting to AI, etc. I have co-workers panicking over the green card changes. And the overall morale is pretty grim everywhere.

Edit: don’t forget how he’s forcing NSF headquarters to move. All the NSF, not just the “bad” research.

Almost everyone has entertained the idea of leaving the US for more stability, which is required for research.

The McNamara fallacy. One of my favorite fallacies vaguely related to Goodhart’s Law

https://en.wikipedia.org/wiki/McNamara_fallacy