How do you think a PhD committee should referee a dissertation where all the ideas came from AI?

It's already the case. I know a professor and he says most of his students PhD ideas are AI generated, meaning they are outsourcing the fun stuff.

and that generation was educated without AI. going to be interesting how future generations of mathematicians will fare who have been using AI from day 1. I suspect they'll be simply much worse at math.

That generation will say "How on earth people could do this without AI", just like we have said "How on earth people could do this without the internet".

The answer is in both cases, 100x slower.

Without agents - sure, 100x slower. But without LLMs? Some jobs I do would be borderline impossible.

I gave an LLM recently a job of analysing past sessions to estimate how much time it would take a human to do the jobs requested.

Ingesting 600k word document and drawing conclusions/patterns from it is something I routinely give to Terra. It would take a human 33h non stop to read it. But finding patterns with good precision - personally I wouldn’t be able to.

> And for theoretical fields labs don’t need human data - synthetic one works well if not better.

How could that be true? I am a mathematical rube, but surely data that works "better" than real data should be extremely suspect?

Imagine you want to understand how well a certain strategy works for finding antiderivatives. You can get meaningful insight into that question by constructing synthetic examples - complicated functions which you intentionally designed to have or not have an antiderivative - and studying how well your strategy works when applied to those functions. This won't necessarily be worse and could easily be better than testing random functions you found which just so happen to be discussed in existing literature.

In areas where you can easily recognise valid data (math, programming, gaming, to some extent also visual and mechanical), the synthetic data can be way broader. Using manmade data as input speeds up early stages of the process but is not really necessary. For later stages you run out of manmade data, and it’s quite easy to generate new one.

With writing/art it’s different because there you need and always will need human taste to judge. To some extent it can be overcome - you train models to be judges on what output is good in taste, but it can only go so far.

In principle you could have a training system that just gets hello world as the only program and develop all the algorithms and structures that we have now and more. In art, if you gave it a single cave drawing, it would either stick to cave drawings or - more likely - develop its own art that would be good according to internal taste, but non enjoyable to humans.

I think there is a bug misunderstanding if this in the art community. Most people there seem to assume AI needs human data to be trained. To create something that people like yes, to create something that works in math/programming/science/engineering - not really.

> The best thing you can do if you’re a scientist, imho - wrapup whatever grant you have now asap with ai, use ai to get more grants if possible, and spend the rest od the year learning what new science you can do with the new tools and participate in creating the new paradigm in your field.

> That’s when you’re an established scientist. If you’re new in the field then this is the most exciting time you could wish for - an opportunity to make a name for yourself.

s/scientist/mathematician/g

There are plenty of fields of research where AI is not nearly as destructive as this, currently. Most experimental science is pretty safe for a while.

Fair point, and plenty of fields (like sociology) that will have great new capabilities now

You got this right, these are incredibly exciting times. Take your friend of a friend's as an example. AI's are trained with existing information, so obviously whatever he expected to discover was already out there. It took the machine 3.5 hours to find and sort out something that would take a human 10 years. Now your F of a F can dedicate the next 10 years to do what the machine cannot do: find meaning and insight.

> AI's are trained with existing information, so obviously whatever he expected to discover was already out there.

I've wondered about that myself. Does the fact that LLM's were able to solve these problems so easily indicate that most of the solution already existed, scattered in the training data, and the LLM was simply able to put together the pieces?

I think it's an interesting conjecture but nothing more without proof.

If I understand correctly, some of the papers in the recent OpenAI release had truly novel insights. Just a bunch but it proves the principle.