If I was chatting with you in one window and ChatGPT in another, please tell me how you would prove that you are the ‘more intelligent’ one.

Intelligence is hard enough to define already, but consider that human intelligence can only be evaluated inasmuch as it allows its holder to successfully satisfy his or her own interests and purposes in the current environment and circumstances. That implies of course having interests and purposeful objectives.

Are you+ChatGPT more intelligent than I am by myself? Almost certainly. You have extended your abilities to fulfill your purposes, and can demonstrably do so better than I can.

However, is ChatGPT itself more intelligent than I am? Well, what are ChatGPT's purposes and objectives? What is its environment? What does ChatGPT want, so we can measure its ability to achieve it?

Three paragraphs desperately trying to rationalize the fact that you wouldn’t be able to prove that you are the more intelligent one in the scenario I presented.

Time to face the truth.

You seem to be under the misapprehension that I have any motivation to enter a dick-measuring competition with ChatGPT, let alone one that would not measure intelligence but instead would be biased towards the outcome you want to prove.

How do you understand intelligence? How do you judge it? And why should it matter to me?

That's just a repackaging of the famous "evolution cannot create complexity" argument against evolution. There are good rebuttals, one is intuition from Dan Dennett that we can build buildings that are taller than the highest crane.

> Well, what is it that we use to train AI? Human generated content.

Systems like alpha zero learn through self play.

For an AI to wreak havoc it doesn't have to be smart. It just has to be capable. Industrious but stupid is the theme of half of speculative fiction on intelligent machines. Grey goo and paperclips are examples.

Tangentially, I love that Murderbot thinks that killing all humans would be stupid, because who would make TV shows for it to watch if the humans are dead?

We are already making humans more intelligent each generations, surely we can make machine more intelligent than us especially since they are not limited the same way we are.

I agree that the feedback loop wouldn't make the models smarter than us, but training data would. I can see how with more computing power we will be able to feed it the whole of YouTube, Spotify and perhaps also millions of physics experiments.

With this data, perhaps a model will be able to understand the hidden patterns of the world (e.g. physics equations governing reality) better than us, don't you think?

Future AI doesn't have to be trained on text, it doesn't need to be a language model. It can be trained on video data to predict its sensory inputs. It would predict the future of reality, not the next text tokens. Gains from training on text are bound by the human intelligence which produced this text. But training on sensory data is only bound by reality itself.

This is demonstrably incorrect on every level.

Reinforcement learning from human feedback (RLHF) makes models dumber and less capable but more compliant.

Meanwhile, high quality synthetic training data (generated by AI) makes models smarter and more capable.

RLHF and human sourced training data is not the ceiling of training intelligent systems, it's the baseline-- the floor.

Chess, Go etc are counter examples.