at some point we'll solve all arbitrarily hard milestones for AIs and yet find ourselves 'nowhere near having real general intelligence'.
At that point we might start questioning our assumptions about intelligence.
Until we can understand and correct unconcious inferences, how can we ever understand intelligence.
In the days when Sussman was a novice Minsky once came to him as he sat
hacking at the PDP-6. "What are you doing?", asked Minsky. "I am
training a randomly wired neural net to play Tic-Tac-Toe." "Why is the
net wired randomly?", asked Minsky. "I do not want it to have any
preconceptions of how to play." Minsky shut his eyes. "Why do you
close your eyes?", Sussman asked his teacher. "So the room will be
empty." At that moment, Sussman was enlightened.Even the SOTA reinforcement learning folks know that they are working in limited (albeit progressively expanding) niches.
Then the company wizes up and the bonus is based on profit with a risk cap...and traders maximize profit and minimize some form of risk (while taking on some other unmeasured liability, e.g., liquidity)
If an AI accomplishs this task, I don't care whether it is general intelligence or not.
https://www.synopsys.com/automotive/autonomous-driving-level...
It's just that Google did not understand the Goodhart law of self driving, which states that you can hit arbitrarily hard milestones and yet get nowhere near real self driving.