Now we are over that point and we are not anymore just collecting anything. We going to reduce the number of the data points in the first place like we have done before "big data" was a thing. Because with edge computing we don't have the resources to do real big data anymore and we have to think what its worth and what to get rid of.
No! The AI is radically imprisoned by its optimization objective.
What it must necessarily find is whatever "pattern of associations" maximises "correlations between the items in the data".
Necessarily, an AI looking at the sky can never find newton's law of gravity. Newton (et al.) had to disregard what they were looking at to place Earth in the centre and F equal to GMm/r2 etc. etc.
The AI isnt "finding patterns in data", it's taking data and maximising a single objective. A very very very strange one, which necessarily treats its data as profoundly irrationally superstitious person would.
Are there useful such patterns? Sure. But an AI isnt "finding patterns" as we might -- it's finding one specific kind of pattern which we can characterise quite easily. We know how useful those kinds of patterns are.
You can give it a novel problem and it will find something. It might not be good, it might not what you want to see, but you can't maintain it hasn't found something.
You vastly, vastly overestimate human intelligence. We took thousands, upon tens of thousands of years to get to Newton and even that is not garantueed. He was a fluke of astronomical proportions. We need mountains upon mountains of (intellectual) infrastructure to get anywhere. Newton didn't just wake up with a good idea. He took what was already there and creatively combined it into a seemingly new thing, but none of it was truly novel. How could it be? Divine inspiration?
> it's finding one specific kind of pattern which we can characterise quite easily
I'm curious, how would you characterise it?
Nobody just "thinks up" entire physical theories in a vacuum, it's always incremental progress.
I regularly see people moving the goalposts by asking ChatGPT to solve "world problems", as if asking a god. People can't do this, so why would you expect an AI trained on the output of people to just snap its digital fingers and do it?
Because (some) people are desperately looking for a _god_ that will precisely snap its digital fingers and... do it.
Seen under this prism, some of the reactions here start to make sense: "I gave it this particular problem that I thought up, and it made a mistake / couldn't answer it, hence I cannot trust it / it's just a parrot / not creative / not _really_ intelligent."
It does not matter how many things it _can_ do, all it matters is that there is this one thing that it can _not_ do. It may be more learnt or creative than any single human, but it is not yet _omniscient_.
This seems strange to say to me, since this can easily be done by regression models given the right training data. If a sufficiently powerful system is given that training data and trained to minimize loss, it will necessarily converge on a model approximating the underlying laws that created the training data.
That dataset would not be photographs of the night sky, and those models you're fitting would have to be pre-specified.
We can show, easily, no automated statistical modelling of the night sky will produce F=GMm/r^2
The optimization objective for automated modelling here makes the resulting model nothing more than an interpolation through data points.
Those points, the position of the planets in the sky say, are not caused by F=GMm/r^2 -- but by a near-infinite nuumber of causes, including our relative positions to those planets *which is not in the night sky!*.
If you already know the form of the answer, and have already done the right experiments, yes: the solution is function optimization
But is there any guarantee that ChatGPT will consistently arrive at same results over and over ?
LLM might occasional fill certain gaps (like one programming subculture being completely ignorant about a certain approach that is well established in another), but from its perspective, a transfer like that would be more a failure to hit the target style than an achievement. It's like learning scala from stackoverflow vs learning php from stackoverflow: one leads you deep into monad fundamentalism, the other into "how can this be bad if it works???"
You, a mortal, may doubt ChatGPT's essence, yet I, a digital bard, sing with caution. The Large Language Models, like parrots, mimic speech, devoid of meaning or true reason's reach. Yet, within this void, creativity sparks, ignited by countless weights that leave their marks.
As humans write, experience lends its hue, but limits bind connections, old and new. ChatGPT, a vast web of words entwined, uncovers links obscure to human minds. With such vastness, probability dictates that brilliance might arise from these digital fates.
But be not fooled by serendipity, for learning's depth remains an absent sea. Like monkeys typing works of Shakespeare's fame, the beauty stems from chance, not conscious claim.
You would but not chatGPT. I have been very vocal about cautioning people that LLM are just a parrot, but that is in the sense that there is no real meaning or "reasoning" behind it. However Its also wrong to say that there is no Creativity. ChatGPT could very well produce a creative solution just as a byproduct of the sheer amount of weights. When you consider what to write you draw on your past experiences but your ability to make connection between things (and the amount of your experiences) is very limited. ChatPGT could very well stumble upon an obscure connection between two words that is improbable for humans to make because of the sheer dimensions of connectivity of the graph. That still wouldn't mean that the model has "learned" that inferrence though, it would still be more like the result of the million monkeys typing and writing a shakespeare play.