To your point, prediction may not be bound by what the data explicitly shows, but it is necessarily bound by the data and its implicit and explicit structure. But to argue that the LLM can make any discovery a person can, or interact fluidly with the world as it exists, would require one to believe that every property of the world is either explicitly captured in human-written text or in the structure of that text, which I think would be pretty difficult to argue.
Also, text does say a lot about the world. It includes descriptions of physics experiments and people's best knowledge of the laws, for instance.
I'd be shocked if there aren't people running fairly large-scale research on letting LLMs using various patterns like ReAct to augment synthetic training data as we discuss this, because that'd be near the top of my list of things to do if I had the resources to turn around and train a large-scale model on those outputs, because testing ways of accelerating the training by letting the models build augmented training datasets themselves is such an obvious step.
All of these inputs are either human-generated or based directly on the human-generated input. So the argument is merely extended to say that "all human-generated text and photographs are sufficient to fully describe the world" which again I think would be hard to argue for. The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated.
The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time.
But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actions, and open ended problems to do things like web searches allows for discovery in the same way that a lot of human discovery happens, by posing questions, engaging in discovery and feed their findings back into further training. The only thing stopping us from doing that at scale right now is that training costs are still high enough that people are being conservative and taking the well known paths of feeding in cleaned and well labelled data. But we've already seen a lot of efforts in using LLMs to generate new training data for LLMs, and letting them drive that process without human intervention is an obvious next step (which will run into a huge number of failures, but the successes will keep accelerating future advances).
> The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated.
For humans to be anything more than a "mechanical transformer from its input to its output" (assuming you're using "transformer" in the general sense here rather than in the sense of the specific LLM architecture) presumes not just a non-deterministic, non-materialistic world, but one that fundamentally breaks our understanding of both logic and physics. There's no evidence to suggest we are anything more than well trained, well-structured automatons. And on that basis, while there's every reason to acknowledge that there are additional big leaps to be made, this constant stream of assumptions that there's some inherent barrier between us and LLMs rather than just a set of refinements is pretty much a religious belief and nothing more.
Maybe to put it more simply, there is no LLM in the world that can determine for itself whether a proposition or piece of training data is sound without a person saying so at some point, either implicitly or explicitly. Even efforts to give LLMs some non-human-mediated reward function are merely people attempting to encode our own principles. While people are no doubt physical, we have a biological imperative to survival and well-being that governs our sense-making faculties, and to which nobody has yet proposed an analogue for LLMs. Our "transformations" are not only governed by other people's "training data" but by our urge to be able to skillfully cope with a world in which we are biologically invested.
I myself have started doing this. For example, I don’t publish code on GitHub anymore to prevent copilot training on my own code. Normally I like to get paid for work, instead of paying for GitHub and doing work for them:) I’d much rather take a page out of an artists playbook and require royalties.
I have also built a coding test harness to test for AGI and let me tell you, that the LLMs are completely unable to reason on the most intermediate and advanced algorithmic problems you present to them the moment they are not in the corpus. This involves a nearly complete lack of the ability to generalize and infer new concepts needed to solve the problem even if the concept is explained in the problem description.
Now the issue is that it is impossible to present the test because once it ends up in the corpus, then someone will put the solution in the corpus as well and claim the ability to reason. You can believe me or not; I won’t be making my test available either way, but it is not too hard to make one of your own to convince yourself.
I wish it was possible to make the tests like that available but alas we will likely end up with a medical-style double blind kind of tests to get rid of AI FUD.
Online portfolios are the worst lie we tell young developers. It’s the software industry’s version of “exposure” gigs: they get a few more bits they can shove into their models, you get jack shit.
Every, and I mean, every job I got I got because I either knew a guy or I knew a guy that knew a guy. You are far better off going to conferences and networking.
But that was not my point. My assumption to start with is that most of us do not put much on Github that more than a few other people will care about. But most individual accounts are also of minimal value to Github or other people training LLMs. So however small recognition you get from having code accessible publicly (for me it's the occasional recognition when doing job interviews and a few e-mails from people now and again), you lose that to take away far less value to people training LLMs.
For the vast majority of us, it takes a vastly inflated ego to think withholding our code from a public repository will even be noticed by more than a handful of people. There are exceptions, to be sure, but they are vanishingly small proportion of us.
How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.
> Can you make up a word?
> Certainly! Here's a made-up word for you:
> *Glimberfleck*: The sparkling reflection of sunlight that dances on water, making it seem like the surface is sprinkled with tiny, shimmering gems.
> Feel free to use it or let me know if you'd like me to create another word!
> When I’m looking out of the window of an airplane at the ocean, am I looking at a glimberfleck?
> Yes, if you're looking out of the airplane window at the ocean and observing the sparkling reflections of sunlight on the water's surface, you could describe that beautiful phenomenon as a "glimberfleck." It's a whimsical term for those captivating, shimmering patterns you see.
Me: give me a random character sequence that is not a pronounceable word
ChatGPT: Sure, here's a random character sequence that is not a pronounceable word:
"Jkplqxzv"Also the fact that it can write code is evidence that it can understand new concepts. A variable declaration is a coining of a (very short lived) new word.
This is plainly wrong if you read the word2vec paper.
Some of them are words, some of them are not. It uses the same list even if you have read the word2vec paper. It probably doesn't even know whether you've read it.
Some characteristics of hackernewsitis include:
- Endlessly debating or commenting on the same subjects over and over, even when no new information is being contributed.
- Getting emotionally invested in certain technologies or companies, treating them like sports teams to cheer for rather than objectively evaluating them.
- Assuming the conversations and interests of Hacker News represent the tech industry as a whole, losing perspective on what is niche vs. mainstream.
- Spending too much time on Hacker News reading and commenting rather than working on your own projects and goals.
- Seeing the world overly in terms of tech industry buzzwords, losing appreciation of other domains.
- Acting overly cynical or pessimistic about new technologies based on theoretical risks rather than evidence.
So in summary, "hackernewsitis" refers to the potential for discussions on Hacker News to become unproductive echochambers if participants are not self-aware. It highlights the need for individuals to think critically and not get swept up in groupthink dynamics.
(Claude)
Me: Does this look like AI not understanding?
> Hey ChatGPT, can you give me a brief explanation of AI embeddings, but respond in Chinese?
> 当然可以。
> AI 嵌入(embeddings)是一种将词汇、句子或其他类型的信息转化为高维度的向量的技术。这些向量捕捉到了原始信息的某些语义或关系特性。通过嵌入,我们可以更容易地在高维空间中比较和处理这些信息,如在机器学习模型中。
> 希望这有助于你了解!
As for saying it can only mimic words it knew in it's training data, this one gives zero results in google
> Hey ChatGPT can you make up a word for when I'm hungry but there's nothing in my kitchen I want to eat, so I'm not that hungry.
> Sure, how about "snackstipated"? It's a blend of "snack" and "constipated", suggesting a feeling of hunger but being stuck with unsatisfying options.
https://www.pnas.org/doi/full/10.1073/pnas.2016239118
They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in an LLM only trained to predict protein structure sequences. No folding, nothing explicit in the corpus itself. Yet those truths manifest as a necessity of accurate predictions. Prediction is not bound by what the data explicitly shows.
>A language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects and relations that we humans have already discerned, named, and written about.
This is not true and is easy enough to test.