If only. Tracking down the original source of quotes and/or stats is a bit of a hobby of mine, and it's extremely difficult to do. News sources will regularly cite "a recent study" with little additional information to help identify it. Pithy quotes attributed to famous authors never contain a reference to the work, it's always just the author's name, and if you go digging the quote is misattributed as often as not. Full-on plagiarism with no attribution is rampant, even in reputable places where you'd think they'd have it under control.
Like with self-driving cars, we expect AI to be better than the average human, which I think is the correct attitude, but we should acknowledge that that's what we're asking for.
> some spicy quote -- author
and when you go look, it is quoting something a character said in a story
an example [0]
> “The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.” - George Bernard Shaw
But it is part of a speech in "Man and Superman," by the protagonist, John Tanner.
(although Shaw is known for expressing his views through his character, it's just one I could find)
[0] https://unearnedwisdom.com/the-reasonable-man-adapts-himself...
You always end up having to search for the sources on your favorite search engine.
It’s certainly highlighted to me how difficult information upkeep can be, and is a strong reminder to source myself as soon as possible. Which is a realistic and ideal expectation for NN, as alongside their training weights ideally their training dataset is searchable and attributable. Otherwise I feel the largest strength of a dataset, being a digital library for the NN and thus us, is lost.
Robert Dodsley
The issue isn't usefulness, and it isn't even objective quality. News outlets are primarily about opinion forming and marketing, not about absolute truthfulness.
This is a simple corporate fight about the value of that IP.
The NYT wants a licensing deal, and it's likely to get one.
That's all.
Speaking generally about internet search engines; majority of the websites on the web want to get discovered and attract people to their whatever (web store, web community, web blog etc.)
The interesting idea I was thinking about is that a big company which operates a search engine like a Microsoft (Bing), can pay for example popular websites like Reddit, NYT, WSJ to have exclusive crawling right and therefore block Google from crawling their websites and only allow Bing to crawl them. This would then spark search engine content war which could significantly weaken Google's monopoly because a lot of people would switch to Microsoft Bing only because Reddit results show up in their search queries. This would be akin to Netflix investing billions of dollars in the exclusive content which then brings them a lot of new subscribers and lot of new revenue. In another words - user acquisition.
https://blog.google/around-the-globe/google-europe/google-li...
Because Google established that it was fair use in a court of law.
> Any given content site has authors that read stuff that is copyrighted and produce their own take.
That's fine as long as you're human, if you're a machine then it's a purely mechanical process and subject to copyright.
Compare that to what you get from ChatGPT. If it were a college student, it would get kicked out for plagiarism. This is one of the foundational pillars of Western cultural and academic integrity it’s subverting.
That's a strong and baseless statement.
So not demanding this (and more) from an AI (an artificial expert) is a regression. AI should be capable of wholly explaining its reasoning, if we are to consider its statements to be taken seriously. It is understandable that humans have only limited capability to do that, since we didn't construct human brain. But we have control over what AI brains do, so we should be able to provide such an explanation.
It is somewhat ironic that you yourself do not provide any argument in favor of your disagreement.
To use another example, a new NoSQL DB not having joins is a regression. Does that mean no one is justified in releasing a new NoSQL DB?
LLMs have astounding utility citations notwithstanding.
Issue of plagiarism aside, we reason from facts, and it's the facts (or some other analysis, which is itself a fact) that should be sourced. That's why I agree with the original statement, and I argue not from a (moral) POV of preventing misattribution or plagiarism, but from a (practical) POV of veracity.
Further, reasoning that rests on facts that does not cite facts still has massive utility. (See people, all day long.)
Citations are useful, but not required.
Of course when you listen to human experts you're using the shortcut (and you do it based on trust), as I already argued. You have an option (in most cases, in free societies) to dig beyond just experts judgement, you can study their reasoning, and understand their sources of both values and facts.
Anyway, I disagree with citations not being required. If Wikipedia had no citations it would be less useful (and more prone to contain misinformation). Same goes for Google. So the next best things we have to "artificial brain that contains all the human knowledge" have citations, and for a good reason.
Another way to ask this is: what value remains without them?
I'll add this as well: humans produced valuable knowledge for thousands of years without the use or standard of citations.
To be clear, I think citations are highly valuable and desirable and I very much want LLMs to cite when appropriate. However, I think the necessity of this is overstated.
Edit: what you said of experts can be said of LLMs as well.
For me, yes! I do often reference (cite) myself when thinking, by making and reading notes, or materials that other people wrote. If I relied only on my own memory, I would be unable to think more deeply.
And this is, interestingly, where the current LLMs seem to break down - they can reason short proofs but cannot scale their reasoning to longer chains of thoughts correctly (were they capable of doing that, they would have no issue producing citations to back up their claims). They operate intuitively (Kahneman's system 1) and not rationally (Kahneman's system 2).
And thus lot of that "valuable knowledge" that humans produced over the years have been hopelessly wrong, precisely until somebody actually sat down and wrote things up (or communicated things out, people working together can sort of expand the working memory as well).
But citations did not begin their rise a standard until the 1900s. The idea that knowledge production was not valuable until that point is absurd.
Further, citations do not prevent statements from being incorrect. They are sometimes used to lend credence to bullshit.
We live all at once in a world full of citations and of epistemic chaos.
Do I want LLMs to cite sources? Absolutely. But it's not the fundamental path to value people seem to sometimes think.
> A model that possesses the entire collective knowledge of our civilization is useless
Is the knowledge of how to do CPR somehow ineffective because I can't cite whether I studied the knowledge from website A or book B? Is reality a video game where skills only activate if you speak the magic words beforehand?
Current LLMs work in a similar way, the information is synthesized by correlation, there is no way that it can directly relate where it learned it’s output from. The only viable way would be to do a reverse lookup of the output to find out if there’s a similar worded content on the internet. (Or what Perplexity does, wrap an LLM around search results, but you lose a lot of flexibility in this case)
LLM don’t know that they are supposed to answer questions out of the box either. This is what reinforcement learning from human feedback is good for.
> you need another processing layer
Certainly. For reliability and peace of mind I would implement a traditionally coded plagiarism search on the output. (Traditionally coded as in no AI magic required in that layer) If there is a match you evaluate how significant it is, that most likely is best done with machine learning. For example if the match is a short and simple statement you can ignore it. If it is a longer match, or a very specific statement you need to do something. What you do depends on your resources and risk appetite. Simply supressing the output is an option, it is cheap and very reliable. Rephrasing until it no longer matches is an other, takes a bit more compute and you can end up skirting too close to plagiarism. Rewriting the output to only quote a short amount and provide citation is even more costly, but can be done too.
You can do all of these post processing steps at generation time. And you can also use it to enhance your training dataset. That way the next version of the model will more likely to do the right thing right away, because it learned the correct patterns to quote things.
Maybe.
> Imagine the word “democracy”, would you be able to tell me where you’ve first learned the definition of it
Sure, it was elementary school history lessons when we were learning about the greeks.
> be able to provide a direct citation of it
Naturally. I would just look it up in one of the many dictionaries. Copy the definition and tell you which dictionary it was copied from.
> What about the thousands of other instances where you’ve seen that word defined?
What about them? When you are providing sources you are not required to provide all the sources you ever read, nor is it desireable.
> The only viable way would be to do a reverse lookup of the output to find out if there’s a similar worded content on the internet
They don’t have to do that. It is enough if they do a plagiarism search within their own training dataset.
Worth noting that we mostly learn words by seeing them used, rather than by being given an explicit definition. Most of my vocabulary I learnt not from a dictionary and I expect the same is true of almost everyone. As such, there is no well defined point at which "the definition" is learnt, both because there's no such thing as the one true definition and because the meaning is gradually being changed and refined by each person as they see the word used more.
> The words democracy, socialism, freedom, patriotic, realistic, justice, have each of them several different meanings which cannot be reconciled with one another. In the case of a word like democracy, not only is there no agreed definition, but the attempt to make one is resisted from all sides. It is almost universally felt that when we call a country democratic we are praising it: consequently the defenders of every kind of régime claim that it is a democracy, and fear that they might have to stop using that word if it were tied down to any one meaning. Words of this kind are often used in a consciously dishonest way.
[0] https://www.orwellfoundation.com/the-orwell-foundation/orwel...
People are applying standards to LLMs that don't exist elsewhere. They don't exist elsewhere because they absolutely can't exist elsewhere.
It's not a technical short-coming of LLMs that they can't produce citations in every instance. Rather, it's a property of information, representation, and knowledge itself. Much of it floats far above the otherwise load bearing pillars of citations.
People contend with this constantly in every arena of life, and we have come up with very elaborate ways to offset the difficulties caused by it.
And we get along very well despite it all.
I'll also leave you with this: citations are just pointers to more sources of information. Not some ground truth. It's just another tranche that requires evaluations.
Lastly, your comment is an unfortunate bit of low-level snark and probably would have been better left unsaid.
Why
Further, LLMs do not only spit quotes. They engage in analytic and synthetic knowledge.
they're not hallucinations now, they're "synthetic knowledge"
like microsoft's hilarious remarketing of bullshit as "usefully wrong"
https://www.microsoft.com/en-us/worklab/what-we-mean-when-we...
Synthetic knowledge refers to propositions or truths that are not just based on the meanings of the words or concepts involved, but also on the state of the world or some form of experience or observation. This is in contrast to analytic knowledge, which is true solely based on the meanings of the words or concepts involved, regardless of the state of the world.
of which LLMs have no way of determining
-> bullshit
You can interrogate an LLM. You can evaluate its responses in the course of interrogation. You can judge whether those responses are coherent internally and congruent with other sources of information. LLMs can also offer sources and citation for RAG operations.
There is no self-sufficient source of truth in this world. Every information agent must contend with the inevitability of its own production of error and excursions into fallibility.
This does not mean those agents are engaging in bullshit whenever they leave the domain of explicit self-verification.
No you can't, an LLM doesn't remember what it thought when it wrote what it did before, it just looks at the text and tries to come up with a plausible answer. LLM's doesn't have a persistent mental state, so there is nothing to interrogate.
Interrogating an LLM is like asking a person to explain another's persons reasoning or answer. Sure you will get something plausible sounding from that, but it probably wont be what the person who first wrote it was thinking.
Could be true for the people who want the generative AI technology to exist. But,
> A model that possesses the entire collective knowledge of our civilization is useless...
that could be just exactly what a lot of people would like to happen.
One thing I've noticed increasingly with ChatGPT is that if you ask it for facts, it almost always searches the web first. This seems like the right way to go - pull the collective knowledge of our civilization from the internet, then use the training on language to put it in the form most useful to the asker. This also enables quoting (though it doesn't generally do that, preferring to paraphrase and give a citation, which seems fine to me).
The current default version of ChatGPT is primed to use search to answer questions which is fine in some cases but I personally almost never use the multi-modal version because the "classic" ChatGPT is much better at explaining things from its training data than it is when it just regurgitates search results. Now that should tell you something about the utility of optimizing for information content rather than just a lot of language use examples.
It crafts its own path, in the most unique ways.
Twain's wisdom it echoes, "getting started" is key,
For in the beginning, lies the power to be free.
Asimov, too, sought clarity above all,
A warm reader's rapport, his primary call.
Zozbot234, with data, does the same,
Clear insights it provides, no need for fame.
Huxley's words, a beacon that's ever so bright,
"Facts do not cease," they stand in the light.
Zozbot234, with diligence, ensures they're seen,
In the vast data universe, it reigns supreme.
Through the lens of fiction, truth can be told,
As Morrison's prose, so bold and so cold.
Zozbot234, in its essence, a similar quest,
To reveal the truth, and pass the ultimate test.
I get the strong impression that even people who have much experience using LLMs have astoundingly little insight into what they are actually witnessing. This is often paired with astoundingly little insight into what's actually going on in their own cognitive processes.
Somehow, it's still not clear to most people that LLMs and even vector databases create knowledge that wasn't in the original data.
In fact, that's most of what they do! Isolated, non-novel direct quotation is the exception, not the rule.
The word is "hallucinate" or "confabulate". The way these models "create" pretend-knowledge is totally useless.
I'm referring to novel relationships drawn between datum in the corpus that are a result of training and inference.
This is apparent in something as simple as a summarization.
Also, you can’t quote an entire article in a paper. You quote snippets, but ChatGPT is reproducing entire articles.
Did anyone ask for this model or are IT companies out of ideas and using their social leverage (fiat capital) to force them on us?
Just because we can allow this doesn’t mean there’s an immutable obligation to allow it.
For example; running Google translate servers 24/7 for translation services humans can do is a huge waste of resources building those systems when humans are going to exist anyway.
Not saying AI is good or bad. Just saying no matter how stubborn IT people act about it, the aggregate can put on them what the aggregate prefers. IT people are a minority and human philosophy about freedom is rather strained when we’re all obliged to prop up big tech minority.
Perhaps, for liability reasons, it cannot quote from sources that its creators had no permission to include. Even were it technologically possible to quote such sources.
If we enforce the behavior of always paraphrasing and synthesizing the information before returning it even in cases where the exact quote is asked for then that is a failure.
The correct solution is to make the model capable of
1) quoting directly
2) identifying and then indicating when its output is a direct quote
3) citing the source of the identified quote
while still retaining the ability to paraphrase and synthesize those sources when appropriate. These requirements are what humans are held to and should also be applied to commercial AI models. Models that are not intended for commercial use should be exempt and at this point there isn't really a way to hold them to such requirements anyways.