Has there been research on neural nets who could cite their sources? In common cases, thousands or millions of items in the training set have contributed to the weights in the pathways that activate for a certain input. But maybe for some inputs, only a small number of items in the training set contributed strongly to the weights on the pathways the are decisive to the output? Could you design a data structure that would keep this information in the neural net?
I believe it's going to be in Bing soon?
I am going to guess that even though it uses the GPT language model, it works fundamentally differently than ChatGPT.
My guess is, Perplexity AI is a traditional search engine, that first finds a list of links that match your questions. And then it feeds text from those links to the GPT language model to write a summary of the results.
Whereas ChatGPT produces text based only on the trained deep neural net.
Maybe I am guessing wrong how Perplexity AI works. Maybe it works in a different manner.
I wonder if this is a special case for Explainable Models [1] — once you select an answer, another model helps describe and source the output.
[1] https://en.m.wikipedia.org/wiki/Explainable_artificial_intel...
But there's definitely uses for a tool that produces a lot of boilerplate content that a human can then proofread (ex: writing cover letters, filling out unstructured paperwork, etc.)
Edit: I personally underestimated the use-case, because the "more optimal" thing to do when auto-filling out paperwork would be to define a formal data-structure with annotations for each field, and then have a system read annotations and spit out autosuggestions. But this is a lot of overhead.
If you can have a system that takes in arbitrary text, parses it, and fills it out with pretty good guesses, then that can be much more viable than the previous approach, as it doesn't require an overhaul of, say, the way the US bureaucracy generates paperwork. It's all on the consumer-end, the producer doesn't need to cooperate.
I asked ChatGPT about it and it game me a fairly surface level answer. I then asked it for more details and it went into detail far beyond what the average person would be able to tell you. I thought that was quite impressive for a fairly obscure bit of mechanical engineering.
1. Would you be able to notice if it made something up, given that it's going beyond the detail that you know?
2. Did it have any details that aren't covered in: https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive
I've seen gpt hallucinate all kinds of things. But very rarely for something like explaining a process or system.
A lot of other stuff is just regurgitation of a wiki article.
Still it is very interesting and definitely a step forward.
I once worked with a very smart woman who at that time worked in marketing for the airside retail stores at a large international airport.
If anyone from outside her immediate sphere asked her what she did, she would typically say "I tell lies for a living".
This sounds like a lot of interviews I've conducted over the years.
Its an extremely well understood piece of technology and has been for decades outside of Toyota. Here is a 2005 paper that accurately models the fundamental behavior of the Prius HSD [1]. There are dozens of pages that explain it in various detail, including ones with live simulators of the power split device. [2] There are tons of forum threads of amateur Prius owners chatting about the internals.
I don't say this as a dunk, but to point out that a person's perception of chatGPT's capabilities are heavily influenced by their familiarity with the domain.
[1] https://www.engr.uvic.ca/~mech459/Pub_References/01500979.pd...
Q: What use is it attempting to measure ChatGPT's capabilities on something vs the average person on the street, or vs the average user with access to google.com? What does that actually tell us?
Maybe I'm showing my age, but if one wanted to learn or discover something, wouldn't one first typically try and find a reliable source/expert in that field or subject?
It would perhaps be interesting if it were such a hard topic that only 1 in 1000 could be capable of understanding it, but I would hazard a guess that most people familiar with the prius who don't understand its drivetrain probably don't understand it because they don't care, not because its too complicated to understand.
It's the same technique a stage psychic uses to demonstrate that they knew all about your late uncle and his peculiar love of ducks.
Sure it seems great at writing boilerplate code, unfortunately it’s really good and writing the kind of bugs that you don’t notice. This is maddening in a completely novel way, like working with an actively malicious autocomplete.
Not understanding the fallibility of LLM is not the fault of the technology. It’s the user’s fault.
Not at all like a con artist. More like a tool that can be dangerous in the hands of the uneducated. That is a very large set of human tools, from nuclear reaction to pharmaceuticals to the automobile. Perhaps we just need “drivers ed” for large language models.
This doesn’t make them useless in general, just a poor fit if you can’t easily fact check the results.
In google you would go to another search result.
For ChatGPT you actually need to verify via Google or actually try it when it is code.
There's already an example of it in the article where it incorrectly stated the tritone of F was Bb.
It’s a tool. It has flaws. It can be useful and if it is, have at it.
Similarly, if ChatGPT gives you an answer high in truthiness but low in accuracy, it could negatively impact you, whether loss in credibility if you repeat nonsense in front of someone knowledgable, or even worse if you use the incorrect knowledge to try to solve a real world problem.
It's like Babelfish, but for monoglots.
But likely much better read and with much better discretion that most of people writing content these days.
Which I think is part of the point that people are missing. ChatGPT’s output might not be as good as the best writing, but most of what people read is mediocre content with questionable accuracy. Being better than the majority of the writing being consumed is still a pretty big deal, even if it’s not better than all of the writing that’s being consumed. Especially since we can expect ChatGPT to get better.
The same is true for AI art. Could you hire a trained artist and work with them to do a better job? Probably, but most people don’t have the time and money to pursue such a project.
There are still trained artisans producing high quality physical goods that you can place an order with. But the masses have greatly benefited from the factories that have produced lower quality products that are much more widely available, and available at a moments notice. It’s not wrong to point out when the mass produced stuff might not have the same quality as the products produced. But if you only focus on this aspect you miss the reason why mass production is important.
I agree with this, but I think we may be missing the point.
Why are people reading mediocre content with questionable accuracy?
The problem is not: how to quickly and cheaply create content which is marginally less mediocre and marginally more accurate than what we currently have.
For most people that consume content, accurate information is not a priority. Being entertained, belonging to a group, or satisfying some immediate need or want matter more for most people, most of the time.
It's doubtful that people who consume content will change their wants to prize accuracy (most of the time). Partly, this is because accurate, precise, and/or qualified statements are often boring and hard to follow.
Consider also incentives that drive content creators: 1) the internet means many people can find a platform for many messages (democratization of speech is supposed to be a virtue of the internet) 2) the "freemium" advertising model means that most messages can be financially rewarded, regardless of accuracy
Engagement and financial rewards are much easier to get with lies, drama and controversy than with accuracy, precision, and qualified statements. In my experience, most people cannot find something worth striving for in the modern world, and settle on whatever comes easiest.
> The problem is not: how to quickly and cheaply create content which is marginally less mediocre and marginally more accurate than what we currently have
I'll grant that this is not a problem you're trying to solve. However, this sounds like exactly the arms race that content creators are engaged in until it stops paying.
> this sounds like exactly the arms race that content creators are engaged in until it stops paying
I don't think they're trying to achieve these things at all. Look at the broken incentives you raised in the first part of your response, that's what this is about, and all this is about.
Content creators mostly don't care about how mediocre their output is, and they mostly don't care about how accurate it is either, as long as the clicks keep coming.
We don't need (and can't expect) ChatGPT to "fix" this, the underlying problem is the broken incentives, not the actual quality - or lack of it - in the content.
> Why are people reading mediocre content with questionable accuracy?
How would you answer this?
I don't believe bad incentives explains it fully.
Some of it is out of necessity. I don’t think it’s controversial to say that many people use Stack Overflow, and though some of the answers there are very good, others aren’t. If someone needs help with a specific problem they’ll often post a question somewhere (Reddit, Stack Overflow, etc.), and the quality of the answer they get is a roll of the die - even whether they’ll get an answer at all.
Some of it is because blog spam has made traditional searches much more difficult. There’s another comment that says that there are comments better than Reddit and Hacker News, but if you go into a discussion about Google you’ll find that a lot of people have been doing things such as appending “reddit.com” to searches because otherwise they’ll drown in a mountain of blog spam.
Because it's literally everywhere you look and is very hard to filter due to huge amount of it everywhere. Particularly if you are trying to learn and grow on a certain subject, it's difficult to filter out what you don't know is 'mediocre' or 'questionable' at least not until you've already consumed it.
If people find it useful, I'm glad they do. That being said, the willful misinterpretation of ChatGPT's capabilities will get someone killed one day.
https://en.wikipedia.org/wiki/Use_theory_of_meaning
From wikipedia: The Investigations deals largely with the difficulties of language and meaning. Wittgenstein viewed the tools of language as being fundamentally simple and he believed that philosophers had obscured this simplicity by misusing language and by asking meaningless questions. He attempted in the Investigations to make things clear: "Der Fliege den Ausweg aus dem Fliegenglas zeigen"—to show the fly the way out of the fly bottle.
if you don't "understand something new", how would you know to trust what it says or not? the usefulness or not has to be based on something, not feelings.
As for how you know it's accurate, you don't really have to know it's accurate most of the time, because if you're using it to try to learn something new it's probably going to become apparent quickly enough. Nobody is reading something and then going "Welp, now I'm done learning."
As a software example, you can ask it about implementing the Fourier transform. If it says something wrong, you'll find out.
The question then becomes how often it is inaccurate and how badly. But in that regard, as long as you don't treat it as an all-knowing oracle, it's at least as useful as having a friend that knows a lot of stuff you want to learn.
people keep saying this, but it isn't true. if you ignore sites like this one and reddit there are plenty of authoritative articles and explanations about things with provenance. and if they are wrong they'll update their information.
Well, then we don't need to argue this since the problem will elegantly solve itself if that's true.
I disagree though. For a lot of things it feels like I can get much better answers than Google, especially when it comes to somewhat conceptual questions.
(Also, I don't use news aggregator comments to learn things unless it's the only possible source. But if you think every blog post or YouTube video that got things wrong has a detailed errata, you'd be sorely wrong. It's so uncommon on YouTube that the practice is usually commended when noticed.)
I disagree though. For a lot of things it feels like I can get much better answers than Google, especially when it comes to somewhat conceptual questions.
(Also, I don't use news aggregator comments to learn things unless it's the only possible source. But if you think every blog post or YouTube video that got things wrong has a detailed errata, you'd be sorely wrong. It's so uncommon on YouTube that the practice is usually commended when noticed.)
---
I never said ChatGPT is useless, lol. it's truly amazing how people can be so bad at reading comprehension yet praise AI bots in the same post.
But in all seriousness, I just simply didn't have much to go off of, and extrapolated based on context in the thread. You could always elaborate on what your opinions are.
For instance I asked it about space time metrics, and I learned about the varying types and in what circumstances each are used. Ask it about their mathematical properties and it gives a succinct explanation contrasting the different approaches. Certainly you can see the value now of conversational style pedagogy.
Think of it like learning about things from mates at the bar. In the thick of it there may be bits of inaccuracy, but you'll certainly know more about a topic than before. Then you can verify bits of things later in papers or books, etc.
It would be a huge improvement if it would be trained to provide references by default.
ChatGPT is a mixed bag. Sometimes it nails it, but I've also seen it hallucinate complete nonsense solutions, for instance in Davinci Fusion using a node type that doesn't even exist in the program.
As for "incredibly intelligent philosopher", I don't even find it an interesting "person" to talk to. Its responses are pedestrian, pedantic and boring. A question like "if God is good then why evil in the world?" is met with a summary of the wikipedia article on the nature of evil. I can read a Wikipedia article, thank you.
I'm a front-end dev. I use it every day. I have CoPilot too. These things help me do my job. No question. They also help me learn better than just about any other tool.
The big one for me is that they have made me more confident taking on bigger tasks. The impact is not trivial at all.
This is the issue. It's more like the dumbest guy in the room confidently convincing the everyone else that they know what they're talking about.
As a pattern matching engine it's extraordinarily good. Ask it about the implications of some obscure theory and it will usually correctly interpret your question and return some sentences summarising the current state of knowledge on that theory. Ask it to explain it in simpler terms and it will simplify the sentence structure, swap some words and usually retain meaning.
But then... ask it for some reading material on the same obscure theory and it'll use its knowledge of scientific papers on those theories and academic citation to return a well formatted list of papers with plausible titles and authors with the significant flaw that none of these papers exist. The dumbest guy in the room would struggle to make up plausible sounding scientific papers even if you gave them the reading list, but then the dumbest person in the room wouldn't be dumb enough to use the reading list to make up some nonexistent papers in the first place! (and ironically nor would the dumbest information retrieval algorithm in the room...)
No. You've fallen for exactly the con the article is describing.
ChatGPT knows nothing. It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information.
If you happened to get information from it that turned out to be ok, you were lucky. Don't expect your luck to last.
For creating bash scripts and english to code it remains remarkably consistent. No need to be a master of multiple languages syntaxes now.
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field.
I like to think about it like asking questions to a well read but dishonest student during an oral exam. They know some things, and try really hard to answer every question, even resort to making things up in hope of getting a passing grade.
For example, I now use GPT3 for every single code debugging problem that I have.
I input the bug/problamatic code into the service, ask it for an answer, and then I take the code and see if it fixes my problem I was having.
It doesn't work half the time, but thats fine. Then I just figure out the bug the normal way.
But the other half the time, it immediately solves my problem, and then I run my code and it works/ does what I want.
How do you know when to dig through those obscure manuals?
It seems to me that it'd be more useful when:
a) I don't care whether the text says things which are not true
Write me a poem about ...
b) I care, and will have to verify it's results, but that cost is worth it
Generate some test data ...
I think we're at a very dangerous intersection between an apparent decline in American's ability to detect nonsense and an automated way to create convincing nonsense.
At the same time if I'm looking for success I normally find it.
Essentially if you work cooperatively with the tool then you'll find it useful, if you are antagonistic towards it you can also have success in breaking it.
Couldn’t get it to work. Couldn’t get any examples out of google of it being sued that way. Eventually looked through the code of the library and found that while some functions worked int he way chatgpt was trying, the functions it had selected didn’t work, didn’t support those arguments, and never had.
Thank you for agreeing with my point.
If you need to check the answers with your FAA Pilot Handbook, wouldn't it be simpler just to read the FAA Pilot Handbook? That handbook, unlike the text generated by ChatGPT, was written by a process that was aware of the semantic relationship between the text being written and the things in the world that the text was referring to. That is what makes the FAA Pilot Handbook a reliable source of information about flying. ChatGPT is not any such process. That's why it is not a reliable source of information--as you agree, since you need to check what it says about flying with the FAA Pilot Handbook.
I work with all available material in many different ways (Anki cards, videos, different explanations of the same thing, etc), and ChatGPT is another way to learn and help me generate learning material. For example I have it ask me questions like a tutor would. Or I ask various questions when I'm unsure about the wider context - e.g. it provides much more about the underlying physics than the Pilot Handbook itself. If I don't understand, I can ask for clarification, or an explanation like I am 5.
Reading the Pilot Handbook is a big part of learning but being limited to it would be hard. I'm very happy about having Chatgpt available.
No, absolutely not. It's much easier to verify a couple specific pieces of information that you're unsure of then go hunting through a large corpus of information trying to pick out the bits that are specific to what you want.
I've used ChatGPT across lots of different areas and I find it incredibly useful. I'm not blindly trusting what it spits out, but it's pretty simple to verify what it's saying. While I definitely do have concerns about the impacts of ChatGPT on a societal level, and what will happen when so much computer-generated content can flood the Internet, but, at a personal level, the complaint that ChatGPT "bullshits with confidence" is not really that much of a problem for what I use it for.
Edit: To give a specific, real-world example, there was a post recently about using ChatGPT to replace a SQL Analyst. Now, ChatGPT definitely will and did churn out wrong answers, but it was incredibly useful to use as a starting point for some complex queries. When it fails, it tended to fail in pretty obvious ways, and to the complaint that it can fail in more subtle ways that look correct, I've certainly dealt with tons of human-generated queries that had the same issues. Are those all useless?
I routinely find ChatGPT giving me completely made-up APIs and incorrect explanations when it come to programming. And I haven't found it much better with math. Sorry, I don't buy it. Maybe it's good at training pilots, and if so great, but it's wrong enough for me that it's hard to trust in general.
That's correct: ChatGPT does not know anything. That's not what it's built to do.
> It does know the most common patterns of characters in the internet which implicitly contain knowledge.
No, it doesn't. It knows patterns of text in its training data, but it knows nothing about the semantics of the text--its relationship to other things in the world, which is what is involved with any text that contains knowledge. That is why ChatGPT is not reliable as a source of information.
I have been a Google Translate user for more than a decade. I use it a dozen or more times a day for multiple languages. It is a very useful tool.
I recently had to translate a passage that included this phrase: 六六大顺
Google Translate gave me: Liuliu Dashun (basically no translation, just a transliteration or romanization)
Bing Translator: Six Six Dashun (a little bit better because 六 does mean six in some contexts.
Baidu: Liu Liu (basically no translation)
Naver: Six or Six Dashun (partial translation, plus added an incorrect particle)
Only ChatGPT gave the correct answer: "Everything going smoothly"
There is a cultural context that is essential to the translation, which apparently only ChatGPT has. It turns out that the number 6 is a lucky number in Chinese. It has the meaning of being “auspicious”. In Chinese culture, there is an idiom called “六六大顺 (liù liù dà shùn)”, which means “Everything comes in a smooth way” or more naturally "Everything going smoothly".
BTW, after seeing the ChatGTP results, I found this contextual info using Google search, in a website http://www.ichineselearning.com
It is ironic that Google search "knows" (helps you find) the cultural context but Google Translate apparently does not leverage this info.
isn't that number 8? (not number 6)
GPT Chat: 10-4 is a phrase commonly used in radio communication that means "message received and understood." It is typically used to acknowledge a message or request.
Google does too https://en.wikipedia.org/wiki/Ten-code
里祝愿在新的2023年风调雨顺， 四季平安， 五福临门， 六六大顺， 十全十美， 万事如意， 好运连绵， 兔年吉祥
This passage has some nuance and wordplay, due to the stylistic device of starting each of the key phrases with a character that can represent a number: 四 (four), 五 (five), 六 (six), 十 (ten) and 万 (ten thousand). These characters also have meaning as words within each phrase. For example, 十 can convey perfection, in addition to the numerical quantity.
So here is Google's lumbering and broken translation attempt:
"Wishing you good luck in the new year of 2023, safe all year, Five Blessings, Liuliu Dashun, perfect, May all go well with you, good luck Auspicious Year of the Rabbit"
Baidu does a better job (including handling the problematic 六六大顺 phrase:
"I wish the new year 2023 a good year, Four seasons are safe, Five blessings are at the door, Everything goes smoothly, perfect, Everything goes well, Good luck, Auspicious Year of the Rabbit"
Neither of these is as good as the result from ChatGPT:
"I wish you a smooth wind and rain in the new year of 2023, peaceful seasons, five blessings at your doorstep, everything going smoothly, perfection in all things, everything going according to your wishes, continuous good luck, and a lucky Year of the Rabbit"
ChatGPT is not perfect. I think the phrase 四季平安 is better translated as "four seasons of peace" (this one came from the translator from Korean company Naver).
It is consistent with how awful our business management class is along with our politics and relationships.
The strength of ChatGPT over a GOFAI expert system like Cyc is how broad it's knowledge base is, but the weaknesses are that is doesn't know whether this "knowledge" (being self-deduced from the training set) is true or not, nor does it have any reliable way to combine facts/sources, since those rules are also self-derived and unverified. I'm sure some of this could be mitigated by more human curation (which seems to be the path OpenAI is going down), but then it seems it is just an expert system (maybe this is all people are looking for anyway).
This is just saying that sometimes ChatGPT is right and sometimes it's wrong, with an implicit stance on the relative frequencies.
In my own experience it would be more accurate to say you're unlucky if it gets it wrong—but then much of this hinges on having a good sense of what it's going to be good at / useful for.
(The other thing I see people miss all the time when assessing its correctness: it is somewhat "on demand"—it may include bits that are vague and only correct-ish, but if you point out an error or request refinement you can generally get it to clear/tighten things up. It's a bit like using a Google Maps-like zooming UI, but in addition to being variably abstract it's always approximating: the potential for error is ever present.)
No, it's saying that ChatGPT is not reliable. It's not reliable because getting things right or wrong is not part of its design at all. All it does is generate text that matches a pattern. There is nothing anywhere in it that filters or evaluates text based on any semantic connection with anything else.
That's another way of saying sometimes it is right and sometimes it is wrong...
> All it does is generate text that matches a pattern
This is about as meaningful as saying, "all things are is collections of atoms".
We are barely beginning to explore the implications of these pattern-extending structures: their most interesting features emerge from their structure, cannot be simply derived from the fact that they do pattern continuation in the same way you're not gonna derive the concept of friendship from analyzing chemical bond structure.
The hype in ChatGPT reminds me of the short-lived hype of Clubhouse; the end result is the same: The bigger players will come in to surpass it, just like they did with GPT-3.
For things that you yourself can verify easily, this isn't too much of a problem. It gives you some boost tips, you see whether the program works or not. But as you come to rely on it, and it matters whether the answers are correct in important ways, what validation/verification/QA steps can you apply to ensure that you're getting the correct answer?
I'm reminded of the FDIV bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug , where people found that their CPU gave mathematically incorrect answers for certain inputs.
I'm also thinking of a concept of "responsibility". If you ask a human for tips on, say, building a high power laser, or making explosives, they might decide whether they want to give you that information, and they might decide to give you some other information along with it ("wear goggles") for your own safety.
Who is liable if an AI gives someone advice, they act on that advice, and serious injury or property damage results? If the AI (or its operator) has no responsibility at all, is that disclaimer adequately communicated to users?
Before that, there was a lot of concern about blogs when they were the hot new thing. Here’s a New York Times article[1] from that period discussing the blogging controversy:
> There are two obvious differences between bloggers and the traditional press: unlike bloggers, professional journalists have a) editors and b) the need to maintain a professional reputation so that sources will continue to talk to them. I've been a journalist for more than a decade, and on two occasions I asked acquaintances whether I could print information that they had told me in social situations. Both times, they made clear that if I published they would never speak to me again. Without a reputation for trustworthiness, neither friendship nor journalism can be sustained over time.
The concerns weren’t incorrect - Wikipedia or Blogs are often unreliable. But the concerns overlooked how useful these things could be despite their problems.
As an aside, it's interesting to see comments here bring up the potential unreliability of ChatGPT - and then say that people should go read Wikipedia instead.
[1] https://www.nytimes.com/2004/12/19/magazine/your-blog-or-min...
> As an aside, it's interesting to see comments here bring up the potential unreliability of ChatGPT - and then say that people should go read Wikipedia instead.
On the other hand, Wikipedia's image changed largely due to the transparency behind its editorial processes- both the edit history and the discussions regarding editorial decisions are available in public. Are there pages where it's important to take the information presented with a grain of salt? Sure, but I have FAR more metadata available to help me discern how reliable the article may be.
And that's kinda the difference- having a way to figure out the source of the information. I was in high school when Wikipedia really took off, and although there were more than a few kids who got caught copying their essays from Wikipedia (and a few more who got chastised for the lesser sin of trying to cite it as a source), the vast majority of folks were perfectly capable of realizing that all you needed to do was scroll down to the list of citations and work from there.
With all that in mind, the contrast with ChatGPT should be pretty clear. You can ask it to explain a topic, and it'll probably do a decent job (or appear to do so) but the process by which it produced that output is extremely opaque. And (as has been noted in other threads) OpenAI hasn't been particularly forthcoming about how they influence its responses, which is pretty ironic considering their name.
That was still true early on when people were making fun of the idea. Like with many things, the public perception changed when it got popular and the people who had been laughing at it realized they also had a use for it.
I also can't agree that the production of Wikipedia articles isn't opaque. In fact, I would say that the perception of reliability of Wikipedia (as noted in many of these comments) makes it more likely to lead people astray .
Do Wikipedia articles often list sources? Sure. Do most people not bother checking the sources, and simply rely on a third-party to accurately summarize them? Also true. Do the few who actually check the sources often find it impossible, because the source is simply an academic book that most people don't have available? True as well. Even if it was available to them, the source is often (though not always) give as the entire book, so tracking down the source for a particular claim means you're searching for a needle in a haystack (and that's for just one claim).
I say this as someone who's actually interested in the source for the claims in Wikipedia articles, and have spent a lot of time trying to track them down. Often to no avail (and when I do manage to track them down, it's not uncommon to find the summary on Wikipedia misleading).
For instance, it takes me all of 30 seconds to write:
> Lincoln was actually in favor of subdividing Texas into four different states, and had the support of Congress to do so. It probably would have happened if he had lived. Source: Harris, William C. (2011). Lincoln and the Border States: Preserving the Union. Lawrence, Kansas: University Press of Kansas.
The source exists, but I have never read the book. Almost no one that reads this comment has easy access to the book, and even if they did I doubt anyone would go through it to fact check me (and by the time anyone did, this conversation would have been long over).
Does ChatGPT sometimes give inaccurate answers? Certainly. But at the moment more of its users seem to be more aware of the potential issues with the platform than the users of Wikipedia, AskHistorians, The New York Times, etc. are.
Determining truth is not in the wheelhouse of a language learning model and it is not defective for not doing so in the same way a tablesaw is not defective for not driving nails. Adjust your expectations accordingly.
The points of this thread are 1) this is a dramatic understatement; ChatGPT's output is not just occasionally incorrect, but usually incorrect in some dimension, and 2) in the absence of any fine-grained confidence value for individual statements, you must pessimistically assume that all statements are incorrect until proven otherwise, which dramatically reduces the utility of the tool compared to how its fanboys portray it as the second coming of Christ.
You and the author of this article are making this false dichotomy where there is no middle ground between "usually incorrect" (which is hyperbolic nonsense and trivially falsified by five minutes of using it), and "always correct" (which even your straw "fanboys" have not done to my knowledge), and then using this dichotomy to set up another one to pretend that the only way to act on information read from a computer screen is either uncritical acceptance or presume that it's bullshit.
Neither of these models are accurate and neither of them have any relation to how people in the real world generally interact with information online or with ChatGPT.
Furthermore, your insistence on "labeling falsehoods" is not something we can do accurately anyways, let alone in the context of a language model which has no concept of truth or falsehood in the first place! You are asking for something completely unreasonable and I can't tell if you're doing it out of ignorance or bad faith.
It's the user's responsibility to verify truth. This was trained on the internet and everyone knows not to believe everything you see on the internet. This isn't any different.
For a lot of people it's a faster way to google something you're familiar with, a more convenient stackoverflow.
When used right its hallucinations don't matter, and the user takes advantage of producing things quickly that would have been tedious to write manually. Like one-off shell scripts.
If your use case necessitates truth then I see why you don't think it's worth paying for.
However this is the only digital product I'm paying a subscription for. It's faster than googling and trudging between useless results, ads, and useless blog content, so it's worth every cent.
Write five jokes about a doctor and a programmer about the different meanings of words they use.
1. Doctor: I'm sending you a prescription.
Programmer: Great! I'll take a look at it and see if I can debug it.
2. Doctor: I need you to take two aspirin.
Programmer: Is that enough to solve the problem?
3. Doctor: I'm going to give you some instructions.
Programmer: Do I need to compile them or can I just run them?
4. Doctor: I need you to take a break from work.
Programmer: Does that mean I should pause the program or stop it completely?
5. Doctor: I need you to stay hydrated.
Programmer: Does that mean I should store more data or drink more water?
I don't think it understands those things with the ability to reason about them, but that demonstrates more than I would expect from a Chinese room.
Explain the joke "Doctor: I need you to stay hydrated. Programmer: Does that mean I should store more data or drink more water?"
The joke is a play on the terms "hydrate" and "store data". In programming, "storing data" is the process of saving information to a computer's memory, while "hydrating" means drinking water to stay hydrated. The joke is implying that the programmer is more comfortable with programming than with taking care of their own health, so they interpret the doctor's advice to stay hydrated as a suggestion to store more data.I don't know how to tell you this without being a little rude, but if you think ChatGPT is incredibly smart or that it can tell you meaningful things about "any field," well, there is a difference in intelligence somewhere in the equation between you and ChatGPT, but its not the one you think.
So with the software stuff you can pretty easily verify the output: either the suggestion fixes the problem or it doesn't. But how can you trust it for something like this where you can't distinguish what's good and bad output? It could be leading you down the garden path with this language exercise. And it's not in the helpful thinking tool/bouncing ideas around category, either: there are rules of grammar that you don't know, and you don't know whether ChatGPT knows them either.
Fundamentally, it's:
1) Ask ChatGPT a series of questions, going from the general down to the more specific. For example, 'Give me a list of options for setting up a headless Raspberry Pi as a Nginx server from scratch using only command line tools'. ChatGPT will produce a decent list of options.
2) Run that list of options in truncated form, just the keywords, through Google or other search engine that allows for a 'verbatim' search. Set to 50 or more results per page to get a decent spread of results at a glance. Get the more specific information for various options.
3) Run that more narrowed information back through ChatGPT to see what it produces. At this point you might be pretty close to the correct specific information you need, but generally I run it through search again just to be sure.
For example, this way I was able to quickly discover my headless RPi setup with the latest Raspberry Pi OS wasn't allowing me to ssh into it over wireless because the RPi engineers did away with default user/password settings for security reasons, so I had to manually include the 'userconf.txt' file in the boot directory on the SD card. Also it was very helpful with what needed to go into the wpa_supplicant.conf file.
Trying to find stuff like that out by search alone, in the past, would have probably taken at least 10X as long.
I wouldn't ever blindly trust what it suggests, however.
Now, compare that to your comment, and it’s INSANE how much the capability (and the conversation) has shifted.
GPT3 was what, three or four years ago? What will we be debating, what will the capabilities be, in another three or four years?
For example can give examples of some programming language, or an API, and natural language explaining what you want to do with it (or code in another language), and it will translate this with remarkable accuracy, 0 shot. We've never had anything that's even close to this capability.
ChatGPT has no more interest in running a confidence game than it does in helping children learn critical reasoning. Which is to say none, zero, zilch in both cases. It has as much volition and desire as a calculator. It’s just a super impressive and useful calculator.
Does this person stubbornly insist on being right, even if they are wrong?
Likely, that's just how humans usually behave - and ChatGPT does the same.
At least that was my thinking when people complain about ChatGPT being a know-it-all bluffer.
Actually ChatGPT will almost always change tune if you tell it that it is wrong. That's because it is a stochastic parrot and that's an "unlikely" event if you actually got the answer right. But we'll see if this gets updated/broken too.
As others have pointed out — what did we expect of 'AI' created in our image? Aren't we all just bullshit artists of varying degree?
HN is the perfect place to learn how much BS you are - I DARE any subject matter expdrts on here to post on any topic and I guarantee another expert will step in and tell you why you're wrong, or at best, nitpick you apart.
I have yet to see a single instance of this with any query that requires even a modicum of expertise. I have asked it a lot of questions about my field. I have asked it to summarize and paraphrase and ELI5 all kinds of shit. Everything it produces is either inane or a regurgitation of the input just with verbs and prepositions changed.
I would like to see a nice example, input and output, of this. I really would.
> Explain how git works in very simple english
Git is a version control system for software development that helps track changes made to the source code over time. It allows multiple developers to collaborate and make changes to the same codebase without interfering with each other.
When you start a project in Git, you create a "repository" that stores all the files and their changes. You can then "commit" changes to the repository, which is like saving a snapshot of the code at a certain point in time. You can also "push" the changes to a remote repository on a server, so that other people can access and work on the code.
If someone else makes changes to the code, you can "pull" the updated version of the repository to your local machine and "merge" the changes with your own code. Git also allows you to create "branches" of the code, so that you can work on different features or fixes without affecting the main code.
By using Git, you can easily revert back to previous versions of the code, compare changes between different versions, and track who made which changes
It is a copy/paste machine that strings together words to make sentences that look like sentences it’s read.
So it’s like an incredibly smart engineer/scientist/philosopher that does not and fundamentally cannot understand any of those concepts (or any concepts at all.)
A parrot that can recite Macbeth’s monologue is not a s Shakespeare scholar.
I'm not sure whether this constitutes "understanding", since that term usually applies to people's cognitive states; however, if a student provided this answer on an exam, they would certainly convince the examiner and pass the test!
Making citations up is not a "mistake", it's a lie. I have absolutely no use for a tool which lies to me, especially not when troubleshooting computer systems.
No it can not explain anything. It's a (stochastic) parrot, nothing more.
> Yep, but it is such an incredibly useful tool
It is not. You are fooling yourself. You are building a castle on a swamp.
It's just a tool. You don't use a shovel to rake leaves, but that doesn't make a shovel any less useful.
Wrote me a script to quickly convert SRT to VTT files as I edit and prep for uploading online. I haven't done anything akin to "coding" since I took a very basic class in college and messed around with HTML/CSS. Shaves several steps off of every video I do (I'm in a high volume world here).
Instead of army of mediocre journalists filling web portals, you just need 1 person to proof read it per business, maybe not even that. I can see the same for software dev in less than decade with current pace, 1 senior going over generated code and doing minor tweaks. Which is probably road to hell long term since how will juniors become seniors, but cost savings can be real and massive, drop in quality small if at all and it may be good enough for many.
It has very good language skills, in my obscure and rather difficult native language which only cca 6 million people use it formulated perfect sentences consistently. I can easily imagine faking social, even love chats easily. This can help lonely or elderly people (with good speech recognition/generation, since language skill itself is there already). With image/video generation porn industry will be steamrolled, 24/7 streams of anything you can imagine, looking realistic enough, tailored just to you.
Or just help with learning foreign languages, heck few months ago a guy was posting here on HN just a smart chat that could be used for training and got standing ovation also from me for that, this has it and everything else.
I understand google's reaction, instead of browsing through paid entries it gives me answer directly, no need to click through search and then wikipedia and then parsing texts. At least in some cases. This could help google massively if they could somehow do it on their scale.
People trying to find errors in it are missing the point now - its not about academic perfection, but often good-enough almost for free. Also, this is not final product, only 4th alpha/beta/whatever, previous ones were much worse. Imagine 10 or 20 years down the road...
People bashing on chatgpt just dont know how to use it. is there going to be an example in the world where it just plagiarizes bob dillan? sure ... so what? is there going to be instances where its not useful? duh.
but if you dont like it, that's fine. not every tool is for everyone. I find it very useful for certain things, and I imagine its only going to improve from here.
LOL where do you get this notion? I haven't seen one chatGPT bot useful for law. They all have no idea what they are doing and are basically dangerous if you don't know better than the LLM. There is literally nothing chatGPT can do for me as an attorney that isn't already solved by using a template.
The tech is there now to be utilized as an assistant. However as the end user you better have enough knowledge to understand the solution or answer it spits at you to fact check it.
The biggest danger I see is utilizing the tool for solutions or answers by novices that don't understand enough to grasp the solution and confirm if in fact the answer it gives is correct or appropriate.
Simplistic example: A non techie manager trying to have it create sql queries to get the information they are asking for , is asking for trouble. However in the hands of a sql developer it could provide enough of the query build out to just require minimal tweaks.
Well the OpenAI hype squad is racking up personal fortunes from their work in a finding deal that is pretty clearly over-valued. Especially versus what Deepmind sold for [1].
While perhaps both sides are not being honest, YC’s own sama is clearly profiting excessively over the hubub. He’s clearly proving to be a symbol of the faction of YC that rakes in money through arbitrage not prolonged value.
AI is exciting, and it can be an useful tool. There are a lot of use cases for it, and in the end I think it will be a good thing. However, what's worse than the detractors are the people blindly hoping that AI will automate every single thing a human touches. It shouldn't be used in that way. It has to be a tool to assist humans, not take away our professions and reasons to get out of bed in the morning.
Experts in each of these areas were consistently better at giving a correct explanation.
ChatGPT is great when occasionally being wrong doesn't matter. It's still pretty good when you can't be wrong, but you're an expert. But in any "no mistakes, I need to talk to an expert" situation, ChatGPT is setting you up for failure.
That's the key part - the information is unreliable/untrustworthy. As long as that is not a problem, it's awesome.
The problem is that it presents wrong data with absolute confidence in a very convincing voice. (Hence people calling it a con artist). It'll become a huge problem as it makes inroads in mis/disinfo campaigns.
(The usual tech problem. Don't let large scale interests touch it, and you'll be OK)
The parent cannot tell what is worse, (a) or (b).
Given that there is no (b) without (a) the question seems like a no-brainer.
We are being inundated with endless web user-generated "content" about ChatGPT^1, the number of HN submissions about ChatGPT is ridiculous, but I have yet to see a single submission that suggests ChatGPT is "not useful". Instead it appears to be a lame reponse to any discussion of ChatGPT's "limitations" to assert, "Yeah but it's really useful." To me, that defensiveness is a red flag. This is like "Watson", Round 2. With the mindless appeal of ELIZA.
1. We'll soon be inundated with machine-generated "content" sourced from previous web "content".^2 NB. ChatGPT relies on GPT-3 and thus only uses web "content" up to 2020. Ask ChatGPT about some current events.
2. But OpenAI is apparently prohibited from using Twitter "content", even though Musk co-founded OpenAI. There is a backstory.
They're both missing the point.
Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful.
It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? Can't cite their sources? Yeah this definitely happens (especially the sources thing), but when you're trying to understand something new and complex and you can't get the "gist" of it, ChatGPT does surprisingly well.
I've had it debug broken configs on my server and router (and explain to me why they were broken), help me practice a foreign language I've slowly been forgetting (hint: "I would like to practice $language, so let's have a conversation in $language where you only use very simple words." -> ChatGPT will obey), and help me understand how to use obscure software libraries that don't have much documentation online (e.g. Boost Yap, useful but with a dearth of blog / reddit posts about it).
Does it sometimes goof up? Yep, but it is such an incredibly useful tool nonetheless for the messy process of learning something new.