News for hackers. Stuff that matters.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by siliconc0w on Friday December 23, 2022 @06:20PM (#34111803)
I've already regularly replaced many search queries with ChatGPT. The spam on G is just too painful to wade through. It's far from perfect and I wish I could 'tune' it to be less 'mealymouth' where the answers feel almost deliberately nerf'd to be qualified and generic but if you could tune it to offer strong opinions on these - not even politics but even product queries like, "best place to buy X" or "best value robot vacuum" that'd be interesting as these are super spammed on traditional search and I think an AI with strong opinions that could have a conversation about the product would be categorially better. Like I need to replace my water heater and I want a reasonable assortment of options for my situation from a dispassionate AI that isn't trying to sell me anything.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by Thorentis on Friday December 23, 2022 @07:47PM (#34112529)
How do you assess whether the information from ChatGPT is correct? It's downfall is that it very convincingly produces false information. At least in a Google search you can compare multiple sources, and blogspam is usually easy to filter out once you're good at identifying it. In ChatGPT, good and bad results look equally plausible.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by heliophobicdude on Friday December 23, 2022 @08:38PM (#34112908)
I started asking in a follow up for it to write a set of test cases.

I could tweak and run them to my liking.

I think for knowledge based things, you are better off using Google. But if it can help you narrow down your search, you can use that for Google.

For example, I queried, what projects are good for practicing data-intensive design.

It gave me a list of projects with descriptions. Perfect for me to Google more information for.

It's very good for narrowing down search and addressing follow ups. It's like the perfect Google Search companion.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by oomrazoom on Friday December 23, 2022 @09:38PM (#34113293)
you.com

Offers a chat feature that pairs the gpt-3 response with search results

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by CSMastermind on Friday December 23, 2022 @08:44PM (#34112956)
Lots of time when I'm searching I'm relying on recognition and not recall. I know the right answer but I don't have it memorized.

If I search for python for loop syntax or metallica bob seger cover or tuck rule game year, etc. I know the right answer if you show it to me and given enough time to think I probably could recall it but I don't have the information at the top of my mind.

If chatgpt returns something that's wrong I'll know it's wrong and then maybe go check google.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by WalterBright on Friday December 23, 2022 @09:10PM (#34113132)
Some people have tried asking ChatGPT to do some D coding. The results looked like plausible D code, but were subtly wrong.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by 0x6c6f6c on Friday December 23, 2022 @09:11PM (#34113136)
> it very convincingly produces false information

This is exactly the scenario I encountered. I have a teammate who is using ChatGPT to ask questions as opposed to searching documentation and it gave incorrect info all around on all the following:

1. Where the property would be found in the event schema 2. What structure the value would be 3. The fact that it's not included at all for AWS AppSync specifically in question, even if the previous two had been answered correctly regarding API Gateway event schema.

All of this was available in the first doc returned in the search results of both API Gateway and AppSync respectively.

The fact it convincingly tells you otherwise causes additional friction since they're pointing at their screen telling me I'm wrong cause look at the response from ChatGPT.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by nprateem on Saturday December 24, 2022 @03:50AM (#34115150)
I think solving this inaccuracy will be openai's biggest challenge. The web is safe for now. Once they do fix it it's basically game over.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by ilaksh on Saturday December 24, 2022 @08:42AM (#34116700)
You can turn the temperature down on the OpenAI API and it is much more accurate. They also have a code-specific model which I don't think ChatGPT is using.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by moritonal on Saturday December 24, 2022 @05:21AM (#34115571)
But, you have to admit that if ChatGPT had a process whereby it'd internally list the sites most likely relevant to your question (essentially Google the keywords) then included those sites as context it'd likely ace your question?

Basically, if Google owned ChatGPT the result would be very different.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by spaceman_2020 on Saturday December 24, 2022 @05:55AM (#34115743)
Whenever chatGPT gives me a function and I try it out and it doesn’t work, I just tell it “hey this isn’t working” and it will correct itself
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by lumost on Friday December 23, 2022 @10:29PM (#34113634)
How do you do this with google search when half the results are spammed copies of the same content?

For many cases you can get enough information out of chatGPT to retrieve a specific corroborating reference. For coding tasks, you can just try the code, ask it for tests, or give it code review like feedback.

I’d suggest asking it about deep technical topics that you are familiar with, then work out it’s limitations. It’s effectively equivalent to a universal teaching assistant, sometimes wrong - usually points in the right direction.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by vel0city on Friday December 23, 2022 @10:35PM (#34113678)
The answer is still the same, I evaluate the source. Which is near impossible to truly know with bots like ChatGPT.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by drusepth on Saturday December 24, 2022 @02:45AM (#34114872)
I just treat ChatGPT as another source. It's become a go-to for questions (according to my log, this week I've asked it about naturopathic medicine, game development, algorithms, FSAs, vocabulary, and general trivia).

It's waaaaay faster to get an immediate answer -- and be able to ask pointed, follow-up questions, which you couldn't otherwise do with a traditional site. And it's a lot easier to just fact-check statements online than to wade headfirst into something you know nothing about in random sites of varying qualities.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by trees101 on Saturday December 24, 2022 @04:07AM (#34115208)
https://www.perplexity.ai

This augments ChatGPT results with Bing results to provide references you can look up.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by rerdavies on Saturday December 24, 2022 @06:03AM (#34115794)
Just ask for links. The example below is contrived. But when you're looking for very specific information, GptChat can save you from wading through pages of Google Search results.

Providing a list of 30,000 pages that are only tangentially related to what you want is not a virtue. -----

> Give me a link to page that gives an introduction to quantum computing.

Here is a link to a page that provides an introduction to quantum computing:

https://www.ibm.com/quantum-computing/learn/what-is-quantum-...

This page gives a brief overview of what quantum computing is, how it works, and some of the potential applications. It also includes links to more in-depth resources for those who want to learn more.

I hope this helps! Let me know if you have any other questions.

> Give me another link.

Here is another link that provides an introduction to quantum computing:

https://www.dwavesys.com/tutorials/getting-started-with-quan...

This page gives a high-level overview of quantum computing, including the basics of quantum mechanics and how it is used in computing. It also discusses the potential applications of quantum computing and some of the challenges and limitations of the technology.

I hope this helps! Let me know if you have any other questions.

...

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by hooande on Friday December 23, 2022 @08:36PM (#34112895)
The biggest problem with ChatGPT is that the training data is frozen in 2021. I would imagine that there have been robot vacuums released in the last year. I wouldn't search for this using ChatGPT because I know that the answers will be out of date. It's the equivalent of limiting a google search to have an end date in Dec 2021. If I'm buying something, I want to know what the recent options are
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by Izkata on Friday December 23, 2022 @09:22PM (#34113206)
> I would imagine that there have been robot vacuums released in the last year.

A really specific example that has an affirmative, not just in a basic release but a leap forward: The beginning of 2022 is when Roborock released their robot vacuum dock that empties, refills, and cleans the mop on their top robot vacuum/mop combo.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by Aicy on Friday December 23, 2022 @08:40PM (#34112920)
You are making a mistake. ChatGPT does not care about truth, but just giving an answer that sounds like what you'd like to hear the most.

So replacing SEO blog spam with ChatGPT is not an improvement. Blog spam optimises for SEO rules, ChatGPT optimises to sound like legitimate (but wrong) blog spam.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by Enginerrrd on Friday December 23, 2022 @09:24PM (#34113214)
Honestly, I agree with what you're saying, but there are alarm bells in my head going off saying that this sounds suspiciously like what the school librarians and teachers used to say against wikipedia. And despite its flaws, Wikipedia is usually the first place I stop for knowledge.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by yosame on Friday December 23, 2022 @09:37PM (#34113287)
The reason why Wikipedia is such a great first stop for knowledge is because it has citations you can follow.

With ChatGTP you have to take the information on pure trust, and the information it gives is not at a level you can trust yet.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by drewcon on Friday December 23, 2022 @11:55PM (#34114112)
Why couldn’t it also offer citations then. That sounds ideal actually.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by pixl97 on Saturday December 24, 2022 @12:23AM (#34114249)
Are we sure the citations were not written by Chatgpt too?
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by statheros on Saturday December 24, 2022 @12:37AM (#34114308)
>> Explain briefly what the Fourier Transform is. Provide sources with more information about the topic.

>> The Fourier transform is a mathematical technique that decomposes a function or signal into its constituent frequencies. It allows us to represent a function in terms of the frequencies that make it up, rather than in terms of its time or spatial behavior.

The Fourier transform has many applications in a variety of fields, including signal and image processing, engineering, and physics. It is a fundamental tool in the analysis of periodic phenomena and is widely used in the study of waves and oscillations.

For more information about the Fourier transform, you may find the following resources helpful:

Wikipedia: https://en.wikipedia.org/wiki/Fourier_transform

Khan Academy: https://www.khanacademy.org/math/differential-equations/lapl...

Wolfram MathWorld: http://mathworld.wolfram.com/FourierTransform.html

Problem solved.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by 6510 on Saturday December 24, 2022 @01:58AM (#34114688)
woah! No need for the "explain briefly" part but when I asked for 50 references it wanted money.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by oezi on Saturday December 24, 2022 @02:05AM (#34114712)
If you ask for an arxiv or pubmed link it also happily makes a link up. But the link just leads to a random paper.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by Engineering-MD on Saturday December 24, 2022 @08:35AM (#34116653)
No https for wolframalpha?
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by owlninja on Saturday December 24, 2022 @12:51AM (#34114376)
I could be mistaken but I think another ChatGPT discussion on HN had people saying it would sometimes cite sources, and other times just make them up.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by numpad0 on Saturday December 24, 2022 @01:15AM (#34114486)
Because neural network frameworks still offer no implements to trace output back to training data, or for that matter, means to compile an AI without brute forcing through parameters, and also because ChatGPT is not interpreting reference materials in text forms but talking from its "memory".
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by drusepth on Saturday December 24, 2022 @02:50AM (#34114892)
That sounds like https://news.ycombinator.com/item?id=33910863 which was posted here on HN a few weeks ago. It's a ChatGPT-esque model that provides URLs for where it synthesized each individual sentence from, but also feels a lot more summary-heavy rather than strictly new text a la ChatGPT. The results leave quite a bit to be desired, but it's nice to know there are people working on exactly what you're suggesting. :)

OpenAI, specifically, has also been working on allowing GPT models to browse the internet and include citations with e.g. https://openai.com/blog/webgpt/

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by Enginerrrd on Saturday December 24, 2022 @08:50AM (#34116754)
>The reason why Wikipedia is such a great first stop for knowledge is because it has citations you can follow.

I don't know the actual statistics, but it seems like in my experience, at least 50% of the time, the linked sources are dead links, or don't have the information they're purported to have.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by edgyquant on Friday December 23, 2022 @10:01PM (#34113451)
Right, this is a huge problem. Wikipedia does not stick to purely scientific topics this pages become hotly debated and political. This isn’t a partisan thing, you can find examples going both ways, but since Wikipedia is not a telling of sourced facts[1] and can not be expected to cover all topics elaborately it has inadvertently controlled the flow of peoples understanding and id say even interests.

1. it actually has policy against this to make understanding sources simpler to the layman for any topic.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by tornato7 on Friday December 23, 2022 @10:20PM (#34113576)
Indeed, the current iteration of ChatGPT can't access the internet, but a future iteration should be able to look up references from reliable sources and fact check itself.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by jsight on Friday December 23, 2022 @11:02PM (#34113843)
I don't know, I've had it get a lot of really basic things wrong. Where do the pilots sit in a Cessna Skymaster? What are their responsibilities? Answers with basic math mistakes too.

Its amazing at some things, but there are foundational mistakes everywhere. Wikipedia has always had much higher quality overall.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by alfor on Friday December 23, 2022 @10:01PM (#34113450)
from what I understand it’s trained to give the good anwser but can be misguided (like people)

I seem it lack the capability to say it’s level of confidence in the anwers. I gave me very good result when I give him lots of context. You can put pages of code about a project and then ask a question to complete. In this case it’s way more accurate (more context)

At the speed it’s progressing I am sure it will be very good in a year or two.

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by lowbloodsugar on Saturday December 24, 2022 @01:22AM (#34114520)
Bahaha and Google does?? Paid content and garbage.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by runeks on Saturday December 24, 2022 @04:40AM (#34115370)
> […] giving an answer that sounds like what you'd like to hear the most.

How did the training of ChatGPT make it do this?

And could a different training avoid this?

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by vikp on Friday December 23, 2022 @10:04PM (#34113475)
I've had the exact same problem, and have experimented with Kagi, DDG, etc, to try to find better results.

ChatGPT, in my opinion, is great for "how do I code X" type questions, but isn't so good at the types of queries you mentioned, due to the lack of a search engine.

My weekend project was an open source combination of Google + GPT that returns pretty good results for these types of queries. You can check it out here - https://github.com/VikParuchuri/researcher

Example - the response to "what are the best current smartphones" is:

`...According to Search Result [2], the best phones have been thoroughly reviewed and tested, and include the Apple iPhone 14 and 14 Pro, the Pixel 7 Pro and the Samsung Galaxy S22 Ultra. Search Result [5] also states that there are strong options available at all price levels, so you don't have to spend a lot to get something great...`

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by morsecodist on Friday December 23, 2022 @10:30PM (#34113643)
What kinds of things do you query? I am genuinely curious. I have seen people say things like this about ChatGPT and it makes me wonder if we are using the same product. I have developed a hobby application with GPT-3 completion and I have ChatGPT more or less daily a week or so mostly for programing related queries. I have found the error rate to be extremely high, like > 20%. Not just sub-optimal but producing code that does not work, in python, which is a very common language. It is really tempting to use it for things like regexes or time format strings (the latter it does do well IMO) because often you want something too specific to google. When it works it's great but my trust in it has been shaken and I don't see myself using it long term.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by scarface74 on Saturday December 24, 2022 @12:26AM (#34114257)
I had a JSON file with the following format:

[{“Key”:”Company”},{“Value”:”${company}”},…]

I asked it to write a Python script that replaces any word surrounded by ${} with the value in its corresponding environment variable and accept the path of the file as a command line argument —json-file using argparse. It worked perfectly.

Then I started asking it to write a script to successively do the following

Given the same json file, write a snippet of YML that looks like sample CF templates parameter section that I gave it.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

It worked perfectly.

Then I told it to accept an optional argument that generated the corresponding meta data section. It worked flawlessly. I gave it sample expected output

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

Then I told it to output the format needed to pass those parameters to a nested stack and gave it an example;

Parameters: Company: !Ref company

…

It worked again.

Finally, I needed it to generate the Python code to generate the CodeBuild Environment section and I gave it an example.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by rish1_2 on Friday December 23, 2022 @10:54PM (#34113791)
I already Q&A site like StackOverflow to correct misinformation or just to share: https://www.gptoverflow.link/
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by pleb_nz on Saturday December 24, 2022 @04:13AM (#34115240)
I use kagi. I reluctantly paid the monthly fee, but by Jebs it's worth it to get pure accurate results with no ads and filters.
Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by ilaksh on Saturday December 24, 2022 @08:40AM (#34116687)
Get an OpenAI API key. Then `npm i -g askleo` and set env var OPENAI_API_KEY

`> askleo how large is the average dog`

`> runleo list files in reverse date order`

Re:A New Chat Bot Is a ‘Code Red’ for Google’s Search Business by more_corn on Saturday December 24, 2022 @09:23AM (#34117019)
The answer is probably AI middleware (something to stand between you and the LLM and help tune your prompts). We did this for a hackathon recently and it worked quite well. We also open-sourced OpenPlayground so you can test prompt tuning against multiple LLMs.