News for hackers. Stuff that matters.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by asimpletune on Saturday April 29, 2023 @06:47AM (#35751745)
I always tell people that if they want learn more about the hype to just try and use it to do actual work. Almost no one ever does, but when they do it becomes almost immediately clear how limited it is and what it is and isn’t good for.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Abroszka on Saturday April 29, 2023 @07:00AM (#35751803)
I use it almost every day for work. It has mostly replaced Google for me. A lot more convenient. Now whenever I use Google it's more or less just to look up the address of a specific site.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by kaba0 on Saturday April 29, 2023 @08:39AM (#35752330)
Well, google has become utterly bad at its job — I fail to find sites I remember verbatim quotes from, so expert google-usage is no longer a possibility. It will gladly leave out any of your important keywords, even if you add quotes around it, absolutely useless. Sure, the average person will search for “how old is X” not “x age”, but for more complex queries the first form is not a good fit.

That said, I can’t really use ChatGPT as a search engine, but I did plug it into a self-hosted telegram bot and I do ask it some basic questions from time to time - telegram is a good UI for it.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Abroszka on Saturday April 29, 2023 @10:03AM (#35752947)
Yeah, I agree. Google did a lot of work to make ChatGPT useful. It's clearly worse than it used to be.

I usually need someone to explain something to me. And I used Google before to land on a site where I could find the explanation (e.g. how to use a library). ChatGPT can explain most things I need and I can skip Google and the other sites. But it's indeed not a search engine, if you need factual information then your best bet is to find the documentations, articles, databases, etc.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by robby_w_g on Saturday April 29, 2023 @10:59AM (#35753505)
I tried phind after seeing it linked here a few times. It felt like it was a slower Google search with extra steps. The sources it used for the answers were the sources I would have found by searching "site:stackoverflow.com [my question]". It did distill the information decently well, but I'm skeptical it properly pulls in the context that the comment replies to the question/answers provides
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @11:03AM (#35753556)
Given how much it hallucinates, that's one very scary echo chamber to be in where you trust info reinterpreted by an algorithm instead of just reading it yourself. Yikes.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Abroszka on Saturday April 29, 2023 @02:25PM (#35755642)
It rarely hallucinates for me. But you need to know what it's capable of and how to use it to work effectively with it. You can't use it well if you think it's an all knowing sentient AI.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @02:47PM (#35755844)
How would you know how frequently it hallucinates if it has "mostly replaced Google" for you? Are you fact-checking all your queries? This is a very strange self-inflicted echo chamber.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Abroszka on Saturday April 29, 2023 @03:03PM (#35755993)
Because the code it generates is working? The recipes it gave to me are also delicious and I don't even have to read someone's life story on a blog before getting to the recipe.

Not really sure why you are so fixated on the echo chamber thing. We are on the internet! The biggest echo chamber humanity has ever built.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @03:58PM (#35756569)
Functional or delicious ≠ accurate to the source material.

Because it's a layer of abstraction, mate. One known to get things wrong, because it's an LLM. If I write a post about Richard Stallman's opinions on paedophilia and Jeffrey Epstein (https://en.wikipedia.org/wiki/Richard_Stallman#Controversies), and it incorrectly tells you that Stallman associated directly with Epstein, or is a paedophile himself, that would not be accurate to the source.

At least with a Google search result you can go more directly to the source. If the scientific method is getting to the truth, why on Earth would you put an obstacle in front of it?

If someone tells me x, am I going to believe them? No. So, why would I believe an LLM if it isn't presenting sources to me that are 1:1 in accuracy to the information it presents to me?

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Kamq on Saturday April 29, 2023 @05:21PM (#35757321)
> Functional or delicious ≠ accurate to the source material.

Alright, but most people aren't looking for accuracy relative to the source material.

They're looking for code that does a specific thing, or food that tastes a certain way.

> If the scientific method is getting to the truth, why on Earth would you put an obstacle in front of it?

Possibly because they aren't doing science. They may be doing software development or baking instead.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @05:33PM (#35757428)
Yup, and it hallucinates plenty with dev too, even over basic stuff like an NGINX config.

Given that it hallucinates in particular over measurements/specs/stats, I'd be extremely sceptical of taking a recipe from it, whether that's generated and original or coming from a known source.

Baking requires very specific measurements, the slightest mistake and it won't turn out well in most cases. Again, why go via an LLM and not a search engine to the actual source? It makes zero sense, especially if it only returns text and you can't see what the recipe produces if it's an existing thing.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Kamq on Saturday April 29, 2023 @06:13PM (#35757774)
> Again, why go via an LLM and not a search engine to the actual source?

I believe the argument presented (possibly in a separate thread) was that search engines have degraded to the point where what they show you is worse than LLM output.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by Dylan16807 on Saturday April 29, 2023 @07:28PM (#35758311)
There's no such thing as "the actual source" for how to make a type of food.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by jstanley on Saturday April 29, 2023 @06:55AM (#35751785)
It's good at writing. It's not good at knowing facts. If you give it all the relevant facts and ask it to do a writeup in a certain style it does a better first draft than a typical human, and in a lot less time.

If you ask it what the facts are, it just gives you a load of nonsense.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by asgerhb on Sunday April 30, 2023 @05:06AM (#35760872)
Even this is not always the case. I gave it a very rough draft and it actually made my structure worse, while at the same time using imprecise language. It looked like the correct style, but the content was not salvageable.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by vidarh on Saturday April 29, 2023 @07:27AM (#35751946)
I've had ChatGPT build a whole website for a project for me (HTML, CSS, backend code for signups, login, payments)
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by bamboozled on Saturday April 29, 2023 @09:28AM (#35752684)
I guess I've done similar things with Django in a very short ammount of itme.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by vidarh on Saturday April 29, 2023 @10:48AM (#35753382)
Even just typing what I had it write for me, and a typing speed well above average, would take many times as long, and I've done enough sites over the years to know I would not keep up that kind of typing speed, because I'd need to check things and go back and forth.

Put another way: People who don't pick up these tools and learn how to be effective at them will increasingly be at a significant performance disadvantage from people at their skill level who do pick them up.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @11:05AM (#35753571)
What you did is nothing a template/boilerplate couldn't otherwise provide.

You're trusting that it is secure and sensible in its output too.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by vidarh on Saturday April 29, 2023 @11:13AM (#35753680)
You know what it wrote for me then? No, it's not. It filled in custom logic per specifications.

But even if it was just boilerplate, I'd have had to apply it, and that takes time to do. I've started dozens of projects over the 28 years in this industry - I have a very good idea of how long it takes both me and a typical developer to do what I've had it do, and it's far faster.

And no, I'm not trusting it to be "secure or sensible" at all, no more than I trust a developer. But overall the quality of what it has produced all of which I've reviewed the same way I would with code delivered by a developer, has overall been of good quality. That does not mean free of bugs, any more than human developers write flawless code on first try, but it does mean I've had it write cleaner code than a whole lot of people I've worked with who'd take many times as long and cost a hell of a lot more.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @11:27AM (#35753834)
Well, post the source code and prompts in a GitHub repo then. Let's have a proper look at the code it output.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by vidarh on Sunday April 30, 2023 @03:27AM (#35760365)
Maybe once I'm done actually using it to speed up the project I'm working on. There's no benefit to me in slowing down to put effort into trying to convince you of something you can easily figure out for yourself if you actually wanted to.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by asimpletune on Saturday April 29, 2023 @12:20PM (#35754315)
Yeah, IDK, of all the things an AI can do for us, code generation seems to be the one I'm actually least interested in. It's anecdotes like this that sort reinforce that feeling.

I would much rather have a "rubber ducky" that I can try and explain my thought process to, and then it can try to question me and poke holes in my thinking. I think my expectations on AI are pretty realistic and I don't really expect it to ever be ever *thinking* in the way I associate with the word, not with today's SoA at least. In that respect I'm just not particularly interested in it generating code, but that also may come down to our individual preferences for how we write code.

At any rate, my issue is that it fails at the "rubber ducky" position I mentioned earlier. It's not really able to follow a train of thought that I have, in a reasonably competent way, and every time I have tried to do work with it I just end up feeling silly for anthropomorphizing something that I know isn't really, even if for a second. Just my $0.02 though, I'm glad so many people seem to like it and am happy for them.

Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by vidarh on Sunday April 30, 2023 @03:22AM (#35760348)
This is a bit like picking up a hammer and complaining it works poorly as a screwdriver when a screwdriver is what you want. Sure, if you want it to do something it's not suited for, then don't use it.
Re:Study: ChatGPT Outperforms Physicians in Quality, Empathe... by hammyhavoc on Saturday April 29, 2023 @11:02AM (#35753542)
Specify the work that you know it excels at first-hand, then list projects/tasks you have completed using it.

How do you know if they don't? Do you expect them to report back to a throwaway comment on HN? The sentiment is shifting on HN, that much is clear, just like it shifted with crypto.