News for hackers. Stuff that matters.
Re:Behind the curtain: what it feels like to work in AI righ... by JumpCrisscross on Thursday April 06, 2023 @05:07PM (#35474467)
> Why do you think OpenAPI is so far out in front?

There is a network effect forming around its models. The strengths of its kit speak for themselves. (It also cannot be understated how making ChatGPT public, something its competitors were too feeble, incompetent and behind the curve to do, dealt OpenAI a massive first-mover advantage.)

But as others note, other models are in the ballpark. Where OpenAI is different is in the ecosystem of marketing literature, contracts, code and e.g. prompt engineers being written and trained with GPT in mind. That introduces a subtle switching cost, and not-so-subtle platform advantage, that–barring a Google-scale bout of incompetence–OpenAI is set to retain for some time.

Re:Behind the curtain: what it feels like to work in AI righ... by paulryanrogers on Thursday April 06, 2023 @06:56PM (#35475658)
Doubtful "feeble and incompetent" are fitting considering much of the research behind OpenAI was pioneered and published by those predecessors. More like "unwilling", and perhaps for good reason. Time will tell.
Re:Behind the curtain: what it feels like to work in AI righ... by JumpCrisscross on Thursday April 06, 2023 @07:27PM (#35475944)
> Doubtful "feeble and incompetent" are fitting considering much of the research behind OpenAI was pioneered and published by those predecessors

Feeble alone, possibly uninspired, would be Xerox PARC. The quality difference between Bard and ChatGPT informs my view of Google’s present incompetence; I won’t claim they always were.

Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:08PM (#35477551)
Google is where ideas go to die.
Re:Behind the curtain: what it feels like to work in AI righ... by sberens on Thursday April 06, 2023 @09:23PM (#35476896)
> other models are in the ballpark

How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better, especially around code generation / understanding.

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @10:42PM (#35477374)
People really don't seem to understand just how far ahead OAI is in its reasoning abilities (https://crfm.stanford.edu/helm/v0.2.2/?group=reasoning)

Maybe PaLM is near there (it's not evaluated on that page) but nothing else even comes close at all

Re:Behind the curtain: what it feels like to work in AI righ... by icelancer on Thursday April 06, 2023 @11:21PM (#35477648)
The level of denial people are willing to sink into regarding how good GPT-4 is compared to everything else is truly crazy. Not a single other project is an order of magnitude close to the quantitative and qualitative (actual experiential results, not just benchmarks) results that GPT-4 brings.
Re:Behind the curtain: what it feels like to work in AI righ... by satvikpendem on Thursday April 06, 2023 @11:45PM (#35477805)
I've had decent success with Open Assistant, an open source model. I'd say it's within the order of magnitude of ChatGPT, given the prompts I'm looking at, including reasoning prompts. This, I believe, is due to the overwhelmingly clean data that OA have managed to acquire through human volunteers.
Re:Behind the curtain: what it feels like to work in AI righ... by spaceman_2020 on Friday April 07, 2023 @01:43AM (#35478455)
I feel that there’s significant insecurity among a lot of coders about GPT-4. A lot of them are ignoring the pace of improvement and highlighting the few off chances where it gets things wrong.
Re:Behind the curtain: what it feels like to work in AI righ... by AussieWog93 on Friday April 07, 2023 @02:41AM (#35478816)
I've not used GPT-4, so it could be different, but regular old GPT-3.5 gets a _lot_ of things wrong.
Re:Behind the curtain: what it feels like to work in AI righ... by snowe2010 on Friday April 07, 2023 @03:52AM (#35479259)
GPT 4 is quite astounding. It might be wrong on occasion, but it will easily point you in the right direction most of the time. It still messes up, but like a twentieth of what 3.5 did. Honestly it is like an incredible rubber ducky for me. Not only can I just talk like I’m talking to a rubber duck but I can get fast, mostly informed, feedback that unblocks me. If I have a bunch of things competing for my attention I can ask gpt about one of them, a hard one, go do something else while it types out its answer, and then come back later and move on with that project.
Re:Behind the curtain: what it feels like to work in AI righ... by AussieWog93 on Friday April 07, 2023 @04:11AM (#35479402)
Is a 95% reduction in errors an exaggeration or is it really that much better? Might just need to drop the $20/mo if it's really improved that much.
Re:Behind the curtain: what it feels like to work in AI righ... by revelio on Friday April 07, 2023 @06:22AM (#35480083)
I'm a recent convert, been experimenting with converting one PL to another.

GPT-3.5 will get the gist of what the code is doing, and then provide what looks like a direct translation but differs in numerous details whilst having a bunch of other problems.

GPT-4 does a correct translation, almost every time.

It kills me that there's a waiting list for the API. I have put together some tools to integrate 3.5 into my workflow and it helps for my current task a lot (for others it's useless). But to really shine it needs to have API access to 4.

Re:Behind the curtain: what it feels like to work in AI righ... by IanCal on Friday April 07, 2023 @07:00AM (#35480317)
I recently finally got access to 4 in the API, it's good. It's much better imo at following the system prompt too. Faster than you see in chatgpt I think, not as fast as 3.5-turbo but definitely less tedious.
Re:Behind the curtain: what it feels like to work in AI righ... by IanCal on Friday April 07, 2023 @06:58AM (#35480307)
My only kind of quantitative answer is that I had 3.5 creating ember templates & data and it would get the templates mostly ok after a couple of iterations of fixing errors, almost never on the first shot (if ever). Often wouldn't quite get it in two, and data structures would often be kinda there but not quite. Required a lot more care with the prompts. 4 gave me working things every time first time (except only where it did things that were fine in a template but not currently supported by the custom framework I'm using them in), and didn't need as much hand holding.

Qualitatively, it's wildly different from gpt-3.5-turbo for discussions. 3.5 feels a little formulaic after a while with some kinds of questions. 4 is much more like talking to an intelligent person. It's not perfect, but I'm flipping between discussing a sporting thing, then medical malpractice, legal issues, technical specifications and it's doing extremely well.

If it's affordable for you, I'd really recommend trying it.

Re:Behind the curtain: what it feels like to work in AI righ... by icelancer on Saturday April 08, 2023 @04:21AM (#35491571)
For code 90%+ reduction is easily correct. For text and other content, I can't say, but I would guess it's not that good.

Anything involving reasoning, code, complex logic, GPT-4 is a breakthrough. GPT-3.5 turbo is more than good enough for poetry and the other text generation stuff.

Re:Behind the curtain: what it feels like to work in AI righ... by int_19h on Friday April 07, 2023 @04:17PM (#35486451)
My favorite part about GPT-4 is that if it generates code that is wrong, and you ask it to verify what it just wrote - without telling it whether it's wrong or not, much less pointing out the specific issue - more often than not it will spot the problem and fix it right away.

And yes, it does indeed make an amazing rubber duck for brainstorming.

Re:Behind the curtain: what it feels like to work in AI righ... by spaceman_2020 on Friday April 07, 2023 @06:12AM (#35480033)
GPT-4 is leaps ahead, and it's improving with every new release. The latest March 23 release is significantly better than the previous one and does a LOT of heavy lifting for my code at least.

At the very least, it's a massive productivity booster.

Re:Behind the curtain: what it feels like to work in AI righ... by sealeck on Friday April 07, 2023 @08:17PM (#35488939)
I think there's a lot of people writing boilerplate programs who are going to be freed from these menial tasks (i.e. no more Java enterprise application development, thankfully).
Re:Behind the curtain: what it feels like to work in AI righ... by JumpCrisscross on Friday April 07, 2023 @12:33AM (#35478060)
> How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better

I have at most moderate confidence in this hypothesis.

Re:Behind the curtain: what it feels like to work in AI righ... by spaceman_2020 on Friday April 07, 2023 @01:41AM (#35478441)
How much does reinforced learning from people using chatGPT help accelerate OpenAI’s advantage?