News for hackers. Stuff that matters.
Re:Behind the curtain: what it feels like to work in AI righ... by fdgsdfogijq on Thursday April 06, 2023 @04:24PM (#35473881)
I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room.

Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

Re:Behind the curtain: what it feels like to work in AI righ... by UncleOxidant on Thursday April 06, 2023 @04:40PM (#35474082)
Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?
Re:Behind the curtain: what it feels like to work in AI righ... by nicpottier on Thursday April 06, 2023 @04:47PM (#35474176)
Until we see otherwise, don't we have to assume there's some secret sauce? Bard doesn't match GPT4 and it isn't for a lack of trying. (though perhaps that will change, so far that's the case)
Re:Behind the curtain: what it feels like to work in AI righ... by xnx on Thursday April 06, 2023 @04:51PM (#35474233)
Google is going slow. They might be behind, but we haven't seen their best effort yet. Google has a 540 billion parameter model.
Re:Behind the curtain: what it feels like to work in AI righ... by nicpottier on Thursday April 06, 2023 @05:01PM (#35474377)
So they say, and I agree this seems likely. But until they release it that's all talk, and they have plenty of incentive to release it.
Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:15PM (#35477612)
And plenty of reason not to: search ads.
Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:14PM (#35477608)
Google also has Sundar and Ruth who’d rather focus on how to get another ad on the SERP than kill the golden goose. They’re not going slow, they just don’t have the leadership for the moment.
Re:Behind the curtain: what it feels like to work in AI righ... by mewpmewp2 on Thursday April 06, 2023 @05:07PM (#35474457)
Maybe it is just time to train and also good training data for prompts which openai has gathered for so long already? E.g. there is a bottleneck on how fast you can train and also gather good data.
Re:Behind the curtain: what it feels like to work in AI righ... by UncleOxidant on Thursday April 06, 2023 @05:12PM (#35474537)
Possibly, but wouldn't Google and Meta have access to way more compute resources and data than OpenAI? Google has been touting their TPUs for several years now.
Re:Behind the curtain: what it feels like to work in AI righ... by Robotbeat on Thursday April 06, 2023 @10:29PM (#35477297)
OpenAI has access to Microsoft and Azure. That’s bigger than Meta, roughly on par with Google in terms of capability and higher in terms of market cap.
Re:Behind the curtain: what it feels like to work in AI righ... by chillfox on Thursday April 06, 2023 @10:50PM (#35477426)
Google has the compute, from the comparisons I have seen Bard smokes GPT-3.5-Turbo on response times. So my guess is that internal politics prevents them from putting out something better. There would have to be immense pressure from the search division to not make them obsolete.
Re:Behind the curtain: what it feels like to work in AI righ... by int_19h on Friday April 07, 2023 @04:32PM (#35486601)
Bard is also a fair bit worse than GPT-3.5, though, so that can be a function of model size.
Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:17PM (#35477625)
Without Nadella footing the compute bills, nobody would be taking about OpenAI. He’s brilliant, he let the start up take on huge risk to quietly claim the gains for m$.
Re:Behind the curtain: what it feels like to work in AI righ... by saynay on Thursday April 06, 2023 @04:58PM (#35474329)
Bard would not trick anyone into thinking it was sentient, yet something they have supposedly did. I just think Google has far more to lose than Bing, so they are being more cautious.
Re:Behind the curtain: what it feels like to work in AI righ... by dragonwriter on Thursday April 06, 2023 @05:06PM (#35474446)
> Bard would not trick anyone into thinking it was sentient, yet something they have supposedly did

Bard is overtly a reduced-resources model compared to the best version of the same technology (which, if true, is probably a boneheadedly bad choice for a public demo when everyone is already wowed by the people who got theirs out first, but easily explains that disparity. Though so does “guy who wanted public attention made stuff up well-calibrated to that goal.”)

Re:Behind the curtain: what it feels like to work in AI righ... by Animats on Thursday April 06, 2023 @09:12PM (#35476823)
> Bard is overtly a reduced-resources model compared to the best version of the same technology

There's a scaling problem. ChatGPT/LLM systems cost far more to run per query than the Google search engine. Google can't afford to make those the first line query reply.

A big business model question is whether Google will insist you be logged in to get to the large language model.

At Google scale, these things are going to have to be a hierarchy. Not everything needs to go to a full LLM system. Most Google queries by volume can be answered from a a cache.

Re:Behind the curtain: what it feels like to work in AI righ... by jacquesm on Friday April 07, 2023 @02:47AM (#35478853)
> Most Google queries by volume can be answered from a a cache.

And given how aggressively they limit the number of search results (in spite of listing some ridiculous number of results on page #1) that percentage may well be very large.

Re:Behind the curtain: what it feels like to work in AI righ... by fdgsdfogijq on Thursday April 06, 2023 @04:52PM (#35474251)
I dont work at Google, I think other FAANGS underinvested in this area as they didnt think it was promising. But I will admit, I am suspicious that Google is incompetent. Probably they can come back given how much money they will be forced to throw at it. But Bard is clearly behind and I dont believe their "abundance of caution" arguments for why Bard cant even code.
Re:Behind the curtain: what it feels like to work in AI righ... by UncleOxidant on Thursday April 06, 2023 @05:02PM (#35474404)
The impression I get is that they purposely limited Bard in certain ways. It does not seem to be willing to code and makes a lot of excuses when you ask it to (at least as of a couple of weeks ago when I last tried it). Did they put this limitation on it because it's not so good at coding or because they don't want it to be abused in some way? In my experience I had to trick it into coding by telling it to pretend that it had created it's own programming language and then implement an algorithm in it. It seemed to think that it had created Python.

> I am suspicious that Google is incompetent.

Google has put a lot of effort and investment into AI. With Bard I get the feeling they're not showing us what they really have - it's like for some reason they're holding back the good stuff, at least that's my suspicion.

Re:Behind the curtain: what it feels like to work in AI righ... by pixl97 on Thursday April 06, 2023 @05:29PM (#35474755)
I am still wondering if Google is experiencing it Kodak moment.

They have the dominant product that makes them billions and billions of dollars at 'relatively' low cost.

The new dominant product is on its way, but it costs far more to operate and will net them far less money, so... um no one wants to kill the goose that is still laying golden eggs, even though its days are numbered already.

Re:Behind the curtain: what it feels like to work in AI righ... by xyzzy123 on Thursday April 06, 2023 @07:12PM (#35475803)
I think it's not just run cost - there's also a risk of ad revenue destruction, that has to be worrying.
Re:Behind the curtain: what it feels like to work in AI righ... by iamleppert on Thursday April 06, 2023 @09:17PM (#35476848)
100% this -- a Google search requires multiple input queries, refinement, and scrolling a list of possible answers that are really just links to other web sites. The ChatGPT experience is far superior to this, for the average consumer and getting close for the power user. It's a better way to ask the Internet what it knows with a more natural interface that everyone already knows how to use -- real natural language. Less cognitive overhead, no busy search results that require clicking back and forth and (for now) no ads. That last part is key -- ChatGPT is doing Google's job right now, and not even having to run ads. Google doesn't even offer a premium no ads option for search and if they did I doubt enough people would buy it anyway to matter.

If I was Google I'd be worried. Very worried indeed. They either need to dramatically change their entire company within 18 months, or accept they are going to loose substantial amount of market -- and once its gone, it's gone in a first mover, winner takes all environment like what we have right now. Just ask Google themselves what it felt like back in the early 2000's when they completely destroyed the other search engines.

Re:Behind the curtain: what it feels like to work in AI righ... by ModernMech on Thursday April 06, 2023 @11:11PM (#35477572)
> The ChatGPT experience is far superior

Now it is. Google used to be good too, until ads started looking like search results, and then the first page became entirely ads.

In the future, when you ask ChatGPT to help you write your resume, it will try to upsell you a premium account in linked in. It will withhold its best resume advice only for LinkedIn premium users after all.

You think Clips was bad? You’ve seen nothing yet.

Re:Behind the curtain: what it feels like to work in AI righ... by brookst on Thursday April 06, 2023 @11:31PM (#35477718)
How does that help Google?
Re:Behind the curtain: what it feels like to work in AI righ... by satvikpendem on Thursday April 06, 2023 @11:47PM (#35477820)
Yep, Bing already has ads now inside its chat answers.
Re:Behind the curtain: what it feels like to work in AI righ... by alain94040 on Thursday April 06, 2023 @09:20PM (#35476875)
I don't see ads as a big problem with ChatGPT. You could put a side-bar on the right and on the fly recommend products relevant to the on-going conversation.

The cost of computing these ads would be a lot more than today's keyword-based approach, that's certainly a problem. But think of hyper-relevant ads, based on the chat itself. There's a lot of information there, that beats tracking people's behavior online all day.

Re:Behind the curtain: what it feels like to work in AI righ... by HyperSane on Thursday April 06, 2023 @11:23PM (#35477661)
And you could even program the LLM to try to subtly manipulate people into using the advertised product.
Re:Behind the curtain: what it feels like to work in AI righ... by visarga on Friday April 07, 2023 @12:10AM (#35477947)
A LLM trying to trick and manipulate? That's how your multi-million dollar model quickly becomes worthless in the eyes of the public.
Re:Behind the curtain: what it feels like to work in AI righ... by HyperSane on Friday April 07, 2023 @01:08AM (#35478246)
Not if it was REALLY good at everything else and really subtle.
Re:Behind the curtain: what it feels like to work in AI righ... by bigDinosaur on Friday April 07, 2023 @01:23AM (#35478325)
People haven't been up in arms about the dishonesty around 'native' marketing.
Re:Behind the curtain: what it feels like to work in AI righ... by Lio on Friday April 07, 2023 @03:28AM (#35479089)
I could see this happening.

I’ve been infuriated with DuckDuckGo on occasion because it refused to exclude certain results.

In fact when you add an exclusion clause it simply boosts those results further instead of removing them.

I’ve been told this is because the underlying search providers refuse to exclude paying customer even when you explicitly don’t want to hear from them.

I could definitely see this happening in LLM answers too and I don’t expect it to be particularly subtle.

Re:Behind the curtain: what it feels like to work in AI righ... by flir on Friday April 07, 2023 @12:12AM (#35477958)
You can inline the ads. Ask it to write you a poem about flowers that includes an ad for interflora.
Re:Behind the curtain: what it feels like to work in AI righ... by visarga on Friday April 07, 2023 @12:06AM (#35477925)
> But think of hyper relevant ads

That depends on ad publishers, right? If they want to sell A, B and C and I am interested in D, then Google's still showing one of A, B or C to me. D doesn't make profit if there is nobody paying for ads.

Google is advertising things we don't need, that's why ad clicks are so abysmal. LLMs won't change that.

Re:Behind the curtain: what it feels like to work in AI righ... by DonHopkins on Thursday April 06, 2023 @10:01PM (#35477123)
Oh how the term "Kodak Moment" has jumped the shark.

https://en.wiktionary.org/wiki/Kodak_moment

Etymology

(moment worth photographing): From an Eastman Kodak Company advertising campaign.

(business's failure to foresee): In reference to the Eastman Kodak Company's decline when cameras and film were overtaken by smartphones and digital technologies.

Noun Kodak moment (plural Kodak moments)

(informal) A sentimental or charming moment worthy of capturing in a photograph.

(informal) The situation in which a business fails to foresee changes within its industry and drops from a market-dominant position to being a minor player or declares bankruptcy.

Kodak Film Commercial - These are the Moments - Baby (1993):

https://www.youtube.com/watch?v=vPSCmnoVgEU

Re:Behind the curtain: what it feels like to work in AI righ... by jacquesm on Friday April 07, 2023 @02:45AM (#35478837)
> With Bard I get the feeling they're not showing us what they really have

I highly doubt this. If they had it they would show it because if they don't react swiftly and decisively their brand will be in 'catch up' mode rather than out front where they are used to being.

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @07:02PM (#35475709)
From what I heard, they don't have scaling capabilities to deploy the bigger models right now which they're working on and that's the biggest issue.
Re:Behind the curtain: what it feels like to work in AI righ... by sanderjd on Friday April 07, 2023 @03:10AM (#35478985)
I have no particular insight into this either way, but I find it pretty hard to believe that OpenAI has more scaling capability than Google.
Re:Behind the curtain: what it feels like to work in AI righ... by revelio on Friday April 07, 2023 @06:24AM (#35480098)
It's not just about hardware, it's about the software infrastructure to go with it. Other than OpenAI most researchers weren't interested in "merely" scaling things up because that was/is seen as simple engineering, unworthy of the great minds who dream up new algorithms.
Re:Behind the curtain: what it feels like to work in AI righ... by sanderjd on Friday April 07, 2023 @08:16PM (#35488932)
So I worked at Google for awhile (not on AI) and this wasn't my impression of what was going on with the AI teams there. But :shrug:, I dunno.
Re:Behind the curtain: what it feels like to work in AI righ... by brookst on Thursday April 06, 2023 @11:30PM (#35477705)
Google was built by brilliant people whose mission was to make any information available to anyone, instantly.

Google is run by smart people whose mission is to maximize clicks on ads. If a user finds what they’re looking for quickly, that’s lost revenue.

Google’s profit motives are not aligned with useful AI. The better AI is, the less people need to click through to lots of web pages and ads, the less revenue for Google.

I don’t think they can catch up without a major pivot in business model. It’s very hard to be deeply invested in providing more value if it means reducing your revenue.

Re:Behind the curtain: what it feels like to work in AI righ... by terpimost on Friday April 07, 2023 @12:56AM (#35478175)
FAANG companies have distribution. They can sell anything. Give them a bit of time. They are not companies of a single product, they milk people better than governments.
Re:Behind the curtain: what it feels like to work in AI righ... by brookst on Friday April 07, 2023 @07:16AM (#35480423)
80% of Google’s revenue is from Google Ads.

https://seekingalpha.com/article/4469984-how-does-google-mak...

Re:Behind the curtain: what it feels like to work in AI righ... by rlt on Friday April 07, 2023 @05:49AM (#35479906)
Classic “innovators dilemma”.
Re:Behind the curtain: what it feels like to work in AI righ... by m463 on Friday April 07, 2023 @01:58AM (#35478532)
I always thought that google would have an advantage because they have so much data.
Re:Behind the curtain: what it feels like to work in AI righ... by redox99 on Thursday April 06, 2023 @05:02PM (#35474397)
I don't know about GPT4, but GPT3.5 I'd bet is pretty traditional and boring. It's power comes from a really good, properly curated dataset (including the RLHF).

GPT3.5 turbo is much more interesting probably, because they seem to have found out how to make it much more efficient (some kind of distillation?).

GPT4 if I had to make a very rough guess, probably flash attention, 100% of the (useful) internet/books for it's dataset, and highly optimized hyperparameters.

I'd say with GPT4 they probably reached the limit of how big the dataset can be, because they are already using all the data that exists. Thus for GPT5 they'll have to scale in other ways.

Re:Behind the curtain: what it feels like to work in AI righ... by brendamn on Thursday April 06, 2023 @08:52PM (#35476692)
> I'd say with GPT4 they probably reached the limit of how big the dataset can be

I’m curious about this too; not just on the dataset size, but also the model size. My hunch is that the rapid improvements of the underlying model by making it bigger/giving it more data will slow, and there’ll be more focus on shrinking the models/other optimisations.

Re:Behind the curtain: what it feels like to work in AI righ... by int_19h on Friday April 07, 2023 @04:22PM (#35486509)
I don't think we're anywhere close to the limit of sheer hardware scalability on this. Returns are diminishing, but if GPT-4 (with its 8+ k context window) is any indication, even those diminishing returns are still very worthwhile.

If anything, I wonder if the actual limit that'll be hit first will be the global manufacturing capacity for relevant hardware. Check out the stock price of NVDA since last October.

Re:Behind the curtain: what it feels like to work in AI righ... by flangola7 on Thursday April 06, 2023 @09:03PM (#35476759)
According to financial reports they are building a $225 million supercomputer for AI. What we can probably expect is the same dataset with even more compute ran on it.
Re:Behind the curtain: what it feels like to work in AI righ... by theturtletalks on Thursday April 06, 2023 @09:13PM (#35476834)
Is there a limit on how big the context size can be?
Re:Behind the curtain: what it feels like to work in AI righ... by kickette on Thursday April 06, 2023 @09:49PM (#35477065)
There’s no theoretical limit
Re:Behind the curtain: what it feels like to work in AI righ... by PeterisP on Thursday April 06, 2023 @10:34PM (#35477316)
There is a soft limit due to the computation required; the currently used model architectures are quadratic with respect to context size, so if you want ten times larger context size, that's going to need a hundred times more effort.
Re:Behind the curtain: what it feels like to work in AI righ... by lexandstuff on Thursday April 06, 2023 @09:18PM (#35476858)
In this interview [1] with Ilya Sutskever, he indicates that they aren't even close to tapping out of data.

[1] https://www.youtube.com/watch?v=Yf1o0TQzry8&t=656s

Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:10PM (#35477565)
That interview is mind blowing stuff.
Re:Behind the curtain: what it feels like to work in AI righ... by rcpt on Friday April 07, 2023 @12:20AM (#35478002)
Yeah great interview style. It's non stop content.
Re:Behind the curtain: what it feels like to work in AI righ... by fock on Friday April 07, 2023 @01:30AM (#35478369)
I wonder if we saw the same video - or maybe it is just ChatGPT being "great" in the wild? I see one guy asking another guy simple questions and getting weaselwords for an answer.
Re:Behind the curtain: what it feels like to work in AI righ... by revelio on Friday April 07, 2023 @06:47AM (#35480227)
The interviewer and his questions is in some way more impressive than the answers, which is weird.
Re:Behind the curtain: what it feels like to work in AI righ... by rcpt on Friday April 07, 2023 @11:49AM (#35482946)
Yeah. He's fast and it's just question after question. I didn't realize that was something I wanted to see in interviews.
Re:Behind the curtain: what it feels like to work in AI righ... by revelio on Friday April 07, 2023 @05:12PM (#35487039)
Right, that's totally it. I came away thinking the interviewer seemed way sharper than the interviewee which is pretty rare. The sheer throughput and speed of interesting questions was incredible. Too bad that many of the answers were not.
Re:Behind the curtain: what it feels like to work in AI righ... by jakeinspace on Thursday April 06, 2023 @11:13PM (#35477594)
To be fair, if the opposite were true, it might not be wise to admit. Saturating available high quality training data is one of the few ways anyone can see OpenAI slowing down.
Re:Behind the curtain: what it feels like to work in AI righ... by tome on Friday April 07, 2023 @06:31AM (#35480134)
Yes, it's a bit strange. I would have thought

1. They would already be using everything they can get 2. They would easily be able to explain what they're not using, without giving away sensitive secrets.

Re:Behind the curtain: what it feels like to work in AI righ... by diego on Thursday April 06, 2023 @05:05PM (#35474432)
If you try Bard or Claude or character.ai they are not far behind GPT4. They might even be on par in terms of raw LLM capabilities. ChatGPT has better marketing and in some cases better UX. A lot of this is self-fulfilling. We think it's far ahead, so it appears to be far ahead.
Re:Behind the curtain: what it feels like to work in AI righ... by dragonwriter on Thursday April 06, 2023 @05:07PM (#35474465)
> If you try Bard or Claude or character.ai they are not far behind GPT4

Bard is way behind ChatGPT with GPT-3.5, much less GPT-4. Haven’t tried the others, though.

OTOH, that’s way behind qualitatively, not in terms of time-of-progress. So I don’t think it is at all an insurmountable lead, as much as it is a big utility gap.

Re:Behind the curtain: what it feels like to work in AI righ... by sashank_1509 on Friday April 07, 2023 @01:02AM (#35478205)
In my experiments:

GPT4>ChatGPT>Claude>Character AI> Bard

Claude and Character AI are great at holding a conversation but they lack the ability to do anything specialized that really makes these LLM’s useful in my day to day life. I ask GPT-4 and ChatGPT questions I would ask in stackoverflow, I can’t do that with Claude or Character AI. Bard actually seems behind even conversationally to the rest

Re:Behind the curtain: what it feels like to work in AI righ... by JumpCrisscross on Thursday April 06, 2023 @05:07PM (#35474467)
> Why do you think OpenAPI is so far out in front?

There is a network effect forming around its models. The strengths of its kit speak for themselves. (It also cannot be understated how making ChatGPT public, something its competitors were too feeble, incompetent and behind the curve to do, dealt OpenAI a massive first-mover advantage.)

But as others note, other models are in the ballpark. Where OpenAI is different is in the ecosystem of marketing literature, contracts, code and e.g. prompt engineers being written and trained with GPT in mind. That introduces a subtle switching cost, and not-so-subtle platform advantage, that–barring a Google-scale bout of incompetence–OpenAI is set to retain for some time.

Re:Behind the curtain: what it feels like to work in AI righ... by paulryanrogers on Thursday April 06, 2023 @06:56PM (#35475658)
Doubtful "feeble and incompetent" are fitting considering much of the research behind OpenAI was pioneered and published by those predecessors. More like "unwilling", and perhaps for good reason. Time will tell.
Re:Behind the curtain: what it feels like to work in AI righ... by JumpCrisscross on Thursday April 06, 2023 @07:27PM (#35475944)
> Doubtful "feeble and incompetent" are fitting considering much of the research behind OpenAI was pioneered and published by those predecessors

Feeble alone, possibly uninspired, would be Xerox PARC. The quality difference between Bard and ChatGPT informs my view of Google’s present incompetence; I won’t claim they always were.

Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:08PM (#35477551)
Google is where ideas go to die.
Re:Behind the curtain: what it feels like to work in AI righ... by sberens on Thursday April 06, 2023 @09:23PM (#35476896)
> other models are in the ballpark

How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better, especially around code generation / understanding.

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @10:42PM (#35477374)
People really don't seem to understand just how far ahead OAI is in its reasoning abilities (https://crfm.stanford.edu/helm/v0.2.2/?group=reasoning)

Maybe PaLM is near there (it's not evaluated on that page) but nothing else even comes close at all

Re:Behind the curtain: what it feels like to work in AI righ... by icelancer on Thursday April 06, 2023 @11:21PM (#35477648)
The level of denial people are willing to sink into regarding how good GPT-4 is compared to everything else is truly crazy. Not a single other project is an order of magnitude close to the quantitative and qualitative (actual experiential results, not just benchmarks) results that GPT-4 brings.
Re:Behind the curtain: what it feels like to work in AI righ... by satvikpendem on Thursday April 06, 2023 @11:45PM (#35477805)
I've had decent success with Open Assistant, an open source model. I'd say it's within the order of magnitude of ChatGPT, given the prompts I'm looking at, including reasoning prompts. This, I believe, is due to the overwhelmingly clean data that OA have managed to acquire through human volunteers.
Re:Behind the curtain: what it feels like to work in AI righ... by spaceman_2020 on Friday April 07, 2023 @01:43AM (#35478455)
I feel that there’s significant insecurity among a lot of coders about GPT-4. A lot of them are ignoring the pace of improvement and highlighting the few off chances where it gets things wrong.
Re:Behind the curtain: what it feels like to work in AI righ... by AussieWog93 on Friday April 07, 2023 @02:41AM (#35478816)
I've not used GPT-4, so it could be different, but regular old GPT-3.5 gets a _lot_ of things wrong.
Re:Behind the curtain: what it feels like to work in AI righ... by snowe2010 on Friday April 07, 2023 @03:52AM (#35479259)
GPT 4 is quite astounding. It might be wrong on occasion, but it will easily point you in the right direction most of the time. It still messes up, but like a twentieth of what 3.5 did. Honestly it is like an incredible rubber ducky for me. Not only can I just talk like I’m talking to a rubber duck but I can get fast, mostly informed, feedback that unblocks me. If I have a bunch of things competing for my attention I can ask gpt about one of them, a hard one, go do something else while it types out its answer, and then come back later and move on with that project.
Re:Behind the curtain: what it feels like to work in AI righ... by AussieWog93 on Friday April 07, 2023 @04:11AM (#35479402)
Is a 95% reduction in errors an exaggeration or is it really that much better? Might just need to drop the $20/mo if it's really improved that much.
Re:Behind the curtain: what it feels like to work in AI righ... by revelio on Friday April 07, 2023 @06:22AM (#35480083)
I'm a recent convert, been experimenting with converting one PL to another.

GPT-3.5 will get the gist of what the code is doing, and then provide what looks like a direct translation but differs in numerous details whilst having a bunch of other problems.

GPT-4 does a correct translation, almost every time.

It kills me that there's a waiting list for the API. I have put together some tools to integrate 3.5 into my workflow and it helps for my current task a lot (for others it's useless). But to really shine it needs to have API access to 4.

Re:Behind the curtain: what it feels like to work in AI righ... by IanCal on Friday April 07, 2023 @07:00AM (#35480317)
I recently finally got access to 4 in the API, it's good. It's much better imo at following the system prompt too. Faster than you see in chatgpt I think, not as fast as 3.5-turbo but definitely less tedious.
Re:Behind the curtain: what it feels like to work in AI righ... by IanCal on Friday April 07, 2023 @06:58AM (#35480307)
My only kind of quantitative answer is that I had 3.5 creating ember templates & data and it would get the templates mostly ok after a couple of iterations of fixing errors, almost never on the first shot (if ever). Often wouldn't quite get it in two, and data structures would often be kinda there but not quite. Required a lot more care with the prompts. 4 gave me working things every time first time (except only where it did things that were fine in a template but not currently supported by the custom framework I'm using them in), and didn't need as much hand holding.

Qualitatively, it's wildly different from gpt-3.5-turbo for discussions. 3.5 feels a little formulaic after a while with some kinds of questions. 4 is much more like talking to an intelligent person. It's not perfect, but I'm flipping between discussing a sporting thing, then medical malpractice, legal issues, technical specifications and it's doing extremely well.

If it's affordable for you, I'd really recommend trying it.

Re:Behind the curtain: what it feels like to work in AI righ... by icelancer on Saturday April 08, 2023 @04:21AM (#35491571)
For code 90%+ reduction is easily correct. For text and other content, I can't say, but I would guess it's not that good.

Anything involving reasoning, code, complex logic, GPT-4 is a breakthrough. GPT-3.5 turbo is more than good enough for poetry and the other text generation stuff.

Re:Behind the curtain: what it feels like to work in AI righ... by int_19h on Friday April 07, 2023 @04:17PM (#35486451)
My favorite part about GPT-4 is that if it generates code that is wrong, and you ask it to verify what it just wrote - without telling it whether it's wrong or not, much less pointing out the specific issue - more often than not it will spot the problem and fix it right away.

And yes, it does indeed make an amazing rubber duck for brainstorming.

Re:Behind the curtain: what it feels like to work in AI righ... by spaceman_2020 on Friday April 07, 2023 @06:12AM (#35480033)
GPT-4 is leaps ahead, and it's improving with every new release. The latest March 23 release is significantly better than the previous one and does a LOT of heavy lifting for my code at least.

At the very least, it's a massive productivity booster.

Re:Behind the curtain: what it feels like to work in AI righ... by sealeck on Friday April 07, 2023 @08:17PM (#35488939)
I think there's a lot of people writing boilerplate programs who are going to be freed from these menial tasks (i.e. no more Java enterprise application development, thankfully).
Re:Behind the curtain: what it feels like to work in AI righ... by JumpCrisscross on Friday April 07, 2023 @12:33AM (#35478060)
> How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better

I have at most moderate confidence in this hypothesis.

Re:Behind the curtain: what it feels like to work in AI righ... by spaceman_2020 on Friday April 07, 2023 @01:41AM (#35478441)
How much does reinforced learning from people using chatGPT help accelerate OpenAI’s advantage?
Re:Behind the curtain: what it feels like to work in AI righ... by black3r on Thursday April 06, 2023 @05:10PM (#35474519)
For a little more than a year I worked in an AI startup doing basically everything other than AI (APIs, webapps, devops...), but from what I've seen there the "secret sauce" to AI success is the training process (dataset, parameters, fine-tuning steps, ...). And OpenAI isn't open about theirs since their beginnings.
Re:Behind the curtain: what it feels like to work in AI righ... by logicallee on Thursday April 06, 2023 @05:10PM (#35474521)
All of the neural network architecture for human level thinking and processing, including vision, speech, emotion, abstract thought, balance and fine motor skills, everything was publicly released in April 2003, twenty years ago this month. It's a 700 megabyte tarball and sets up an 80b parameter neural network.

What? Huh? Yes the human genome encodes all human level thought.[1] Clearly it does because the only difference between humans that have abstract thought as well as language capabilities and primates that don't is slightly different DNA.

In other words: those slight differences matter.

To anyone who has used GPT since ChatGPT's public release in November and who pays to use GPT 4 now, it is clear that GPT 4 is a lot smarter than 3 was.

However, to the select few who see an ocean in a drop of water, the November release already showed glimmers of abstract thought, many other people dismiss it as an illusion.

To a select few, it is apparent that OpenAI have found the magic parameters. Everything after that is just fine tuning.

Is it any surprise that without OpenAI releasing their weights, models, or training data, Google can't just come up with its own? Why should they when without turning it into weights and models, the human neural network architecture itself is still unmatched (even by OpenAI) despite being digitized twenty years ago?

No, it's no surprise. OpenAI performed what amounts to a miracle, ten years ahead of schedule, and didn't tell anyone how they did it.

If you work for another company, such as Google, don't be surprised that you are ten years behind. After all, the magic formula had been gathering dust on a CD-ROM for 20 years (human DNA which encodes the human neural network architecture), and nobody made the slightest tangible progress toward it until OpenAI brute forced a solution using $1 billion of Azure GPU's that Microsoft poured into OpenAI in 2019.

Is your team using $1 billion of GPU's for 3 years? If not, don't expect to catch up with OpenAI's November miracle.

p.s. two months after the November miracle, Microsoft closed a $10 billion follow-on investment in OpenAI.

[1] https://en.m.wikipedia.org/wiki/Human_Genome_Project

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @07:01PM (#35475700)
They have been collecting human feedback data for 2 years + probably have a lot of data from Copilot + are training with large context models + have invested a ridiculous amount in curating pretraining data -- the kind of stuff that won't get you a ton of pubs (so you won't see Google researchers having focused on it a lot) but apparently turns out to be super important for a good LLM
Re:Behind the curtain: what it feels like to work in AI righ... by sangnoir on Thursday April 06, 2023 @09:40PM (#35477016)
> Do you think OpenAI has discovered something they haven't been open about?

They have not, which makes me curious about which company gp works for because the "F" and "G" in FAANG are publicly known to already have LLMs. Not sure about Amazon, but I'm guessing they do too.

As an outsider, the amazing thing about ML/AI research is that you get a revolutionary discovery of s technique that changes everything, and a few months later another seminal paper is published. My bet is ChatGPT is not the last word in AI, and OpenAI will not have a monopoly on discoveries that improve the state of the art. They will have to contend with the fact that Google, Meta & Amazon own their own datacenters and can likely train models for cheaper[1] than what Microsoft is paying itself via their investment in OpenAI.

1. OpenAI may get a discount, but my prediction when they burn through Microsoft, they'll wnd up being "owned" by Microsoft for all intents and purposes.

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @10:45PM (#35477399)
Issue is the data moat OAI is building. They'll have hundreds of millions of high quality user interactions with ChatGPT they can use to finetune their models. What will anyone else including Google have?
Re:Behind the curtain: what it feels like to work in AI righ... by Teever on Friday April 07, 2023 @12:17AM (#35477984)
As people make chat bots with openAI and tie them into existing chat services, organizations that offer these chat services will get their hands on that kind of data too.

Discord comes to mind.

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Friday April 07, 2023 @02:12AM (#35478636)
That's possible ya. Hope it happens
Re:Behind the curtain: what it feels like to work in AI righ... by nmfisher on Friday April 07, 2023 @12:20AM (#35477996)
I'm not sure how deep that moat is. As soon as you open up the API, anyone can distil ChatGPT (or at least, some smaller part of it) by fine-tuning another model on its outputs[0].

I'm guessing that this is the #1 fear for people inside OpenAI have right now.

[0] For the record, I have zero problem with this.

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Friday April 07, 2023 @02:01AM (#35478548)
1. ToS make it hard for a commercial entity to do so. So some third parties would have to collect the data first

2. You won't be able to get the hundreds of millions or more interactions that OAI will have (both due to cost of API as well as it being not easy to figure out a good way to generate that many queries for a good multiturn conversaton). Maybe you can make up for it by querying smartly. We don't know if we can right now.

Re:Behind the curtain: what it feels like to work in AI righ... by sangnoir on Friday April 07, 2023 @06:04PM (#35487594)
> What will anyone else including Google have?

Google has been collecting user interactions since 2007 via GOOG-411, which was a precursor to the Google Assistant - I suspect Google has billions of user interactions on hand through the latter. Facebook has posts and comment, Amazon has products pages, reviews and product Q&As and all of them have billions of dollars to draw upon if they choose to buy high-quality data, or spin-up / increase teams that create and/or categorize training data.

They also have deep roster of AI researchers[1] to potentially obsolete LLMs or make fine-tuning work without access to of ChatGPT records.

1. I suspect Google alone has more AI researchers that OpenAI has employees

Re:Behind the curtain: what it feels like to work in AI righ... by andsoitis on Thursday April 06, 2023 @09:41PM (#35477018)
Having a model does not a platform or a product make. You also need users and mindshare.

OpenAI is enjoying first mover advantage around the platformication and product-ification of LLMs.

Re:Behind the curtain: what it feels like to work in AI righ... by hackerlight on Thursday April 06, 2023 @10:39PM (#35477355)
It probably is the secret sauce which remains undisclosed. Differences that seem small can lead to large differences in model quality.
Re:Behind the curtain: what it feels like to work in AI righ... by majormajor on Thursday April 06, 2023 @10:45PM (#35477398)
A lot of FAANG data folks aren't on the teams that were doing research into this stuff and weren't using the latest fruits of that research.

OpenAI has released a ton more easy-to-use-for-everyone stuff that has really leapfrogged what a lot of "applied" folks everywhere else were trying to build themselves, despite being on-the-face-of-it more "general."

Re:Behind the curtain: what it feels like to work in AI righ... by nr2x on Thursday April 06, 2023 @11:11PM (#35477575)
Exactly!
Re:Behind the curtain: what it feels like to work in AI righ... by rqtwteye on Thursday April 06, 2023 @10:50PM (#35477428)
I think it’s the way things go usually. The big players have a business to run so they can’t focus much on innovation. OpenAI has the only purpose right now to push AI and nothing else. Once they have a real business they will also slow down.
Re:Behind the curtain: what it feels like to work in AI righ... by lacker on Thursday April 06, 2023 @04:36PM (#35474030)
This reminds me of back in the mid 2000's, there were a lot of smart people working on search algorithms at different companies. But eventually, you'd talk to someone smart working on Yahoo Search, and they would just be kind of beaten down by the frustration of working on a competing search engine when Google was considered to be by far the best. It got harder for them to recruit, and eventually they just gave up.

So... I don't know where you're working. But don't twiddle your thumbs for too long! It's no fun to be in the last half of people to leave the sinking ship.

Re:Behind the curtain: what it feels like to work in AI righ... by anon7725 on Thursday April 06, 2023 @04:36PM (#35474035)
Can confirm. People are scrambling to remain relevant.
Re:Behind the curtain: what it feels like to work in AI righ... by TechnicolorByte on Thursday April 06, 2023 @07:42PM (#35476089)
How does that manifest specifically?
Re:Behind the curtain: what it feels like to work in AI righ... by ProllyInfamous on Friday April 07, 2023 @06:47PM (#35488066)
Go read the Goldman Sachs report from last week, which predicts 300m jobs disappearing and 60% of jobs "affected beyond minimally."
Re:Behind the curtain: what it feels like to work in AI righ... by BulgarianIdiot on Thursday April 06, 2023 @04:37PM (#35474046)
Calling an API doesn't mean no value is captured. There are vastly complex integrations of LLM as a small component in larger systems, with their own programming, memory, task models and so on.

If you think GPT is just about chat, you've misunderstood LLMs.

Re:Behind the curtain: what it feels like to work in AI righ... by xnx on Thursday April 06, 2023 @04:49PM (#35474204)
Not sure why LLM would make Facebook (ads), Apple (hardware), Amazon (hosting, retail), Netflix (tv) obsolete. It's definitely something Google needs to think about, but there's no reason to think they won't again be the leader soon.
Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @05:19PM (#35474634)
Imagine Walmart launching a ChatGPT interfaced bot for shopping that customers take a liking to. Walmart starts acquiring both new customers as well as high quality data they can use for RLHF for shopping. Eventually Walmart's data moat becomes so big, that Amazon retail cannot catch up and customers start leaving Amazon.

For AWS, if MS starts giving discounts for OAI model usage to regular Azure customers, that's gonna be a strong incentive to switch

For Apple, A Windows integrated with GPT tech may become a tough beast to beat.

Re:Behind the curtain: what it feels like to work in AI righ... by twelve40 on Thursday April 06, 2023 @08:57PM (#35476720)
> Imagine Walmart launching a ChatGPT interfaced bot for shopping that customers take a liking to.

I can't imagine that, because it doesn't seem to fit the use case. Especially not to the point of bankruptcy of Amazon, maybe as a small novelty? Can you list some killer features that the chat would bring that would make the existing shopping experience irrelevant? Maybe not everything is a nail to the hammer?

Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @10:36PM (#35477337)
"I'm looking for X. Can you give me a few options with the benefits and drawbacks of each"

or

"I'm looking for X. Can you ask me a few questions and give me a choice of products based on that"

or

"I'm looking for a product that does X, Y, Z. Can you find such a product for me"

or

"Does this product do Y/is compatible with Z/is an appropriate give for person P"

I'm not saying this will happen. I'm saying it's a risk that Amazon should take seriously.

Re:Behind the curtain: what it feels like to work in AI righ... by twelve40 on Thursday April 06, 2023 @11:18PM (#35477630)
sure, this is basically a fancy version of the current FAQ's, customer's QnA and current recommendations (similar products, recommended products). Would it be just so much better in experience that it kills Amazon? i kind of doubt it. Also, we're currently talking model that was trained on pre-2021 data and here we have an inventory in the millions that changes daily, so the tech has to catch up, too.
Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Friday April 07, 2023 @02:09AM (#35478612)
Asking for your preferences to figure out your product choices or searching for a product for you based on your given requirements is a lot more than just FAQs and Q&As. You can even imagine a fancier version where you describe say what kind of setup you want and it gives you combinations that are nice.

Also this could aggregate information not just on the product page but across multiple pages which is time consuming to do by oneself.

As for 2021 cutoff -- ChatGPT can now browse the internet and if Walmart built a bot that interfaces with ChatGPT, I'm sure they would be feeding it the latest info

Re:Behind the curtain: what it feels like to work in AI righ... by lostdog on Friday April 07, 2023 @02:45AM (#35478838)
Everybody hates salespeople, and salespeople optimize for their own company's goals, just like an AI chatbot will have to.

There's no way you will want to interact more with Walmart's chatbot than necessary.

Re:Behind the curtain: what it feels like to work in AI righ... by iudqnolq on Friday April 07, 2023 @12:39PM (#35483680)
Why would you want an approximation of the average person on the internet to give you that advice?

When you're trying to answer that question now you'd presumably use heavy filters: wirecutter, trusted blogs, top reddit comments, etc. GPT won't.

Re:Behind the curtain: what it feels like to work in AI righ... by atonse on Thursday April 06, 2023 @04:54PM (#35474278)
I actually think Apple is in a unique position here again with the hardware/software integration.

Once again, their ability to do computation on device and optimize silicon to do it, is unparalleled.

A huge Achilles heel of current models like GPT-4 is that they can’t be run locally. And there are tons of use cases where we don’t necessarily want to share what we’re doing with OpenAI.

That’s why if Apple wasn’t so behind on the actual models (Siri is still a joke a decade later), they’d be in great shape hardware-wise.

Re:Behind the curtain: what it feels like to work in AI righ... by xnx on Thursday April 06, 2023 @05:18PM (#35474614)
Google has some impressive on device AI software such as Google Translate (translation), Google Photos (object detection, removal, inpainting), and Recorder (multi-speaker speech to text). Most of this is possible without their Tensor chip, but is more efficient with it.
Re:Behind the curtain: what it feels like to work in AI righ... by tayo42 on Thursday April 06, 2023 @05:22PM (#35474671)
Sounds like the same thing that happened with datacenters? No one has ops or hardware sysadmins, no one sets up large networks except a few in those centralized cloud companies and couple other niche uses. website ops job changed
Re:Behind the curtain: what it feels like to work in AI righ... by 2-718-281-828 on Thursday April 06, 2023 @08:17PM (#35476395)
come on, it's not that bad, at least you're doing linear regression
Re:Behind the curtain: what it feels like to work in AI righ... by VirusNewbie on Thursday April 06, 2023 @10:17PM (#35477225)
I work at a FAANG and out unreleased models are fantastic
Re:Behind the curtain: what it feels like to work in AI righ... by letitgo12345 on Thursday April 06, 2023 @10:40PM (#35477367)
Hope so. Don't want a single corporate entity (OAI/MS) dominating the entire economy. This sector desperately needs competition
Re:Behind the curtain: what it feels like to work in AI righ... by Robotbeat on Thursday April 06, 2023 @10:39PM (#35477356)
Not releasing the models may be the same as the models never existing in the end.
Re:Behind the curtain: what it feels like to work in AI righ... by philjohn on Friday April 07, 2023 @04:25AM (#35479483)
It depends - a great model tuned to your business domain could be insanely valuable as a competetive advantage.
Re:Behind the curtain: what it feels like to work in AI righ... by carabiner on Thursday April 06, 2023 @10:39PM (#35477357)
Not just NLP, even 3D art: https://www.youtube.com/watch?v=SzGEfYh9ITQ

Top comment: I love seeing my job get transformed from 3D artist into prompt writer into jobless in a year or less, yay!

Re:Behind the curtain: what it feels like to work in AI righ... by Washuu on Friday April 07, 2023 @01:20AM (#35478308)
I do a lot of stylized 3D art: I still have time before AI figures that out!~
Re:Behind the curtain: what it feels like to work in AI righ... by zone411 on Thursday April 06, 2023 @11:02PM (#35477510)
I'm quite surprised at how little progress FAANG companies have made in recent years, as I believe much of what's happening now with ChatGPT was predictable. Here's a slide from a deck for a product I was developing in 2017: https://twitter.com/LechMazur/status/1644093407357202434/pho....
Re:Behind the curtain: what it feels like to work in AI righ... by Analog24 on Friday April 07, 2023 @01:18AM (#35478299)
Scaling up an LM from 2017 would not achieve what GPT-4 does. It's nowhere near that simple. Of course companies saw the potential of natural language interfaces, there has been billions spent on it over the years and a lot of progress was made prior to ChatGPT coming along.
Re:Behind the curtain: what it feels like to work in AI righ... by zone411 on Friday April 07, 2023 @05:38AM (#35479862)
You're making incorrect assumptions. This project wasn't about scaling any existing approaches. It was original neural net research that produced excellent results without self-attention, using a new optimizer, new ideas to prevent overfitting, but with some feature engineering, etc. Scaling it up to GPT-2 size matched its performance, and very likely scaling it up to GPT-3 levels would have matched it as well. Unclear for GPT-4 since there is very little known about it. Of course, in the meantime, most of my ideas are no longer SOTA and there has be a ton of progress in GPU hardware and frameworks like PyTorch/TF.

I understand your skepticism, but you can check out my melodies project from a year ago as a current example. There is nothing matching it yet: https://www.youtube.com/playlist?list=PLoCzMRqh5SkFPG0-RIAR8....

What you're saying about these companies recognizing the potential is clearly incorrect. It's six years later and Siri, Alexa, and Google Home are still nearly as dumb as they were back then. Why do you think Google initiated their "code red" if they saw the potential?

Re:Behind the curtain: what it feels like to work in AI righ... by Analog24 on Friday April 07, 2023 @12:43PM (#35483741)
Do you have any publications to back up your claims about your work? They seem more than a bit grandiose. If you're ideas are as novel and useful as you say then you should publish them.

And I'm sorry, but you're completely wrong about companies recognizing commercial potential. I worked on Alexa for five years, it is a far harder problem than you think. It is nowhere near as simple as "we just weren't looking at the right NN architecture or optimizer!" You're acting like it was a novel idea to think LMs would be extremely useful if the performance was better (in 2017). I'm just trying to tell you that isn't the case.

Re:Behind the curtain: what it feels like to work in AI righ... by zone411 on Friday April 07, 2023 @02:46PM (#35485382)
No, I have no plans to openly publish any of it. Some of my researcher employees have published their own stuff. I've previously written about how it was a huge commercial mistake for Google and others to openly publish their research, and they should stop. Indeed, now OpenAI has not published a meaningful GPT-4 paper, and DeepMind has also become more cautious. This mistake has cost them billions, and for what? Recruiting? Now they lost many people to OpenAI and wasted time and effort on publishing. Publishing is fine for university researchers or those looking to get hired. I did record some research ideas in an encrypted format on Twitter for posterity: https://twitter.com/LechMazur/status/1534481734598811650.

If any of the FAANG companies recognized the commercial potential and still accomplished so little, they must be entirely incompetent. When this 2017 deck was created, I had 50k LOC (fewer would be needed now using the frameworks and libraries) plus Word and Chrome plugins. The inference was still too slow and not quite feature-complete, and it was just a writing assistant with several other features in early testing, but it seems more than enough for me to know quite well how difficult is the task.

Re:Behind the curtain: what it feels like to work in AI righ... by Analog24 on Friday April 07, 2023 @03:13PM (#35485772)
The fact that you think creating a writing assistant plugged into Word is equivalent to building general purpose, always-on voice assistant tells me all I need to know.
Re:Behind the curtain: what it feels like to work in AI righ... by zone411 on Friday April 07, 2023 @09:44PM (#35489579)
What? We were talking about making a language model. I mentioned the plugins in relation to the question of commercializing. I'm very clear about what my project was and was not doing. I get that you're bitter because Alexa became a joke with how little progress was made and the struggles of Amazon in getting the top talent are well known. How much did it cost Amazon to fail like this? Is the whole team gone? Is that the billions you mentioned?
Re:Behind the curtain: what it feels like to work in AI righ... by Analog24 on Friday April 07, 2023 @11:05PM (#35490078)
"It's six years later and Siri, Alexa, and Google Home are still nearly as dumb as they were back then". You can't even keep a coherent discussion and you are delusional about the significance of your work. You shared a slide with nothing but generic pie-in-the-sky use-cases and you act like it gives you some credibility on the subject ("let's make an AI system that can do the work of your non-professional employees!"). And to top it off you act like you've been successful here! Again, you shared nothing but a slide with generic use-cases that a 12 year old could think up. I don't know what you think you proved. Enjoy your imaginary pedestal.
Re:Behind the curtain: what it feels like to work in AI righ... by qqtt on Thursday April 06, 2023 @11:04PM (#35477523)
ChatGPT is cool and novel, but FAANG's requirements for ML/AI go far beyond what ChatGPT provides as a product. ChatGPT is good at answering questions based on an older data set. FAANG typically requires up to date real time inference for huge rapidly changing data sets.

Working on the practical side of ML/AI at FAANG, you will probably be working with some combination of feature stores, training platforms, inference engines, and so on - all attempting to optimize inference and models for specific use cases - largely ranking - which ads to show which customers based on feature store attributes, which shows to show which customers - all these ranking problems exist orthogonal to ChatGPT, which is using relatively stale datasets to answer knowledge based questions.

The scaling problems for AI/ML for productionizing these ranking models from training to inference is a huge scaling problem. ChatGPT hasn't really come close to solving it in a general way (and also solves a different class of problems).

Re:Behind the curtain: what it feels like to work in AI righ... by yanderekko on Thursday April 06, 2023 @11:27PM (#35477688)
Agreed. For my job maintaining real-time models with high business value to be disrupted by a chatbot, an LLM would have to be able to plug into our entire data ecosystem and yield insights in realtime. The backend engineering work required to facilitate this will be immense, and if the answer to that is "an LLM will create a new backend data architecture required to support the front-end prompt systems", then... well, suffice to say I can't see that happening overnight. It will require several major iterative and unpredictable pivots to re-envisage what exactly engineers are doing at the company.

For the time being, I expect LLMs to start creeping their tendrils into various workflows where the underlying engineering work is light but the rate of this will be limited by the slow adaptability of the humans that are not yet completely disposable. The "low hanging fruit" is obvious, but EVPs who are asking "why can't we just replace our whole web experience with a chatbot interface?" may end up causing weird overcorrections among their subordinates.

Re:Behind the curtain: what it feels like to work in AI righ... by og_kalu on Friday April 07, 2023 @06:42PM (#35488012)
Isn't this as straightforward as semantic search over an embedded corpus ? Unless i'm missing something, i don't think the backend engineering would take much
Re:Behind the curtain: what it feels like to work in AI righ... by yanderekko on Friday April 07, 2023 @09:37PM (#35489527)
I think generating useful embeddings off of a lot of realtime data flows (eg. user clickstream data) is in fact fairly difficult. Furthermore, if you had such embeddings it's unclear if an LLM would add value to whatever inference you're trying to do. If the LLM is not only be used for inference but to actually retrieve data ("find and summarize the clickstream history of his user") then I would not expect this to be doable in realtime.
Re:Behind the curtain: what it feels like to work in AI righ... by fdgsdfogijq on Thursday April 06, 2023 @11:27PM (#35477692)
I can tell you that we have applied teams working on open problems, which can be solved out of the box with ChatGPT. Its a huge deal
Re:Behind the curtain: what it feels like to work in AI righ... by nerdponx on Friday April 07, 2023 @11:44AM (#35482878)
What's an example of such a problem?
Re:Behind the curtain: what it feels like to work in AI righ... by uxcolumbo on Saturday April 08, 2023 @03:50AM (#35491452)
What kind of problems?
Re:Behind the curtain: what it feels like to work in AI righ... by alfor on Friday April 07, 2023 @12:27AM (#35478033)
ChatGPT is human level intelligence, it’s not just novel and cool, it’s the thing. Remember, GPT-4 training was finished 6 months ago. Listen to people at OpenAI, their concern is: disruption to the world, UBI, getting people used to superintelligence as part of our world. I think they have quite a few things in the pipeline.

So yes ads optimisation/recommendations still need to be reliable for the time being, but for how long?

Re:Behind the curtain: what it feels like to work in AI righ... by mach1ne on Friday April 07, 2023 @01:30AM (#35478371)
GPT-4 is not human level intelligence, nor is is above or below. It’s quite a different kind of intelligence not entirely comparable to humans. That’s probably why we’re moving the AGI goalpost; we visualize AGI as a robot human, but these machines may simply be founded on too different principles to ever receive that honor.
Re:Behind the curtain: what it feels like to work in AI righ... by alfor on Friday April 07, 2023 @07:02AM (#35480330)
I think it’s mostly different because they crippled the public version for now: no internet access, everything is done in one pass. In our mind we get an idea, we inspect it, we try different variations, we simulate how it will be perceive(consciousness). In this way we iterate before putting the information out. This is not difficult and is getting added on to it externally.

Chat GPT is just to get us used to the idea, it’s the toy version.

Re:Behind the curtain: what it feels like to work in AI righ... by nerdponx on Friday April 07, 2023 @11:43AM (#35482854)
This is pure speculation, in a very implausible direction.
Re:Behind the curtain: what it feels like to work in AI righ... by jazzyjackson on Friday April 07, 2023 @01:40PM (#35484471)
I would be interested to know which part you feel is implausible, to me it seems inevitable

You have a language model produce an outline with steps and then recursively set agents to consume and iterate on a task until another language model finds the results satisfies the specification.

This includes interactions with the real world (via instructions executed over an API) and using the success of those interactions for reinforcement learning on the model.

Re:Behind the curtain: what it feels like to work in AI righ... by alfor on Friday April 07, 2023 @09:09PM (#35489325)
Many are building such things.

But I think they are mostly pointless as OpenAI is so far ahead of everyone external it’s not even funny. Most externals things with the API will be obsolete in a few months.

They had GPT4 6 months ago or more! They have access to the full model without crippling. They (for sure) have larger, more powerful models that are not cost effective/safe to release to the public.

Now they have a new data flyweel with people asking millions of questions daily.

Put your speculation hat on and listen attentively to the interviews of Sam Altman and Ilya Sutskever.

You will see were their minds go: UBI, safety, world disruption, etc.

Re:Behind the curtain: what it feels like to work in AI righ... by blazespin on Friday April 07, 2023 @01:01AM (#35478199)
Folks need to start getting over themselves. It's pretty trivial to get GPT4 to explain how transformers work, where the bottlenecks are in both performance and learning, and start modifying pytorch.

It's really not that complicated. Gatekeeping is so over.

Re:Behind the curtain: what it feels like to work in AI righ... by rlt on Friday April 07, 2023 @05:43AM (#35479888)
> Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

I would call that “applied AI” and there’s no shame in figuring out novel ways to apply a new technology.