News for hackers. Stuff that matters.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by neilv on Monday February 19, 2024 @08:35PM (#39437022)
Is this the kind of thing employees had in mind when they backed Altman?
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by ori_b on Monday February 19, 2024 @08:54PM (#39437147)
Yes, at least if they joined in 2020 or later. I interviewed and got an offer around that time, post for-profit conversion, and the pitch was that the "shares" (actually, profit sharing rights) would be bought by investors every few years.

(FWIW, I ended up not taking the position, mainly because I would not have enjoyed the specific work they wanted to hire me for.)

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by smy20011 on Monday February 19, 2024 @09:19PM (#39437312)
It must be a big offer
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by samspenc on Monday February 19, 2024 @09:49PM (#39437490)
I was going to say, question for GP, do you regret not taking the offer? Do you know how much the shares would have been worth today if you had accepted the offer?
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by FreakLegion on Monday February 19, 2024 @10:26PM (#39437728)
Do you regret not plowing all of your money into DOGE in 2020? In the scheme of the things you could do to make money with hindsight, accepting a job offer barely ranks.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by Mistletoe on Tuesday February 20, 2024 @12:40AM (#39438382)
Absolutely I do. And I could have sold when my Aunt told me how the teachers had a Doge pool at work and the guy at the electrical counter was looking at his Doge and telling me about it. There are smart ways to invest in the silliest of endeavors.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by sangnoir on Tuesday February 20, 2024 @01:25AM (#39438556)
Do you also regret not playing the winning lottery numbers, now that you know what they are (or can easily look them up)?
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by biztos on Tuesday February 20, 2024 @02:21AM (#39438794)
Every time my girlfriend buys lottery tickets I tell her to buy the winning ones, and every time she gets the numbers wrong. Every. Friggin. Time.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by wifipunk on Tuesday February 20, 2024 @05:18AM (#39439612)
You should try telling her to pick the wrong numbers. She’ll either be successful or rich!
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by grapescheesee on Tuesday February 20, 2024 @06:40AM (#39440130)
Write out a list with all the possible numbers, and her get one of each before buying more. That way you can have better odds. The real issue will be how long it takes to buy them all.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mewpmewp2 on Tuesday February 20, 2024 @04:47AM (#39439451)
Absolutely, now that you mention it. In addition every day there are millions of option call/put plays which when I found out did well I will have absolute regrets.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by master-lincoln on Tuesday February 20, 2024 @05:16AM (#39439599)
I pity you as you are so full of regrets. Try to enjoy life without looking back so much
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mewpmewp2 on Tuesday February 20, 2024 @03:35PM (#39446549)
Now I regret having lived a life full of regrets.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by seattle_spring on Monday February 19, 2024 @10:56PM (#39437902)
The chances of making some money from stock in a Silicon Valley darling is not really comparable to having plowed money into a meme coin that was literally created as a joke.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by laserlight on Tuesday February 20, 2024 @01:35AM (#39438596)
You cannot take as many Silicon Valley stock bets (as an employee) as shitcoin bets. Therefore, making money from the latter is easier.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by j0hnyl on Tuesday February 20, 2024 @08:56AM (#39441289)
I actually agree with you. I think it's easier to make money on shitcoins than on on employer stock.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by abeyer on Tuesday February 20, 2024 @02:04AM (#39438711)
Not comparable at all... the latter knows and admits it's a joke/money-grab and is not trying to pretend it's actually doing something good for the world.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by OJFord on Tuesday February 20, 2024 @05:23AM (#39439635)
Unless you were weighing DOGE vs FTC or something though and went with the latter, it's different? It's not just Oh if I knew then what I know now I could've made so much money; it's I almost made this decision, or could have, but explicitly decided against it.

I had a badly timed offer of something (I don't really remember might just have been an interview, or contracting work, on the back of some OSS contribution) at Cloudflare pre-IPO, and I do occasionally regret/wonder what-if, because it's not just a random hypothetical, it was an actual opportunity I made a decision against. (Doesn't help that it's still a company on my interested-in list, much like the trap in investing of liking a stock but wishing you'd bought it lower when you first thought of it, and so continuing to not buy as it climbs (or not even investing necessarily but just being a consumer in an inflationary environment).)

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by ori_b on Tuesday February 20, 2024 @02:52PM (#39446056)
No. There are plenty of well paying jobs that actually match my skills. I probably would have made a bit more at OpenAI, but at the cost of not enjoying my work, and probably not excelling at it.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by smackeyacky on Monday February 19, 2024 @08:49PM (#39437119)
In the last round of over-hyped pump and dump companies the usual thing was to list the company so the shares could be openly traded.

This seems very much like a "get out now" signal to everybody involved. Greater fool farming on an accelerated timeline.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by nl on Monday February 19, 2024 @11:33PM (#39438085)
> This seems very much like a "get out now" signal to everybody involved. Greater fool farming on an accelerated timeline.

If you were an early FB employee trying to diversify your wealth base in 2011[1] it turned out to have been a bad decision in retrospect but made sense then.

It's easy to say in retrospect "greater fool" vs "bad decision" but right now it is reasonable to diversify risk and doesn't imply any bad faith.

[1] https://www.theguardian.com/technology/2011/feb/10/facebook-...

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by hiddencost on Tuesday February 20, 2024 @12:03AM (#39438242)
Wrong. It was a good, principles decision with a surprising outcome.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by outside415 on Tuesday February 20, 2024 @02:42AM (#39438898)
you can keep your principles, I prefer to have captured more upside.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by rightbyte on Tuesday February 20, 2024 @04:31AM (#39439381)
A bad bet is still a bad bet if you win, though.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by nl on Tuesday February 20, 2024 @07:22AM (#39440392)
The point of it being a good decision was that it wasn’t a bet: they were locking in a known as opposed to it being unknown. I guess you can say choosing not to play is a bet of some kind but it is a different class of bet I think.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by rightbyte on Tuesday February 20, 2024 @07:43AM (#39440534)
Sure. If you had better insight or wisdom than the market in general, ye, you can claim it was not a (bad) bet. Still probably bad to go essetially all in on one stock though, even if you are 90% sure?

Personally, I thought that Facebook had peaked around IPO and would slowly go irrelevant.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by outside415 on Tuesday February 20, 2024 @02:28PM (#39445720)
I’m still holding all of my Facebook stock from around IPO time when I worked there.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by outside415 on Tuesday February 20, 2024 @06:06PM (#39448156)
what constitutes a bad bet is largely in the eye of the beholder unless there are clear parameters to help define an EV. Also different investors will have different tolerances for risk. I have extremely high tolerance for risk and have been rewarded for that tolerance with outsized gains in the market over the years that have dwarfed all of my jobs combined in income except for my most recent stint at NVIDIA.

a financial advisor would always tell you to unload all shares at each vest since you are holding more shares in the form of future vestings.they would then tell you to diversify it into ETFs in a mean variance optimized portfolio tailored to your risk appetite using standard deviation of historical returns and correlation of underlying basket of goods. this is a solid approach that produces predictable safe returns.

I would rather let it ride since I have insider information at most companies I work for and have a better understanding of future returns on that stock that the average person on the street.

I say, take average advice, receive average returns.

:)

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by nl on Tuesday February 20, 2024 @07:30AM (#39440448)
Fair, although still doesn’t imply it was greater fools buying. They just had different risk profiles.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by dilyevsky on Monday February 19, 2024 @09:00PM (#39437186)
Relax, this is likely just their retention strategy to keep employees that otherwise may jump ship and go work for meta/apple/google (who are publicly traded)
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by jasonjmcghee on Monday February 19, 2024 @10:52PM (#39437875)
A tender offer is not a "get out now" signal. It's often a "the company is healthy" signal.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by rvz on Tuesday February 20, 2024 @02:09AM (#39438734)
> It's often a "the company is healthy" signal.

The same was said about Stripe. Imagine buying Stripe shares at a $96BN valuation when it is now down more than 50% from the peak.

I would indeed sell some if I saw that extreme valuation of any private startup.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by nl on Tuesday February 20, 2024 @07:34AM (#39440470)
Actually it’s back to $70B on secondary markets: https://finance.yahoo.com/news/stripe-popping-off-secondary-...

Secondary markets always jump around because of lower liquidity. I’d wait until after IPO before making any conclusions there.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by lumost on Monday February 19, 2024 @09:49PM (#39437495)
It depends on ones trust in public markets. If a large firm is only held by a few parties, then the public markets provide little benefit to the firm or stakeholders as any significant share movement would require a private deal anyway.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by azemetre on Monday February 19, 2024 @08:56PM (#39437163)
You're making it seem like OpenAI is a dumpster fire and not literally the only company that is MILES ahead of its competition. Not too mention they're supposedly a billion in yearly revenue.

Not even Google can compete.

It could also be a signal that the company recognizes that they would not be where they currently are without their workers. Maybe they're giving a reason for their workers to stay (become multi-millionaires overnight) and not go to their competitors.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by _heimdall on Monday February 19, 2024 @09:47PM (#39437479)
> literally the only company that is MILES ahead of its competition

Its worth noting that, at least when working in a truly novel and untested industry, whoever is leading the pack would likely be the first to know when they tech hits a dead end.

By no means am I saying that's actually the case, but there is still a real possibility that LLMs and the underlying architecture don't pan out with regards to the company's goal of developing anything resembling an AGI. If there is a core challenge with the architecture that doesn't scale, OpenAI would just run into the wall when everyone else was still MILES back.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by kouru225 on Monday February 19, 2024 @09:59PM (#39437553)
It doesn’t need to pan out as a way to create AGI though. It just needs to be useful and profitable.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by _heimdall on Monday February 19, 2024 @10:03PM (#39437576)
At least from what I've seen and how I've seen others use LLMs, the general consensus seems to be that they're useful for the basics today but are more of a promising tech than something that's already landed.

If OpenAI features were to freeze at what we have today I would be surprised if the company stayed around without a major pivot.

Again I'm in no on way saying this is actually the case, only a hypothetical since the tech is still very new and we don't know what we don't know.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by nl on Monday February 19, 2024 @11:56PM (#39438200)
> If OpenAI features were to freeze at what we have today I would be surprised if the company stayed around without a major pivot.

This isn't really true. GPT4 is incredibly useful right now and being used by lots of business processes without any improvements being needed.

Every incremental improvement (eg Gemini 1.5 huge context window) opens up even more possibilities.

AGI isn't required at all.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by lmeyerov on Tuesday February 20, 2024 @01:03AM (#39438473)
Very much this. It's incredible what we can do with it - our limiter building louie.ai has been more ourselves than OpenAI. Though of course a better LLM (smarter, faster, cheaper) does enable more.

The main problem with freezing is the moat disappears. Others are steadily catching up, and on specific benchmarks, even surpassing. Groq.com is insane wrt perf.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by EVa5I7bHFq9mnYK on Tuesday February 20, 2024 @03:05AM (#39439001)
Groq's competition is nvidia, not open.ai OpenAI can run their models on groq machines as well.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by lmeyerov on Tuesday February 20, 2024 @04:43PM (#39447349)
groq has been public on launching an api, so running groq+mixtral vs openai.com (vs others) is a real topic, esp. when ~half of openai's revenue is their api
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by renonce on Tuesday February 20, 2024 @01:03AM (#39438474)
But it’s costly. It would be even more useful if it were 1000 times cheaper than it is now. And we don’t yet know if OpenAI earns more than it spent on renting cards or not
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by nl on Tuesday February 20, 2024 @07:29AM (#39440440)
Sure. Agree with that. But we can be very confident in the price of compute decreasing over time.

(Although we can make inform estimates on OpenAIs cost structure for serving models well enough to guess they are probably breaking even on it, or pretty close to it. Eg listen to the gradient dissent interview with the Together AI founders and it’s clear they are doing the same, and unless you think OpenAI is remarkably worse at serving technology it implies they are in the breaking even ballpark)

And we still don’t need AGI for it to be a great business.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by hn_throwaway_99 on Monday February 19, 2024 @10:12PM (#39437624)
> but there is still a real possibility that LLMs and the underlying architecture don't pan out with regards to the company's goal of developing anything resembling an AGI.

There is, but in one sense, who cares? The technology is already super useful, even if it ends up not being the end all and be all of AGI. That is, there are really only 2 options:

1. LLMs are an important stepping stone on the way to AGI, in which case OpenAI is in a great position as the company with the best LLM.

2. LLMs turn out to be a "local maximum" in the search for AGI. I think that even if that actually is the case, OpenAI is still in a great position with much of the infrastructure, data, expertise, etc. even if they require totally new model approaches.

Also, I wouldn't worry so much about #2 because if we do ever get AGI, our economy would surely collapse, or else it would look completely unrecognizable to our current economic systems. That is, I'm always struck that when people talk about AGI fears they talk about things like Skynet and misinformation, but nearly our entire society is organized around people selling their labor for a price. I don't know what our economy will look like when/if AGI becomes real, but I do know it will change drastically if it means that the vast majority of people won't be able to price their labor more than $0.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by _heimdall on Monday February 19, 2024 @10:48PM (#39437852)
> LLMs are an important stepping stone on the way to AGI, in which case OpenAI is in a great position as the company with the best LLM

We don't actually know this though. Assuming an AGI hasn't yet been developed, we don't know whether LLMs will actually get us there. We know they seem to have more use than previous ML systems, but until we have an AGI we can't say what will get us there.

Further, are we really assuming that developing AGI is either a shared goal or a given regardless of what people would actually want to happen? It sounds like we agree on the fundamental impacts an AGI would have on our current societal structures, do we as a society not get a say in the change? Have we effectively blessed a handful of people working in the private sector to make that decision for everyone? And if so, when do we grapple with the moral questions, like whether an AGI has rights similar to humans, or if unplugging one is murder, etc?

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by hn_throwaway_99 on Monday February 19, 2024 @10:58PM (#39437912)
I think you misunderstood my comment. I gave those two bullet points as mutually exclusive options, where one of them must, pretty much by definition, be true.

That is, you responded "We don't actually know this though. Assuming an AGI hasn't yet been developed, we don't know whether LLMs will actually get us there." Exactly, we don't know if this is true, in which case if it's false my second bullet point "LLMs turn out to be a 'local maximum' in the search for AGI." is the true statement.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by _heimdall on Monday February 19, 2024 @11:15PM (#39438003)
I guess I'm not quite sure how those are the two options. For the second scenario, if LLMs are the local maximum and hit a wall, OpenAI would only be in a good spot if the challenges hit don't invalidate core differentiators of their company.

For example, if GPU-based systems are the limiting factor they wouldn't have the edge. If the problem turns out to be in the human skills and background needed to develop an AGI they similarly wouldn't have the advantage.

Wouldn't there have to be a third scenario, where they walked down a path that doesn't pan out at all and requires a fundamental rethink effectively going back to square one?

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by atq2119 on Tuesday February 20, 2024 @12:48AM (#39438417)
If human skills behind the model are a limiting factor, then OpenAI is in a pretty good position: they have a headstart, and software tends towards "winner takes all" because of the low marginal cost. Look at Google search for a great example of this in a related (perhaps even the same...) market.

Of course, a pretty good position doesn't guarantee winning, but GP didn't claim that.

But yeah, there are probably outcomes in which LLMs are a local maximum and ultimately dead end, and OpenAI will have a hard time holding on because the market turns out more competitive. And somebody might beat them to whatever the next important invention is. We'll see.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by azemetre on Monday February 19, 2024 @10:54PM (#39437888)
What do you mean by tech dead end? Has Oracle reached a tech dead in? Are they still a successful business? What about SalesForce, did they hit their tech brick wall yet? Are they still a successful business?

I mean we all work in an industry where we literally see inferior tech win out all the time, not just in usage but in money made.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by _heimdall on Monday February 19, 2024 @11:02PM (#39437936)
OpenAI's stated goal is to develop an AGI. My point was just that until they get there we'll have no idea whether LLM tech will lead to an AGI.

The company could always pivot as they learn the limitations of LLMs and potentially find other options, though I would argue that their current valuation is extremely high and based largely on the promise of tech they really haven't developed yet. If roadblocks are hit they could be in a tight spot having to live up to such high expectations.

That said, as much as I'd like to see the business goal of developing an AGI crash and burn, I wouldn't bet against them. I just hope we somehow solve the seemingly unsolvable alignment problem first.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Monday February 19, 2024 @09:32PM (#39437403)
They are not leading their competition in terms of technology; they are ahead in a public theater and as the first to market. The claims of "regulate us," "it's almost AGI and it's dangerous," and "let's put a moratorium on larger models" are all just hype, deceiving the public. They utilize the work of scientists and other companies worldwide without acknowledging their contributions. Certainly, they have done an impressive job, and the scientists and engineers working there deserve respect and recognition. However, they are not significantly ahead of current research. They are not even weeks ahead of research. The current landscape is a game of budgets, winning public opinion, and securing investments.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by dotnet00 on Monday February 19, 2024 @09:50PM (#39437501)
When you put it that way it kind of reminds me of FTX, they too were all about leading the way on regulation. IIRC the impetus for the entire thing coming crashing down was because the other big exchange's owner thought it was extremely hypocritical how they talked about inviting regulation while operating just like most other crypto firms, completely outside of decent financial controls.

Of course not to imply that OpenAI is the same as FTX.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by dudeinjapan on Tuesday February 20, 2024 @12:50AM (#39438423)
Sam is more trustworthy than Sam.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by infecto on Monday February 19, 2024 @09:51PM (#39437511)
Yeah I dont see what you’re seeing. Ignoring research, OpenAI is bringing tools to market that nobody else to quite close to yet. Yes, they are getting closer but not as you describe it.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by season2episode3 on Tuesday February 20, 2024 @12:18AM (#39438298)
I would not say that OpenAI is all that far ahead of the competition. I would not count Google out, and I would be willing to bet that OpenAI/Microsoft + Google will be the duopoly of the AI age.

At this point Gemini is as good as GPT4 (and anecdotally I think it's better at many coding assistance tasks, based on my experiments on the LMSys chatbot arena). Sora is getting a lot of press for text-to-video, but Google's Lumiere has already been out for a few weeks and produces pretty good results.

I have no doubt that OpenAI has been cooking things up. GPT-4 is an older model which others are only just catching up to now. They have an A+ team, a giant war chest, and a lot of momentum. But just because they have momentum does not mean that they have a moat.

I'd be willing to wager that the top-of-the-line foundational models are going to converge and become indistinguishable for almost all tasks. Even the open foundation models (e.g. Mixtral) are getting really good. Foundation models are not moats.

The players that have moats are Microsoft (with deep enterprise software & B2B expertise, allowing them to sell AI-powered software & ward off competition from upstarts) and Google (where their decade of investment into custom silicon allows them to train & run inference for cheaper than anyone else, by far).

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by hanselot on Tuesday February 20, 2024 @02:23AM (#39438810)
I don't have a large amount of time to devote to this. But no, google is not out. They just choke slammed ChatGPT with the Gemini announcement.

10m context with that retrieval rate is such a monstrous leap. And to top it off, we got LargeWorldModel in the same week, capable of 1M token context with insane retrieval rate in the open source space. So not only is the open source world currently technically ahead of ChatGPT, so is Google. Which is why they had to announce SORA, because google's model is so far ahead of the competition. That's also why it will probably be ages before we get access to SORA. Now don't get me wrong, the average person can't afford 32 TPU's to run LWM, but we already have quants for it, which is a step towards enabling the average person (that somehow has 24-48gb of VRAM to get a taste of that power).

What is also striking is the fact that the new models are all multimodal as a standard. We not only leapfrogged in context size, but also in modalities. The model seems to only benefit from having more modalities to work with.

I think the statement Bill Gates made claiming that "LLM's have reached a plateau" itself indicates they don't believe they can make more money from training better/larger models. Which indicates that they already did as well as they could with their existing people, and are now "years" behind google. I never thought google could catch up, especially after their infamous "We have no moat" situation. But it seems they actually doubled down and did something about it.

To a lot of people, last Thursday was a very nihilistic day for Local Models, as the goalposts shifted from 128-200k context to 10M tokens with near perfect retrieval. It's literally insanely scary. But luckily we got LWM, and that means we have only been 10xed.

Now the local people will work on figuring out how to bridge the gap, before being leapfrogged again. What is really insane is that, we have had LLAMA2 for over a year now, and nobody else figured out how to get this result from it, despite it being around so long.

I still believe there are modifications to the architecture of MoE that will unlock new powers that we haven't even dreamed of yet.

Sorry, this was supposed to be well thought out, but it turned more into stream of consciousness, and I honestly had no intention of disagreeing with you.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Tuesday February 20, 2024 @04:23PM (#39447102)
> But luckily we got LWM, and that means we have only been 10xed.

If I remember the paper correctly, it was something about a 4M context in there. So not 10x, but 2.5x.

> What is really insane is that, we have had LLAMA2 for over a year now, and nobody else figured out how to get this result from it, despite it being around so long.

This isn't true. For now, the task of extending context to 10M tokens is brute-forced by money (increased HW requirements for training and inference and increased training time are also a financial domain). And for now, there simply is no leapfrogging solution for open source or commercial models, which will decrease the costs by orders of magnitude.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by hn_throwaway_99 on Monday February 19, 2024 @10:20PM (#39437681)
Actually, I felt pk-protect-ai's post was spot on. That is, OpenAI does have, currently, the most impressive interfaces and models. But (as was famously commented on by a Google exec) they don't really have any unique moat. That is, while what OpenAI has built may be extraordinary, there is no "secret sauce" there that prevents others from copying them. And this isn't a knock on OpenAI at all. On the contrary, my understanding is that OpenAI was really the first to take the big risk in scaling their GPT models (i.e. spending the hundreds of millions to train their models before they knew what the outcome would be).
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mupuff1234 on Tuesday February 20, 2024 @02:33AM (#39438849)
> was famously commented on by a Google exec

It was just a rank and file IC.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by infecto on Tuesday February 20, 2024 @07:12AM (#39440321)
People keep saying this but after 1-2 years, nobody has gotten to their level yet.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Tuesday February 20, 2024 @04:41PM (#39447319)
This isn't true. Benchmarks do not define the usefulness of the models. Mixtral is much more useful for me right now than GPT-4. Look at LLaVA, or the new one, and the very impressive LWM (the text-only version is LLaMA2-based with a 512K context, which does not require TPU to run the inference). The fine-tuned LLaMA 34B is much faster than GPT-4 Turbo and less annoying, providing pretty impressive quality of results.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by infecto on Wednesday February 21, 2024 @08:05AM (#39453283)
Who mentioned anything about benchmarks? YMMV, it depends on exact use cases, from my extensive testing using my cases, GPT-4 Turbo still is much easier to direct than any of the others you have mentioned.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by azemetre on Monday February 19, 2024 @10:31PM (#39437757)
Calling it public theater is pretty laughable. The amount you get for $25 a month as a consumer is insane.

I literally have a 24/7 consultant with surface level understand of any topic in human history. This consultant also happens to be an amazing artist for $25 a month that gives me the rights to commercialize any art piece they make for me.

This is extremely hard to beat. This will be extremely hard to beat.

As a business they are succeeding.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by caconym_ on Monday February 19, 2024 @11:38PM (#39438112)
You get a "consultant" whose every word has to be verified against an independent second source, and an """artist""" containing a digest of essentially everything actual human artists have ever uploaded to the web, who's somehow still only able to generate derivative slop utterly undifferentiated from what it's spitting out for everybody else.

LLMs are useful tools, but if they don't lead to something you can actually somewhat rely on to generate correct/consistent/factual results (see e.g. the recent Air Canada chatbot lawsuit) then the hype is a bust. TBD.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mthoms on Tuesday February 20, 2024 @01:14AM (#39438513)
> You get a "consultant" whose every word has to be verified against an independent second source So, no different from YouTube Videos or [insert any website here]? That’s still a ton of value to capture and repackage. Especially when it’s packaged into an interface that humans find much more natural (ie. “conversation”) than the existing offerings.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by caconym_ on Tuesday February 20, 2024 @11:51AM (#39443561)
Hence why I said "LLMs are useful tools". But the current hype pitches them as much, much more than ancillaries to traditional web search, hence why I said "[...] then the hype is a bust. TBD."
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mthoms on Tuesday February 20, 2024 @03:32PM (#39446523)
I just don't believe that LLM's have to reach the impossible bar you set of near-perfect correctness in order to be world changing. Thus, I disagree that any outcome less than that should be considered a "bust".
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by caconym_ on Tuesday February 20, 2024 @04:37PM (#39447275)
I'm not sure what else you can call a scenario in which LLMs, currently hyped as (among other things) near-term replacements for human workers in all sorts of professions and positions, are not eventually able to satisfactorily (and cost-effectively) replace even the most low-skilled, largely script-driven customer service workers.

The bar I'm setting is far from "impossible"; even human children generally won't seamlessly confabulate when you ask them a question they don't know the answer to. Again citing the recent Air Canada case, these models can't even reliably answer simple questions that are definitively and objectively answered in documentation that is presumably made as freely available to them as is technically possible under the limitations of current technology.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mthoms on Saturday February 24, 2024 @01:23PM (#39493850)
>LLMs, currently hyped as (among other things) near-term replacements for human workers in all sorts of professions and positions

No well-informed sources are saying that. If you're saying that mainstream reporting and other non-tech folks are wrong about what the possibilities are then... obviously.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by caconym_ on Friday March 08, 2024 @12:36PM (#39643487)
> No well-informed sources are saying that.

Yes they are, actually! I've talked to people who obsessively read practically every LLM paper that passes through the arxiv, with undeniably deep and broad knowledge of the current state of this tech, who seriously believe it's going to surpass humans within a year or two. That it may already have, in the deep dark top secret labs beneath OpenAI HQ.

However,

> If you're saying that mainstream reporting and other non-tech folks are wrong about what the possibilities are then... obviously.

If it was obvious then why are you still replying to my comments, which have very obviously been specifically addressing the mismatch between hype and reality? If the current approach doesn't scale, the hype will have been a bust! Objectively! That's what I've been talking about this whole time!

"Mainstream reporting and other non-tech folks" is a bit disingenuous, though. The primary drivers of the current unrealistic hype are software vendors and associated clingers-on looking to make a quick buck. They'll say anything, regardless of whether it's true, and as a result of those mostly-falsehoods our public lives will be flooded with awful AI tools that make everything shittier and more difficult. I can't wait!

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by FeepingCreature on Tuesday February 20, 2024 @03:38AM (#39439172)
No company in the history of companies has ever advertised themselves with the concept "we're going to destroy the world", primarily because it doesn't actually work. OpenAI are true believers, they say that sort of thing because that is what they genuinely think.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Tuesday February 20, 2024 @04:21AM (#39439337)
No single neural network currently has the capability to destroy the world, and it will not occur until the architecture for NNs is altered to become self-driving rather than reactive. Even then, it will not be sentient. The deception occurs when the attention shifts from the people operating new powerful technology (tools) to this technology itself.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by FeepingCreature on Tuesday February 20, 2024 @07:12AM (#39440315)
Meanwhile, every AI company ever: "We are very excited about agents. We are working hard on agents. We want to roll out agents as soon as possible. Also, persistent memory and online learning would be nice to get."

Like, are you following the things that OpenAI and Deepmind are saying at all? The things that make current LLMs not a threat, they aim to tear down as soon as they can arrange.

OpenAI just released a video network, and one of their core touted benefits was that you could use it as an action controller!

And, um. Do you really think, when the AIs can take a simple prompt and turn it into a ten thousand step plan that requires dynamic skill acquisition, resourcing and persistence, that generating the prompt will be the one single task that stumps them? When we are at that point - and to be clear, every leading AI organisation is sprinting to reach that point earliest - then the difference between doom and safety will be one sentence: "When you are done with that, generate a new prompt." This is not how a world with a long expected lifespan looks.

edit: To be clear, I'm still not accusing OpenAI of making up the doom stuff. Even though when I phrase it like that it sounds like they're directly working on things that obviously end the world, which seems contradictory, I don't think they see it like that. To be honest, I can't explain why any doomer works at OpenAI, except in the way that people sometimes move towards explosions and gunfire. I think it's just a bug in the human brain. We want to have the danger in sight.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Tuesday February 20, 2024 @04:28PM (#39447157)
The agents and action controllers are the reactive components in the toolchain, aren't they?
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by FeepingCreature on Tuesday February 20, 2024 @04:39PM (#39447296)
"Self-driving" is just "reactive" with an open recursive structure. In principle, a network that processes a prompt, generates a plan, recurses a finite number of times, judges how well it did, generates a training plan to improve, outputs a corresponding follow-up prompt, and then waits for you to press a button before it repeats the whole thing with the follow-up prompt, ad infinitum, is still "reactive" - but nobody would argue that whoever presses the button is performing irreplaceable cognitive labor.

So I just don't think this captures an important distinction at the limit. If a system can generate a good action plan, turning it into an agent is just plumbing.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Tuesday February 20, 2024 @05:01PM (#39447558)
Actually, we can't be certain that humans themselves are not reactive. It is just that their reactions are either built in (self-preservation, reproduction), or based on much broader input (sensory, biochemistry, etc.). The current level of reactivity of the LLMs is very limited by their architectures, though, and as long as these architectures stay in place, you can't expect them to be "self-driven".
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by FeepingCreature on Wednesday February 21, 2024 @12:54AM (#39450601)
I just don't think this is the case. I suspect reactivity in LLMs is mostly limited by training. Human text data is just not suited to the way an AI needs to output data to plan long action chains - justification, not reasoning.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by pk-protect-ai on Wednesday February 21, 2024 @05:29AM (#39452160)
This might be true as well along with what I said.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by chaxor on Tuesday February 20, 2024 @02:07AM (#39438722)
To be fair, they are not miles ahead of everyone completely - they're good at shipping products. Other companies (such as deepmind/Google) have capabilities approximately on par, or sometimes better than OpenAI - they just aren't very good at making products out of their research.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by tomp on Tuesday February 20, 2024 @03:16AM (#39439057)
Noone has released a GPT-4 competitor after a year.

I'd say that's quite a lead.

Maybe this has changed now with new Gemini model, but still that means one year ahead of everyone.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by chaxor on Tuesday February 20, 2024 @04:08AM (#39439287)
Again, that's a webapp product for the public - not state of the art research. These are very different things.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by blackoil on Tuesday February 20, 2024 @07:56AM (#39440642)
Research without ability to create better models and products don't mean much. If the research is so good we should see somethings coming out of it.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by rvz on Tuesday February 20, 2024 @02:13AM (#39438757)
Or OpenAI is just better at marketing / hyping their products and creating the illusion that they are ahead.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mlazos on Tuesday February 20, 2024 @02:55AM (#39438950)
lol what you’re describing is literally the free market. We just don’t know who the greater fool is until OpenAI booms more or busts.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by mnky9800n on Tuesday February 20, 2024 @03:38AM (#39439174)
Don't do it! They are about to raise 7 trillion dollars to build new ai chips! Now with more flavour but same great crunch!
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by JeremyNT on Tuesday February 20, 2024 @09:54AM (#39442008)
I mean, watching from the outside, it's hard to guess why OpenAI would be so valuable.

They were the first to market with their LLM chatbot, but zooming out that forced Google's hand in releasing their own, and there aren't any moats here other than "how much copyrighted data you're willing to train on."

Eventually, OpenAI/MS will be the Bing to Google's, well, Google. It's one thing for early adopters to go and seek out ChatGPT, but Google has so many products already, there's no reason to think that anybody would switch to ChatGPT just for a chatbot if they're already using a Google app for something else.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by chihuahua on Monday February 19, 2024 @08:55PM (#39437153)
Wow, people working for a company with the goal of profiting financially from it? That's really sad.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by speedylight on Monday February 19, 2024 @08:58PM (#39437172)
Well I mean they are supposed to be non profit. Not that I’m complaining, OpenAI does great stuff.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by djbusby on Monday February 19, 2024 @09:06PM (#39437223)
Staff (can/do) get paid at a not for profit
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by motoxpro on Monday February 19, 2024 @09:38PM (#39437437)
Do you have the idea that non-profit means that everyone is a volunteer? Non-profit just means "no profit" not "no pay"
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by baq on Tuesday February 20, 2024 @02:11AM (#39438745)
Non-profit doesn’t mean zero revenue.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by ethanbond on Monday February 19, 2024 @09:05PM (#39437216)
Nothin’ wrong with that. It’s the lying or “divergence from stated goals” that’s bothersome.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by tgv on Tuesday February 20, 2024 @03:22AM (#39439103)
Don't you think it changes their goals? They now only care about the implications of their work as long as it is financial. Society? Fuck them, we've got big money to make!
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by madsbuch on Tuesday February 20, 2024 @03:55AM (#39439243)
The problem is when speculation is a (large) part of you compensation. You might not do what's right in your work.

The same is true for the real-estate market: When a corner of the market (eg. apartments in the city) outperforms the rest of the market, you might stop doing what's right for you and start speculating (being in the process of looking for a place to live, I can see several people who lost significant amounts after having bought in 06/07 and sole 10 years later - hopefully for them, they bought the right place to live and did not care about the losses).

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by m3kw9 on Monday February 19, 2024 @08:51PM (#39437124)
First, it was bs panic by board members and everyone knows this and that in turn ruined everyone’s bag, and for no good reason. Think about it, they wanted to disband the entire company because the world was about to end was the reason
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by dorkwood on Monday February 19, 2024 @08:56PM (#39437164)
Yep, now with the release of Sora we see that they're actually making the world a better and more creative place. People love to spout doom and gloom for no reason.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by drcode on Monday February 19, 2024 @09:00PM (#39437181)
that's what I keep saying

nightmare fuel puppies playing in the snow is what will make the world a better place

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by _heimdall on Monday February 19, 2024 @09:54PM (#39437526)
People definitely like to spout doom and gloom, but that doesn't avoid the fact that OpenAI is actually trying to develop artificial intelligence.

Ignore the fear mongering or political/military concerns for a moment. There are very real questions that have to be answered, from logistical questions like how we'll recognize when we develop an intelligence or the moral questions like does an AI have rights similar to humans.

Diamond nanobots ripping us apart to repurpose our atoms isn't a terribly useful conversation. But whether anyone can own an AI, whether turning one off is murder, or whether AIs deserve all the same rights and humans absolutely are. IMO those should have been prerequisites before we even considered questions like how to align an AI or risks of their existence, we shouldn't be going down that road at all if we're setting ourselves up to get caught with our pants down.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by dmitrygr on Monday February 19, 2024 @08:40PM (#39437057)
This is why they did.
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by dmix on Monday February 19, 2024 @08:57PM (#39437168)
Or they trusted Sam’s ability to run an AI company better than the 2 other e/acc advocates who wanted to pause AI development and brought in a temporary CEO who was on a podcast that same year calling for a multi yr pause on AI development.

Everyone points to money but they are also working on the most exciting technology in recent history. Crippling both that and likely their long term job prospects vs Sam, whom they never got a straight story for why he was being taken out.

People love to over simplify and reduce things while glossing over all the nuance.

Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by camillomiller on Monday February 19, 2024 @09:32PM (#39437405)
The only reason they didn't want the decels to do their shtick is because that would mean, indeed, loosing heaps of potential money. But you can believe in the fable about the most exciting technology yada yada yada
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by reducesuffering on Monday February 19, 2024 @09:28PM (#39437386)
You mean EA, not e/acc?
Re:OpenAI Deal Lets Employees Sell Shares at $86B Valuation by lord_zero_cool on Tuesday February 20, 2024 @08:50AM (#39441201)
OpenAI is nothing without its people