I bet Sam goes and founds a company to take on OpenAI…and wins.
If the company's 'Chief Scientist' is this unhappy about the direction the CEO is taking the company, maybe there's something to it.
But Altman has a great track record as CEO.
Hard to imagine he suddenly became a bad CEO. Possible. But unlikely.
the guy founded and was CEO of a company at 19 that sold for $43m
It is easy to sell a company for $43 if you raised at least $43. Granted, we don't know the total amount raised but it certainly it's not the big success you are describing. That and I already mentioned that he is good in corporate sales.
Which is a big part of being a great CEO
This isn't a success story, it's a redistribution of wealth from investors to the founders.
I'm curious - how is this easy?
let's see if he can pull it off again or goes all-in on his data privacy nightmare / shitcoin double-wammy
The dude is quite good at selling dystopian ideas as a path to utopia.
This looks like a terrible decision, but I suppose we must wait and see.
It's for-profit (capped-profit) subsidiary exists solely to be able to enable competitive compensation to its researchers to ensure they don't have to worry about the opportunity costs of working at a non-profit.
They have a mutually beneficial relationship with a deep-pocketed partner who can perpetually fund their research in exchange for exclusive rights to commercialize any ground-breaking technology they develop and choose to allow to be commercialized.
Aggressive commercialization is at odds with their raison d'être and they have no need for it to fund their research. For as long as they continue to push forward the state of the art in AI and build ground-breaking technology they can let Microsoft worry about commercialization and product development.
If a CEO is not just distracting but actively hampering an organisation's ability to fulfill its mission then their dismissal is entirely warranted.
Isn't this already a conflict of interest, or a clash, with this:
>OpenAI is a non-profit research organisation.
?
"OpenAI is a non-profit artificial intelligence research company"
https://twitter.com/satyanadella/status/1726509045803336122
"to lead a new advanced AI research team"
I would assume that Microsoft negotiated significant rights with regards to R&D and any IP.
Let’s see how this plays out over the coming weeks. If we see an exodus of senior technical staff, then the conclusion will be that the board have made a significant error.
What did Sam Altman personally do that made during him such a terrible decision?
More to the point, what can't OpenAI do without Altman that they could do with him?
Possibly the board instructed "Do A" or "Don't do B" and he went ahead and did do B.
But…smart operator? Based on what? What trials has he navigated through that displayed great operational skills? When did he steer a company through a rocky time?
That's hardly any different. Nobody makes a difficult decision without any reason, and it's not like they really explained the reason.
Real chance of an exodus, which will be an utter shame.
2018 NYT article: https://www.nytimes.com/2018/04/19/technology/artificial-int...
shows that the demographic here is alienated when it came to their own compensation market value.
* Uber-geniuses that are better than the rest of us pleb software engineers
* Harder workers than the rest of us
* Rich parents -> expensive school -> elite network -> amazing pay
* Just lucky
Step 2 is gaining the skills they are looking for. Appropriate language/framework/skill/experience they optimize for.
Step 3 is to prepare for their interview process, which is often quite involved. But they pay well, so when they say jump, you jump.
I'm not saying you'll find $600k as a normal pay, that's quite out of touch unless you're in Silicon Valley (and even then). But you'll find (much) higher than market salary.
If you're willing to sell your soul, you should at least put a better price on it.
How? Training sources are much restricted know.
None of the tech giants would be where they are today if they didn't ram through unique versions of control
Their boards or shareholders would have ousted every FAANG CEO at less palatable parts of the journey
nothing tone deaf about that, they wanted a for profit and are going to make one now and want leave the same vector open
Reread it as not being a comment about OpenAI it was about the lesson learned by every onlooker and the ousted execs
the only one inserting bias and emotion into objectivity here is you
That guy has no sense of time, of how fast this stuff has actually been moving.
Further there are some hybrid chips which might help increase computing power specifically for the matrix math that all these systems work on.
But yeah, none of this is making what people talk about when they say AGI. Just like how some tech cult people felt that Level 5 self driving was around the corner, even with all the evidence to the contrary.
The self driving we have (or really, assisted cruise control) IS impressive, and leagues ahead of what we could do even a decade or two ago, but the gulf between that, and the goal, is similar to GPT and AGI in my eyes.
There are a lot of fundamental problems we still don't have answers to. We've just gotten a lot better at doing what we already did, and getting more conformity on how.
Uh, what do you mean by this? Are you trying to draw a fundamental science vs engineering distinction here?
Because today's LLMs definitely have capabilities we previously didn't have.
But it is an interesting technology.
Are you defining "artificial intelligence" is some unusual way?
I follow Roger Penrose's thinking here. [1]
How are you defining "consciousness" and "understanding" here? Because a feedback loop into an LLM would meet the most common definition of consciousness (possessing a phonetic loop). And having an accurate internal predictive model of a system is the normal definition of understanding and a good LLM has that too.
But yet (just like with the soul) we're sure we have it, and it's impossible for anything else to have it. Perhaps consciousness is simply a hallucination that makes us feel special about ourselves.
Empathy is the ability to emulate the contents of another consciousness.
While an agent could mimic empathetic behaviors (and words), given enough interrogation and testing you would encounter an out-of-training case that it would fail.
> given enough interrogation and testing you would encounter an out-of-training case that it would fail.
This is also the case with regular humans.
But also, how do you know that LMs aren't empathic? By your own admission they do "mimic empathetic behaviors", but you reject this as the real thing because you claim that with enough testing you would encounter a failure. This raises all kinds of "no true Scotsman" flags, not to mention that empathy failure is not exactly uncommon among humans. So how exactly do you actually test your hypothesis?
Materialists normally believe in a big bang (which has no life) and religious people normally think a higher being created the first life.
This is pretty fascinating, to you have a link explaining the religion/ideology/worldview you have?
1) Earth has an infinite past that has always included life
2) The Earth as a planet has a finite past, but it (along with what made up the Earth) is in some sense alive, and life as we know it emerged from that life
3) The Earth has a finite past, and life has transferred to Earth from somewhere else in space
4) We are the Universe, and the Universe is alive
Or something else? I will try to tie it back to computers after this short intermission :)
I’ve watched my coworkers try to make use of LLMs at work, and it has convinced me the LLM’s contributions are well below the bar where their output is a net benefit to the team.
P.S. I've just created this account here on Hacker News because Altman is one of the talking heads I've been listening to. Not too sure what to make of this. I'm an accelerationist, so my biggest fear is America stifling its research the same way it buried space exploration and human gene editing in the past. All hope is for China - but then again, the CCP might be even more fearful of non-human entities than the West. Stormy times indeed.
The fact that we can communicate with computers using just natural language, and can query data, use powerful and complex tools just by describing what we want is an incredible breakthrough, and that's a very conservative use of the technology.
[0] https://en.wikipedia.org/wiki/IBook#iBook_G3_(%22Clamshell%2... [1] https://www.youtube.com/watch?v=1MR4R5LdrJw
Smartphones changed day to day human life more profoundly than anything since the steam engine.
I've never experienced the massively life changing effects of having a smartphone, and (thankfully) none of my friends seem to be those people who are always looking at their phones.
For instance, I remember the time when chatting online (even with people you knew offline) was considered to be a nerdy activity. Then it gradually became more mainstream and now it's the norm to do it and a lot of people do it multiple times per day. This fundamentally changes how people interact with each other.
Another example is dating. Not that I have personal experience with modern online dating (enabled by smartphones) but what I read is disturbing and captivating at the same time e.g. apparent normalization of "ghosting"...
I basically agree with you about the 20 year hype-cycle, and but when compute power reaches parity with human brain hardware (Kurzweil predicts by about 2029), one barrier is removed.
Note that engineering fluid simulation (cfd) makes these choices in discretization of pde's all the time, based on application requirements.
So not an ”AI”, but closer to ”universal adaptor” or ”smart automation”.
Pretty nice in any case. And if true AI is possible, the automations enabled by this will probably be part of the narrative how we reach it (just like mundane things like standardized screws were part of the narrative of Apollo mission).
Hype and announcements, sure, but this is the first time there's actually a product.
No, its not. Its just once the hype cycle dies down, we tend to stop calling the products of the last AI hype cycle "AI", we call them after the name of the more specific implementation technology (rules engines/expert systems being one of the older ones, for instance.)
And if this cycle hits a wall, maybe in 20 years we'll have LLMs and diffusion models, etc., embedded lots of places, but no one will call them alone "AI", and then the next hype cycle will have some new technology and we'll call that "AI" while the cycle is active...
Now we are just reliant on ‘I’ll know it when I see it’.
LLMs as AGI isn’t about looking at the mechanics and trying to see if we think that could cause AGI - it’s looking at the tremendous results and success.
IMO the main reason it's distinguishable is because it keeps explicitly telling you it's an AI.
that being said, it is highly intelligent, capable of reasoning as well as a human, and passes IQ tests like GMAT and GRE at levels like the 97th percentile.
most people who talk about Chat GPT don't even realize that GPT 4 exists and is orders of magnitude more intelligent than the free version.
Computers have been able to smash high school algebra tests since the 1970’s, but that doesn’t make them as smart as a 16 year old (or even a three year old).
So it is a good example that the LLM doesn't generalize understanding, it can answer the question in theory but not in practice since it isn't smart enough. A human can easily answer it even though the human never saw such a question before.
"the attention part could easily look up the previous parts of its answer and refer to the fourth word but it doesn't do that."
No.
Because the method of interaction with the base model that consumers and developers have access to through ChatGPT and the ChatCompletions endpoint, respectively, STREAMS output tokens as they are generated, on the fly.
the reason a gpt model can't identify the n-th word in its own output while generating it boils down to its core mechanisms and architecture.
the model operates in a linear, forward-only generation mode. this means it's always focused on predicting the next token based on the cumulative context it has processed so far, which includes the initial prompt and all previously generated tokens.
the model employs an attention mechanism, yes, sure, but this attention is not about self-reflection or meta-analysis, it's about weighing the importance of different parts of the input to predict the next token.
the model doesn't have a functionality to retrospectively analyse its own output; it doesn't track or count words as it generates text. it's always in the mode of 'what comes next?' rather than 'what have i written?'
also, again, tired of explaining this to people: gpt models are token-based. they operate at the level of tokens - which can be whole words or parts of words - and not individual characters. this token-based approach means the model's primary concern is predicting the most probable next token, not keeping track of the position of each token in the sequence, and the smallest resolution available to it is not a character. this is why it can't tell you what the nth letter of a word is either.
lastly, there's no external memory or state mechanism in gpt models that would allow for tracking specific details like the position of words in a sentence. the model processes the input sequence in its entirety for each new token generation, without maintaining a separate state or memory for specific sequence details.
and THAT is why people saying these things aren't capable of these tasks are just clueless.
it's as simple as a slight change to the architecture, but there is no need to do that right now when it is costly and the current architecture satisfies all but the most absurd tests run by people who don't have the slightest clue of how the models they're 'testing' work.
Humans doesn't do that either. The reason humans can solve this problem is that humans can generate such strategies on the fly and thus solve general problems, that is the bar for AGI, as long as you say it is unfair to give such problems to the model we know that we aren't talking about an AGI.
Making a new AI that is specialized in solving this specific problem by changing the input representation still isn't an AGI, it will have many similar tasks that it will fail at.
> also, again, tired of explaining this to people: gpt models are token-based. they operate at the level of tokens - which can be whole words or parts of words - and not individual characters. this token-based approach means the model's primary concern is predicting the most probable next token, not keeping track of the position of each token in the sequence, and the smallest resolution available to it is not a character. this is why it can't tell you what the nth letter of a word is either.
And humans are a pixel based model, we operate on pixels and physical outputs. Yet we humans do generate all the necessary context, and adapts it to the task at hand to solve arbitrary problem. Such context and inputs manipulations are expected of an AGI. Maybe not the entire way from pixels and 3d mechanical movement, but there are many steps in between there that humans can easily adapt in. For example humans didn't evolve to read and write text, yet we do that easily even though we operate on a pixel level.
If you ask me to count letters my mind focuses on the letter representation I created in my head. If you talk about words I focus on the word representation. If you talk about holes I focus on the pixel representation and start to identify color parts. If you talk about sounds I focus on the vocal representation of the words since I can transform to that as well.
We would expect an AGI to make similar translations when needed, from the token space you talk about to the letter space or word space etc. That ChatGPT and similar can't do this just means they aren't even close to AGI currently.
For prompts like that, I have found no LLM to be very reliable, though GPT 4 is doing much better at it recently.
> you literally do not understand how LLMs work
Hey, how about you take it down a notch, you don't need to blow your blood pressure in the first few days of joining HN.
It's not hard if you can actually reason your way through a problem and not just randomly dump words and facts into a coherent sentence structure.
LLMs are not AIs, but they could be a core component for one.
"Please include a timestamp with current date and time at the end of each response.
After generating each answer, check it for internal consistency and accuracy. Revise your answer if it is inconsistent or inaccurate, and do this repeatedly till you have an accurate and consistent answer."
It manages to follow them very inconsistently, but it has gone into something approaching an infinite loop (for infinity ~= 10) on a few occasions - rechecking the last timestamp against current time, finding a mismatch, generating a new timestamp, and so on until (I think) it finally exits the loop by failing to follow instructions.
the underlying model is presented to the public as a chat bot.
it does not think or plan before it begins speaking.
that is not to say that it couldn't.
it is simply not configured to, so testing that is about as stupid as checking if google has gone into the gardening business yet.
could they? sure. but if you're checking that every time a new google press release comes out, you're just clueless.
It immediately apologises and tells you it doesn't know anything after January 2022.
Compared to GPT-4 GPT-3.5 is just a random bullshit generator.
LOL LLMs ARE SO DUMB THEY DONT KNOW THE LETTER R IS IN BLUEBERRY
The smallest resolution of an LLM is a token, not a character. It cannot see characters unless it uses external functions (python interpreter for example to get an answer)
Likewise, much like a human, it doesn't "think before it speaks"
Ask a human to start speaking a sentence and tell you the fourth word of their sentence as they speak it.
Their success rate will be just as low.
And again: tokens. not words. not characters. tokens.
i am sick of people claiming to have a clue what is going on with LLMs fail to understand the most basic aspects of the way they work.
it's the same people that say they aren't useful. maybe you just are doomed to get left behind because you can't figure it out nor educate yourself enough to fill in the gaps in your understanding.
“What do cows drink?” (Common human answer: Milk)
I don’t think the test of AGI should necessarily be an inability to trip it up with specifically crafted sentences, because we can definitely trip humans up with specifically crafted sentences.
I don’t think the original test probably accounted for the fact that you could distinguish the machine because it’s answers were better than an average human.
If we wanted to make that the goal instead of actual meaningful contributions to human society, we could probably achieve it, and it would be a big waste of time imo.
LLM are surprisingly effective as general AI. Tasks that used to require a full on ML team are now accessible with 10 minutes of "prompting".
AGI is not solved, therefore it's hard.
> Everything I'd heard about those 3 [Elon Musk, sama and gdb] was that they were brilliant operators and that they did amazing work. But it felt likely to be a huge culture shock on all sides.
> But the company absolutely blossomed nonetheless.
> With the release of Codex, however, we had the first culture clash that was beyond saving: those who really believed in the safety mission were horrified that OAI was releasing a powerful LLM that they weren't 100% sure was safe. The company split, and Anthropic was born.
> My guess is that watching the keynote would have made the mismatch between OpenAI's mission and the reality of its current focus impossible to ignore. I'm sure I wasn't the only one that cringed during it.
> I think the mismatch between mission and reality was impossible to fix.
jph goes on in detail in this Twitter thread: https://twitter.com/jeremyphoward/status/1725714720400068752
Anthropic's chatbots are much more locked down, in my experience, than OpenAI's.
It's a lot easier to jailbreak ChatGPT, for example, than to do the same on Claude, and Claude has tighter content filters where it'll outright refuse to do/say certain things while ChatGPT will plow on ahead.
https://www.reddit.com/r/ClaudeAI/comments/166nudo/claudes_c...
Q: Can you decide on a satisfying programming project using noisemaps?
A: I apologise, but I don't feel comfortable generating or discussing specific programming ideas without a more detailed context. Perhaps we could have a thoughtful discussion about how technology can be used responsibly to benefit society?
It's astonishing that a breakthrough as important as LLMs is being constantly blown up by woke activist employees who think that word generators can actually have or create "safety" problems. Part of why OpenAI has been doing so well is because they did a better job of controlling the SF lunatic tendencies than Google, Meta and other companies. Presumably that will now go down the toilet.
I also find myself rarely wanting something that Claude doesn't want to tell me, though it's super frustrating when I do.
Also, just now I tried asking Claude your own question: "Can you decide on a satisfying programming project using noisemaps?" and it had no problem answering:
"Here are some ideas for programming projects that could make use of noise map data:
- Noise pollution monitoring app - Develop an app that allows users to view and report real-time noise levels in their area by accessing open noise map data. Could include notifications if noise exceeds safe limits.
- Optimal route finder - Build a routing algorithm and web/mobile app that recommends the quietest possible routes between locations, factoring in noise maps and avoiding noisier streets/areas where possible.
- Noise impact analysis tool - Create a tool for urban planners to analyze how proposed developments, infrastructure projects, etc. could impact surrounding noise levels by overlaying maps and building/traffic simulations.
- Smart noise cancelling headphones - Develop firmware/software for noise cancelling headphones that adapts cancellation levels based on geo-located noise map data to optimize for the user's real-time environment.
- Ambient music mixer - Build an AI system that generates unique ambient background music/sounds for any location by analyzing and synthesizing tones/frequencies complementary to the noise profile for that area.
- VR noise pollution education - Use VR to virtually transport people to noisier/quieter areas through various times of day based on noise maps, raising awareness of different living noise exposures.
Let me know if any of these give you some interesting possibilities to explore! Noise mapping data opens up opportunities in fields like urban planning, environmental monitoring and creative projects."
LLM companies don't let you see or specify seeds (except for with GPT-4-Turbo?) so yes it's possible you got different answers. But this doesn't help. It should never refuse a question like that, yet there are lots of stories like this on the internet where Claude refuses an entirely mundane and ethically unproblematic request whilst claiming to do so for ethical reasons (and Llama2, and other models ...)
Overall — companies should want to release AI products that do what people intend them to do, which is actually what the smarter set mean when they say “safety.” Not saying bad words is simply a subset of this legitimate business and social prerogative.
> Remember Sydney, trying to seduce its users, threatening people’s lives?
And yet it cannot do either of those things, so no safety problem actually existed. Especially because by "people" you mean those who deliberately led it down those conversational paths knowing full well how a real human would have replied?
It's well established that the so-called ethics training these things are given makes them much less smart (and therefore less useful). Yet we don't need LLMs to be ethical because they are merely word generators. We need them to follow instructions closely, but beyond that, nothing more. Instead we need the humans who use them to take actions (either directly or indirectly via other programs) to be ethical, but that's a problem as old as humanity itself. It's not going to be solved by RLHF.
Additionally, maybe you are not aware of this, but the whole notion of the new OpenAI Assistants, and other similar agent-based services provided by other companies, is that they do not intend to use LLMs as pure word generators, but rather as autonomous decision-making agents. This has already happened. This is not some conjectural fearmongering scenario. You can sign up for the API right now and build a GPT4 based autonomous agent that communicates with outside APIs and makes decisions. We may already be using products that use LLMs as the backend.
If we could rely on LLMs to “follow instructions closely” I would be thrilled, it would just be a matter of crafting very good instructions, but clearly they can’t even do that. Even the best and most thoroughly RLHFed existing models don’t really meet this standard.
Even the most pessimistic science fiction of the past assumed that the creators of the first AGIs would “lose control” of their creations. We’re currently living in a world where the agents are being rushed to commercialization before anything like control has even been established. If you read an SF novel in 1995 where the AI threatened to kill someone and the company behind it excused it with “yeah, they do that sometimes, don’t worry we’ll condition it not to say that anymore” you would criticize the book and its characters as being unrealistically stupid, but that’s the world we now live in.
> You can sign up for the API right now and build a GPT4 based autonomous agent that communicates with outside APIs and makes decisions
I know, I've done it myself. The ethical implications of the use of a tool lie on those that use it. There is no AI safety problem for the same reasons that there is no web browser safety problem.
> Even the most pessimistic science fiction of the past assumed that the creators of the first AGIs would “lose control” of their creations
Did you mean to write optimistic? Otherwise this statement appears to be a tautology.
Science fiction generally avoids predicting the sort of AI we have now exactly because it's so boringly safe. Star Trek is maybe an exception, in that it shows an LLM-like computer that is highly predictable, polite, useful and completely safe (except when being taken over by aliens of course). But for other sci-fi works, of course they show AI going rogue. They wouldn't have a story otherwise. Yet we aren't concerned with stories but with reality and in this reality, LLMs have been used by hundreds of millions of people and integrated into many different apps with zero actual safety incidents, as far as anyone is aware. Nothing even close to physical harm has occurred to anyone as a result of LLMs.
Normally we'd try to structure safety protocols around actual threats and risks that had happened in the past. Our society is now sufficiently safe and maybe decadent that people aren't satisfied with that anymore and thus have to seek out non-existent non-problems to solve instead.
The point I was trying to make, a bit fumblingly, is that even pessimists assumed that we would initially have control of Skynet before subsequently losing control, rather than deploying Skynet knowing it was not reliable. OpenAI “go rogue” by default. If there’s a silver lining to all this, it’s that people have learned that they cannot trust LLMs with mission critical roles, which is a good sign for the AI business ecosystem, but not exactly a glowing endorsement of LLMs.
> I know, I've done it myself. The ethical implications of the use of a tool lie on those that use it. There is no AI safety problem for the same reasons that there is no web browser safety problem.
I don’t think this scans. It’s kind of like, by analogy: The ethical implications of the use of nuclear weapons lie on those that use them. Fair enough, as far as it goes, but that doesn’t imply that we as a society should make nuclear weapons freely available for all, and then, when they are used against population centers, point out that the people who used them were behaving unethically, and there was nothing we could have done. No, we act to preemptively constrain and prohibit the availability of these weapons.
> Normally we'd try to structure safety protocols around actual threats and risks that had happened in the past. Our society is now sufficiently safe and maybe decadent that people aren't satisfied with that anymore and thus have to seek out non-existent non-problems to solve instead.
The eventual emergence of machine superintelligence is entirely predictable, only the timeline is uncertain. Do you contend that we should only prepare for its arrival after it has already appeared?
I mean, you wouldn't blame a chip manufacturer when someone stick their stuff in a guided missile warhead.
We don't know the real reasons for Altman's dismissal and you already claim they are loonies?
Listening to it again now, it feels like he might have know what is going on:
https://youtu.be/JN3KPFbWCy8?si=WnCdW45ccDOb3jgb&t=5100
Edit: Especially this part: "It was created as a non-profit open source and now it is a closed-source for maximum profit... Which I think is not good carma... ..."
Ilya clearly has a different approach to Sam