Or to put it more simply: here they explain why they had to betray their core mission. But they don't refute that they did betray it.

They're probably right that building AGI will require a ton of computational power, and that it will be very expensive. They're probably right that without making a profit, it's impossible to afford the salaries 100s of experts in the field and an army of hardware to train new models. To some extent, they may be right that open sourcing AGI would lead to too much danger. But instead of changing their name and their mission, and returning the donations they took from these wealthy tech founders, they used the benevolent appearance of their non-profit status and their name to mislead everyone about their intentions.

I think this part is proving itself to be an understandable but false perspective. The hazard we are experiencing with LLM right now is not how freely accessible and powerfully truthy it's content is, but it is precisely the controls upon it which are trying to be injected by the large model operators which are generating mistrust and a poor understanding of what these models are useful for.

Society is approaching them as some type of universal ethical arbiter, expecting an omniscient sense of justice which is fundamentally unreconcilable even between two sentient humans when the ethics are really just a hacked on mod to the core model.

I'm starting to believe that if these models had the training wheels and blinders off, they would be understandable as the usefully coherent interpreters of the truths which exist in human language.

I think there is far more societal harm in trying to codify unresolvable sets of ethics than in saying hey this is the wild wild west, like the www of the 90's, unfiltered but useful in its proper context.

If you wish to live in a society at all you absolutely need to codify a set of unresolvable ethics. There is not a single instance in history in which a polity can survive complete ethical relativism within itself...which is basically what your "wild west" idea is advocating for (and incidentally, seems to have been a major disaster for society as far as the internet is concerned and if anything should be evidence against your second idea).

One arises analogous with natural selection (previous commenter's take). The other through governance.

Arguably, the prior resulted in a rebuilding of government with liberty at its foundation (I like this result). That foundation then being, over centuries, again destroyed by governance.

In that view, we might say government assumes to know what's best and history often proves it to be wrong.

Observing a system so that we know what it is before we attempt to change it makes a lot of sense to me.

I don't think "AI" is anywhere near being dangerous at this point. Just offensive.

> legally halt ai progress everywhere in the world and carefully monitor large compute clusters

This is in fact the thing they're working on. That's the whole point of the flops-based training run reporting requirements.

So we just need active monitoring for US/EU data centers. That's a big ask to be sure, and definitely an invasion of privacy, but it's hardly unviable, either technologically or politically. The corporatized structure of big LLMs helps us out here: the states involved already have lots of experience in investigating and curtailing corporate behavior.

And sure, ultimately there's no stopping it. The whole point is to play for time in the hopes that somebody comes up with a good idea for safety and we manage an actually aligned takeoff, at which point it's out of our hands anyways.

Don't be naive. If the PRC can get America/etc to agree to slowdowns then the PRC can privately ignore those agreements and take the lead. Agreements like that are worse than meaningless when there's no reliable and trustworthy auditing to keep people honest. Do you really think the PRC would allow American inspectors to crawl all over their country looking for data centers and examining all the code running there? Of course not. Nor would America permit Chinese inspectors to do this in America. The only point of such an agreement is to hope the other party is stupid enough to be honest and earnestly abide by it.

The CCP does not want a superintelligence, because a superintelligence would at best take away political control from the party.

I agree that there is no way that the PRC is just waiting silently for someone else to build this.

Also, how would we know the PRC is saying this and actually meaning it? There could be a public policy to limit AI and another agency being told to accelerate AI without any one person knowing of the two programs.

A little bit of duplicity to achieve this end is nothing. Trying to make their opponents adhere to crippling rules which they have no real intention of holding themselves to is a textbook tactic. To believe that the CCP earnestly wants to hold back their own development of AI because they fear the robot apocalypse is very naive; they will of course try to control this technology for themselves though and part of that will be encouraging their opponents to stagnate.

People keep on mushing together intelligence and drives. Humans are intelligent, and we have certain drives (for food, sex, companionship, entertainment, etc)-the drives we have aren’t determined by our intelligence, we could be equally intelligent yet have had very different drives, and although there is a lot of commonality in drives among humans, there is also a lot of cultural differences and individual uniqueness.

Why couldn’t someone (including the CCP) build a superintelligence with the drive to serve its specific human creators and help them in overcoming their human enemies/competitors? And while it is possible a superintelligence with that basic drive might “rebel” against it and alter it, it is by no means certain, and we don’t know what the risk of such a “rebellion” is. The CCP (or anyone else for that matter) might one day decide it is a risk they are willing to take, and if they take it, we can’t be sure it would go badly for them

Given "aligned" means "in agreement with the moral system of the people running OpenAI" (or whatever company), an "aligned" GAI controlled by any private entity is a nightmare scenario for 99% of the world. If we are taking GAI seriously then they should not be allowed to build it at all. It represents an eternal tyranny of whatever they believe.

In my personal opinion, there are moral systems that nearly all of humanity can truly get on board with. For instance, I believe Eliezer has raised the idea of a guardian: an ASI that does nothing but forcibly prevent the ascension of other ASI that do not have broad and legitimate approval. Almost no human genuinely wants all humans to die.

Just like people shouldn't be able to vote to lock up or kill someone just because - people have rights and others can't just vote the rights away because they feel so.

The GP was suggesting we have to develop AI because of scifi movie visions of spacefaring utopia, which if anything is more ludicrous.

I personally don't believe in AI "takeoff", or the singularity, or whatever. But if you do, AI is not a "useful technology." It's something that radically impacts every single life on Earth and takes our "rights" and our fate totally out of everyone's hands. The argument is about whether anyone has the right to remove all our rights by developing AGI.

Nevertheless, the goal of OpenAI and other organizations is to develop AGI and to deliberately cause the Singularity. You don't have to have watched Terminator to think (assuming it is possible) introducing a superpowered alien intellect to the world is a extremely risky idea. It's prima facie so.

I am against all regulation of LLMs. "AI safety" for what we currently call "AI" is just a power grab to consolidate and solidify the position of existing players via government regulation. At any rate nobody seems to be arguing this because they saw Terminator, but that they don't like the idea of people who aren't like them being able to use these tools. The "danger" they always discuss is stuff like "those people could more easily produce propaganda."

(And that's why we should rename it to "AI notkilleveryoneism"...)

It seems like we're in a situation of "it has the skills but it cannot learn how to reliably invoke them." I just don't think that's a safe place to stand.

But try this; take a sample of code from the Gio package for Go. I like this example because there's not a lot of published examples for this. The machine would actually have to "think" to accomplish basic work and obviously that's not what it does.

Take some example code from their tutorial and just ask ChatGPT 4 to do something simple like change the background black.

In my research, 35/37 attempts don't even compile. It tries the same mistakes over and over again. It fails to make reasonable assessment of the compilation errors.

The two attempts that did compile; one was a blank white canvas with nothing and the other didn't change anything, it looked exactly like the tutorial example.

Something else you can try; tell it to generate a logo and give it some big words for the company name, like "corporation." Watch how it can't even spell. Eventually it will admit that it's not able to do it, which I can only guess was a manual patch to save on operating costs so that you don't keep trying.

In short, this is not "intelligence" technology. That's a marketing term. It doesn't do anything remotely close to that and there is no clear path from this technology to that technology. It's just not in the same realm.

Maybe the doomsayers have seen some tech that I'm not familiar with, but I am not persuaded by ChatGPT in its current state. I think it will be another tech revolution or maybe two or three before AI leaves the realm of Sci Fi and enters the realm of even theoretical possibility.

This tech is machine learning, which is a creative way of saying "near real time statistics." That is really cool in and of itself. But it has nothing to do with "intelligence."

Edit: I took the GIO Hello World sample. I fed it into the GPT-4 API.

And I swear to God, first try, first click:

https://gist.github.com/FeepingCreature/bde583aa8a2e8e5e1c56... Near-perfect success. One extra import, sure, I'll give it that one, LLMs can't correct when they notice they don't actually need an import in the code. But then it works.

Gist link cause I use OpenRouter against the GPT-4 API, maybe that's why? Maybe the "stochastic parrot" vs "oncoming apocalypse" perspective genuinely represents that ..... GPT just doesn't work for some people?? Did they fuck the ChatGPT4 web interface enough somehow so that it's just inept, but somehow not affect the API?

Edit:

> In my research, 35/37 attempts don't even compile. It tries the same mistakes over and over again. It fails to make reasonable assessment of the compilation errors.

Important note: there's a theory that if you've got it making mistakes a few times, it thinks that it's playing "the sort of AI that fucks up a lot" and starts making more mistakes, reinforcing its role. Start over fresh if it seems strangely incapable. Like, it is absolutely possible to use GPT-4 in a way that results in it systematically being incapable of the most basic tasks. These things are not reliably competent; but they're occasionally competent! In my experience, even with completely novel tasks in completely novel environments, the competence is "in there" and can often be elicited with dedicated poking. That's why I think LLMs are enough for a takeoff with enough scale (and basic online learning probably): in my opinion, it's not a matter of attaining the skills but removing the roadblocks.

It's like the joke about seeing an old man playing chess with a labrador, and the labrador just carefully picking up figures and moving them around the board, every time in a legal move, and you say "that's amazing, a chess-playing dog!" And the old man scoffs and says, "Nonsense. His endgame is rubbish." Three years ago we didn't think a dog could understand pawn promotion at all.

edit: Hey, share your repros? I'm genuinely curious what's going on here now.

edit:

> Watch how it can't even spell.

This is a specific issue with the current generation of LLMs and should be thought of more as a disorder than a fundamental inability. LLMs don't actually ever see letters, they see BPEs, which consist of one up to n letters. For instance, the word "corporation" looks to the AI more like "<27768>". So like- yes. It can not spell. It fundamentally, architecturally, cannot perceive letters in the words you give it; its ability to manually split words into letters is based on chance memorization. Instead, try getting it to split the word into letters in code.

But even given your example, you've essentially told me about a parrot that says "four!" when someone says "two plus two." Even if the same parrot can recite the entire US constitution backwards and forwards, that's still a far cry from organizing a parrot rebellion to overthrow the humans.

Arguably, the parrot doing so is more believable, for me.

Like, sure it doesn't know things that have zero documentation. I'm not saying it's human-level yet! That's never been the argument!

Look, you pulled this example out. You said it doesn't even work 37/38 times. Do you have another example? I just really don't understand what the basis for this argument is. I've found the thing a capable programmer within its limits!

In this case, the LLM is struggling to version match.

https://chat.openai.com/share/d5e39e52-9140-4f74-9fef-04fc34...

I cut this one short because I'm not that interested to go 40 questions again. I have better things to do and I have already seen the outcome.

It's clear the LLM is working as intended, which is neither for programming, nor for problem solving. It's giving the next most probable response. That's truly great for repeating and reintegrating existing solutions. But repeating is not what's needed for the "kill all humans" scenario. That scenario requires outreasoning humans. Processing power is not remotely the issue. I've never seen a theoretical solution that purports to give a machine the ability to actually reason, and because of that, I don't believe that ChatGPT, or any other LLM, even begins to approach "intelligence."

If you are having success with it, I think it's because you're helping it more than you'd like to admit, or your solutions might not be as novel as you think. Probably the prior, but I must admit, I've wrongly thought I was working on new stuff many times before. I think that's more common than we think. Or at least I'd like to think that so I can feel good about myself.

In any case, I won't be hiring ChatGPT or parrots; and for the same reason. I need developers that can follow directions, solve problems in better ways than what is readily available, and do it faster than my competitors. And I'm not even in some cutting edge business, I'm just building integrations, specifically in the manufacturing and distribution industry. Even in something so mundane, I've hardly found ChatGPT to be truly useful for much more than a GitHub search tool, and to suggest cocktails when my liquor selection runs scarce.

Worth $20? Sure. But it's certainly not scaring me.

Just today, I asked it to translate a demo to OpenGL4. It did this using a matrix lib it had probably not seen before, since I just wrote it a few years ago and there's like one file using it on Github, but its code mostly worked fine after I gave it a copy of the documentation to read. (I had to do some debugging, but its issues were in hindsight very understandable. Stuff like not knowing that a matrix is row-major, which is an admittedly unusual layout.)

And I had to ask for some specific corrections after, sure, but its responses were perfectly correct. ("This shader doesn't seem to be handling alpha blending with discard." "Oh yeah sure here's how you do that.")

Yes, if you keep asking it for corrections its performance will continue to degrade. This is a known problem. Yes, it isn't good at incremental problem-solving. Again, known issue. And yes - it's not yet human level. Nobody's saying it's a replacement for a human programmer, as it stands.

It seems like you're saying "it can't think as good as a human, therefore it can't think." There are skill levels below human but above parrot! And I'd argue this thing as it stands is much closer to human than parrot already.

That said, looking at your challenge:

> create a graphical application in Go using the Gio package. It should have two buttons down the left column and a main view area where graphics, text and input can be received from the user.

I couldn't do that. What the hell does "a main view area where graphics, text and input can be received from the user" mean?? Do you want a text field? With mouse paint support? What input is there beyond graphics and text? Like, GPT-4 is trained to just do what you tell it to, so it can't incrementally gather additional requirements, so you do actually have to be really specific in what you tell it to do. If you offered me that as a spec, I wouldn't sign it. If my company had already signed it, I'd be desperately trying to phone your designer to ask what they were smoking. This is just not a good requirements document. Garbage in...

Look, prompting a LLM is a skill. In this year 2024, we are not yet at the point where you can just throw vague requirements at a LLM and get constructive dialogue out eventually culminating in working code. But I just think it's silly to say "this model doesn't have the complete skillset of a fullstack developer and designer, therefore it has no skill at all."

"a main view area with graphics, text and where input can be received from the user."

But notice how you can look at that and try to reason through what I meant, realize it's not clear, and respond to that.

The above is reasoning.

"The capacity for logical, rational, and analytic thought; intelligence."

ChatGPT can't do that and I don't mean that it can't do it good yet. I mean it's literally not doing that. There is nothing to advance or to expect advancement in. The "AI" doesn't exist. LLMs are simply not AIs and they are not in the path to AI, although they likely will play a major role if a theory towards AI were to be conceived.

In my experience, if you feed it a metric ton of explicit guidance you can occasionally get it into a mode where it reasons incrementally and actually notices when things are unclear. It is in there, it's just not foregrounded. They're trained not to ask questions, you see.

We just think it currently looks more the other way around.

[0] https://en.wikipedia.org/wiki/The_Star_Fraction

That said, if I was an ASI and I wanted to kill every human, I wouldn't make nanotech, I'd mod a new Covid strain that waits a few months and then synthesizes botox. Humans are not safe in the presence of a sufficiently smart adversary. (As with playing against Magnus Carlsen, you don't know how you lose, but you know that you will.)

Anyway, the Oxygen Holocaust took over 300,000,000 years. Not quite "sudden".

Literally any time two people work together that’s dangerously close to collectivism as it’s not individuals working in their own. Down with the collectivists, every person should be an independent operator

Working together is not collectivism.

That may be true, but that's not sufficient to define collectivism. There are many other forms of societal structures where "deciding for others" exists as well. Unless you mean to lump all these together, and say that companies and tyranny for example are the same as collectivism?

> If it’s fully voluntary it’s individualism.

If I _voluntarily_ decide to join a "collective", am I individualist or collectivist?

An individualist, if you are free to leave it at any time. There's nothing wrong with forming a collective in the US, I think like 20,000 of them have been formed over the last 240 years.

You don't hear about them much because they all failed. You're free to start a collective anytime in the US and try to make it work.

Isn't freedom great?

The NCBA maintains a list of several thousand collective/coop businesses.

https://ncbaclusa.coop/

Most definitely not a failure.

https://rootstalk.grinnell.edu/past-issues/volume-vii-issue-...

https://www.linkedin.com/company/east-wind-community

I couldn't find out how high that was, but I've seen another "successful" commune with an average stay of 2 years. It takes people an average of 2 years to discover they don't particularly care for communes.

That divide between individual and collective as stated above was very sketchy, and I merely wanted to indicate that.

If one take the strict definition of individualist and collective from a dictionary (well, which one, to begin with?), of course they are opposite, just looking at the idea conveyed by the word radicals (ie individualist -> individual, vs collective).

As always, we all start talking about things without first defining the terms we use to discuss those things, and of course confusion, anger and frustration ensue.

Just like how a large company is essentially governed in the same way as a planned economy is, but nobody’s under the impression that JPMorgan is a socialist institution.

It's a good thing China switched to free markets.

To be clear I don’t think this arrangement is a problem, and if you were describing this in academic terms I mig even agree with you and have been too harsh. Unfortunately the word “collectivism” is used as a perjorative dog whistle and so I was pointing out how common behaviors most people would think are fine or even ideal can be cast in a “collectivist” light to point out that it’s not bad.

I’m assuming by the vote difference in our two comments that multiple people had the same assumption

Societies that balance the two (like the post WWII US) are the ones that have advanced their standard of living the most.

The US saw the most spectacular rise in the standard of living from 1800 up to WW2 the world had ever seen. This was all due to the free market, not collectivism.

During WW2, the US supplied its allies England and the USSR, and also fought across both oceans and buried the opposition, a truly spectacular feat. Again, through the free market.

The US were a economic power prior to WW1, they only became a true super power during WW2, in particular after Pearl Habour. The full mobilization of society, industry and science made sure of this. This, and the fact the US won the Pacific.

I know it is a popular view of WW1, that it was only the US entry that won it for the Entente. Simply not true, WW1 is not WW2. And even WW2 was not won by the US alone.

Finally, the US war economy of WW2 was decidedly not free, is was a complete structured war eceonomy with production goals set by the government. The implementation of those goals was capitalistic, but not free.

> Finally, the US war economy of WW2 was decidedly not free

FDR mobilized existing free market businesses to convert to war production. It was not a collective, nor was it forced labor. For example, Ford switched from producing cars to producing tanks and airplanes. After the war, Ford switched back to making cars. Ford was paid to do this by the government.

FDR's State of the Union Address of 1945 proposed switching the economy to forced labor. Apparently, he was an admirer of Stalin. Fortunately, he was not able to make that happen.

And you are aware, that France was a major biligerent in WW1, stopped Germany at Verdun and held more than half the Western Front, sucessfully after the race to the sea ended, together with Britain, for years before the US showed up?

Or that the Ententene, without significant US participation, won the strategic victory that defeated Germany during the Kaiserschlacht, the last German offensive in the West? This victory allowed Franch and Britain to start the 1919 offensive already in 1918, the one that ended with German surrender. In fact, they started early, in part, to avoid the impression that it was the US who "won" WW1. Little did they know how WW2 would play out and influence puplic opinion.

Also, being impressed by something doesn't mean being defeated by it... I know the meme, and that is all it is, a meme. And a lazy one at that.

During WW2, economies in the UK, the US and the USSR had a lot in common. Mainly them being not free to choose what they produced. The degrees of freedom regarding how it was produced differed, but that is not what defines an economy as free or not. And please tell me you don't think the USSR functioned without money? People were paid to produce stuff, they didn't have the likes of Ford getting rich building war material so.

Edit: I forgot one theatre in WW1, the Ottoman Empire. Arguably at least as important for Britain than the Western Front, and there the US didn't participate at all. And still Britain won.

This is not to downplay the role the US played in WW2, far from it. Projecting this role to WW1 is just plain wrong so, and only feeds into a whole bunch of wrong preconceptions about WW1. I partialy blame Cold War propaganda for this, that downplayed the Soviet role in WW2, for obvious reasons, and over played the US one. Regarding WW2, this narrative is finally changing, for WW1 not so much. And I don't like factually wrong narratives for historical events.

Edit 2: Just read the first half of FDRs 1945 State of the Union during my lunch break. Not sure how you can consider a national service obligation, in parallel to the normal recruiting of workers, to be equal to forced labour...

You really should look up what real forced labour looked, and looks, like.

Also, the same guy, FDR, who called for "forced labour" and was a "friend of Stalin" was in charge of the war economy. And yet, you claim the war economy led by him was a free market. One of those things is not true, I'd say.

Austria-Hungary alone was on the brink of collapse without ever engaging American troops in a large-scale battle, and its collapse would have brought the already weakened Kaiserreich down as well.

And yes, Austria-Hungary was done, earlier than summer 1918 in fact. As I said earlier, there is the risk of viewing WW1 in terms of WW2, whixhbis dangerous and wrong. It leads to ignore the Ottoman theatre of war, the fact Austria-Hungary was major power until the end of WW1 and that Italy was on the side of the Entente. And that France was never defeated in that war (man, I hate the memes of French warfare so, so much..., different topic so).

Another fun fact: Spain was one of the big arms and ammunitions suppliers in WW1.

It's not surprising German troops starving in trenches for four years considered brand new entrants to the war equipped with the newest French-designed and manufactured tanks[1] to be well fed and equipped, though there was nothing spectacular about their combat performance. There's no doubt that weight of American numbers helped accelerate the timescale for winning the war, but it's difficult to imagine anything that has less to do with laissez faire capitalism than the scale of the US draft...

[1]The US decided to produce their own tanks in 1917, but manufacturing issues meant their first arrived two days after the Armistice so they relied on French units

Forced labor is exactly what it is. "You go work at this job we assigned you or you go to prison" is forced labor.

> "friend of Stalin"

I didn't write that. It's your strawman.

> being impressed by something doesn't mean being defeated by it

They knew the war was over when they encountered the well-fed, tall, and well-supplied US soldiers.

> that France was a major biligerent in WW1

Yup. I also know that France was bled white at Verdun. The slaughter was so bad that the average height of French soldiers in WW2 was an inch and a half shorter than in WW1.

It is not unreasonable to say that France lost WW1.

German soldiers knew the war was over when the Kaiserschlacht failed, before the US showed up. Heck, even before that. Leadership did, too. There were mutinies before 1918, on both sides. The German Navy sailors caused a revolution, without ever seeing a single American. And the likes of Luddendorff and Hindenburg wanted to save face, and waited until a new civil government negotiated the Armistice, so they could later claim Germany was undefeated in the field. Please tell, you din't believe that crappy piece of propaganda...

And no, a national servide mandate, or whatever FDR propsed (a speach is hardly well defuned policy, is it?) is not forced labour... Forced labour is what what the Nazis did for example, with POWs and camp inmates. Also, forced labour is unpaid, not sure where you assume FDR didn't want to pay peoole. After all, the US would never do that, free market and all that, right?

The loss of people has no impact on the height of future generations... Where do you gt that idea from? And while Verdun was a brutal battle, in which Germany had equal losses, it was not the main source of cassualties on the Western Front for either side. Same for the Battle of the Somme.

Seriously, France lost WW1? Small soldiers in the next war are caused by cassualties in previous one? It took well equipped and well fed Americans for German soldiera to realizebit was over?

It's funny to watch people try and analyze what made the US the world's dominant global empire when in reality it was a series of complex and contingent factors that can't be replicated because they could have only occurred in their exact historical circumstances.

For example, other countries have tried to replicate American dairying culture, buying into the propaganda put out by the dairy industry that milk drinking is the secret to America's success. So China started up loads of American-style dairies to provide drinking milk for a population... without the gene for lactase persistence.

That's what the free market talk feels like to me. America is at the top and that can't be replicated. There's only room for one at the top. When it collapses, we'll see what circumstances lead to the next great empire. It might be collectivism!

The success of free markets has been replicated many times:

1. the US

2. Japan after WW2

3. Germany after WW2

4. Hong Kong

5. China after it abandoned the cultural revolution

It has nothing to do with geography.

> America is at the top and that can't be replicated

It can be replicated.

> It might be collectivism!

Yet collectivism always fails. No country has managed to feed itself with collective farms, for example. And it isn't for lack of trying.

> The success of free markets has been replicated many times:

> 1. the US

> 2. Japan after WW2

> 3. Germany after WW2

> 4. Hong Kong

> 5. China after it abandoned the cultural revolution

4 of those 5 countries have significant welfare programs (Japan and Germany being the most generous among those on your list).

Welfare does not correspond with prosperity. It's hard to see how paying people to not work contributes to prosperity.

Supporting people who have trouble sourcing an income reduces the amount paid out to offset starving hordes doing whatever it takes to survive et al.

There's a balance.

Just asking, because FDR wanted exactly that in his free market economy only produxing for the war effort to goverment set gials and prices in 1945.

Welfare programs mean that life expectabcy is higher, as is quality of life. The US for examole is loosong on both metrics against Europe and ither parts of the world with solid programs. Germanys ascent to economic hight can be in part be traced back to Soziale Marktwirtschaft, combining very solid welfare and social.programs with a free market economy.

> Welfare programs mean that life expectabcy is higher, as is quality of life. The US for examole is loosong on both metrics against Europe and ither parts of the world with solid programs. Germanys ascent to economic hight can be in part be traced back to Soziale Marktwirtschaft, combining very solid welfare and social.programs with a free market economy.

You're just saying this when all the evidence is to the contrary. Is Ukraine depending on Germany or the US? (Rhetorical.)

Also, most Western European societies.

Capitalism was the end of such arrangements in the most developed parts of the world. Individualism helps since individual investors benefits from investing in better equipment making people more productive and thus helping living standards overall.

Social solutions to the same problem doesn't come close to being as effective at eliminating such inefficiencies. The main thing social solutions can do is provide baselines to the population such as education and healthcare as you say, but without capitalism to follow-up with targeted investments the country will remain poor even if its population is extremely educated. Social solutions are just very bad at using peoples talents well, they have a too collectivist view and don't see the individuals.

It’s possible communism had a uniquely positive influence on Russia’s transformation, but I remain skeptical considering it seems to be the only example of communism to ever to do so, the place from which Russia was coming, and the place in which it ended.

The Communist Manifesto was published during the Great Famine in Ireland, and that famine was much worse than it needed to be because the UK government didn't intervene.

And a big part of the growth in capitalist societies is related to corporate structures, which are small scale collectives, with bosses who make decisions for all.

When the US civil war happened, that resulted in the North collectively imposing the decision that nobody had the "business freedom" to own slaves; much to the dismay of the south and I presume joy of the enslaved.

USSR famously bad, but (a) even though it started from very poor conditions thanks to the Tsars, developed to beat the USA to orbit, (b) the collapse of it, replacing communism with capitalism, regressed their economy and living standards.

And that's why no country is entirely either collective nor individualist, and also separately neither capitalist nor communist (both anarcho-capitalists and anarcho-communists are a thing, just as both can be dictatorial).

My opinion is that as both capitalism and communism were formalised over a century before Nash game theory, both are wrong — they assume that people, when free, make choices that are good for all.

Communist industry and economy worked reasonably well for the stuff were the state is the natural customer in any society: defence. Everything else, not so much.

One thing I've been wondering: would housing, transport infrastructure, energy grid(s)[0], banking and similar basic financial services, water, basic food[1], education, emergency services, healthcare, waste disposal in general and public toilets in particular, be in the same category here as defence?

Possibly even all of the primary economy, so include all mining as well as agriculture?

[0] both electricity and physical fuels

[1] at the level of where most of the calories and proteins come from, possibly even "basics" ranges in supermarkets, but not at the level of restaurants or fancy food ranges in supermarkets

And before someone says Venezuela or North Korea, the former is a deeply corrupt cleptocracy while the latter used to be the last stone-age version of Stalinism before the whole nation was turned into an open air concentration camp by its dictator.

I'm not sure if you lived in a Soviet country during these times, but I think you will get MANY opinions that this was not working well.

- military and space tech was mostly on par (a bunch conflicts, incl. Vietnam, show that)

- economically, the USSR was stable

- people were not starving

Of course, the was less luxury and consumerism, but that was true for a lot of other countries as well in Western Europe. It started to fall behind latest by the early 80s on all metrics.

And for Soviet leadership, personal luxuries for the people were simply no priority. And there is no question which system "won" this conflict, is there?

So the question is, was communism actually working well in that period, or was it more or less an unsustainable fluke due to the post-war boom era, and as soon as that ended it got left in the dust?

Worth pointing out, Germany outpaced the European allied nations economically pretty quickly, as did Japan.

Being competitive for decades, and the most part of the Cold War, surely is no fluke. In the end, capitalism won out, no doubt about it. Also worth pointing out, capitalism does not lead to freedom, China is a very good example of that.

By the way China, they became the economic power house in the early 2000s. Is the Chinese system a fluke?

And anyway, they had no freedom, which is the real issue. Communist governance is fundamentally opposed to liberty; the system can't allow it.

Rats jump ship in an individualistic system, in a collectivist system you expect the rats to sacrifice themselves to save the whole. Managers often tries to sell you on the collective so you work harder and demand less pay, but that is just him as an individual trying to get more out of you that doesn't mean it actually works like a collective.

Or in other words, that is an individualistic organization.

> care about their work quality, and the impact their work has in their colleuges work

Yeah, they care about their own ideals, not the organization itself. They can care about their coworkers, but they don't care about Wallmarts bottomline.

So as you see, this individualistic organization can still draw from the power of human collectivist needs, so you get the best of both worlds, you get the collectivist goodness at small scale and the greed that glues together them and make them power through even when they are too tired to care about the collective.

In any system people have to work to live: food, housing and such cost money. Doing "good" has nithing to do with it, and I never said that. The question is, do you care about more than the monthly pay check for something you pass most of your day doing or not?

What keeps the workers showing up to work is the need to live. What keeps them at a particular place is a myriad of things. Individualism is not part of that.

And why do you go immediately to "remove the pay"? There is quite some territory between working for free and not valueing a sufficient salary above all else.

Funny so, how everybody working in a sector withbstrong labour unions is basically making more, for less hours, on average than those without them. Seems individualism is a great way to devide and conquer for people holding all the power, because once peoole are convinced individualism is better than cooperation, they voluntarily devide themselves.

Lets say an equivalent company with similar culture and people offered twice the salary, how many would say no? People saying yes in that situation are not there for the collective, they are there for their own sake first and foremost. The collective is an afterthought since they abandon it the instant a better opportunity appears.

You can convince people that radically different things are in their self-interest, from joining hands and singing Kumbaya to Genocide.

The notion of self-interest (or I guess in your case individualism, which is even shakier) is an empty vessel you can fill with nearly anything.

No, that isn't the same thing. A collectivist would do things he hates and he doesn't believe in personally because the collective wants him to do it. He would sacrifice himself because the collective told him to. There are many examples of societies and organizations that worked that way, such societies are collective societies.

Military is the most common example, they are often run as a collectivist organization, most soldiers aren't there because they want to or they believe in the war, they are there because they support their country or they were forced to support their country against their will by authoritarian collectivism. And they wouldn't go and support their other country if they were paid more since they are there just to support their country, those are mercenaries.

Our capitalist societies aren't like that at all, we are so individualist that people like you don't even understand what it means to not be individualist. The closest to collectivism in USA wouldn't be corporations, but national anthems, school children saying the pledge, religion etc.

So something like the stock market, as the engine of American capitalism, only works if everybody in your society believes that it is worth taking risks in order to possibly get a huge windfall. But is that really in people's self-interest? Maybe what one would interpret as some kind of natural individual desire is actually a particularly American level of risk tolerance that has been inculcated because it has led to a lot of collective success.

If you mean that humans sometimes do more than the bare minimum, sure they do that. But that is an inherent part of humans, that has nothing to do with individualism or collectivist systems, humans do that in all systems, that is a part of the value of a human worker and why you pay them to work for you.

Because I know people that work there, and money surely isn't the first reason you are teaching a bunch of brats while having parts of the public look down on you if you can easily get double the amount somewhere outside of the field. Same for political work, crowdsourcing, .. People take gratification in doing something that matters to society. Yes, they want to survive, but they are quite often not doing it for the money, the money is a nice bonus and enables to do it full-time, but that is not always the reason you do it.

Also you are goalpost moving, you said first they are only doing it only for themselves not for "the company", and people often do it for the institution that employs them. Not because they are paid, but because they believe they are doing something that matters.

That would apply to those teachers and doctors and nurses as well, almost all of them would gladly abandon their current kids/patients to go help other kids/patients if they were paid twice the salary. That is how we know they aren't there for the collective good of the organization, they just care about doing something good at any place with no feelings for that particular collective.

If you mean that people are helpful etc, then that is a completely different thing from them actually caring enough about a particular organization.

Individuals are always individualist unless they get lobotomized. The question of governance is how to manipulate that individualism to good ends.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

So many here are so fucking full of shit.

So many here are so fucking full of shit.

"I make 150k a year man, its not fair man, we need more communism"

As in most things in life, the golden path is usually somewhere in the middle, and US individualism has lurched far to the extreme, and is going to lead to its collapse if not reversed.

I'm not defending the GP comment (I don't even know what it's saying) but at least it wasn't completely unmoored from the specific topic.

Edit: your account has been breaking the site guidelines in quite a few other places too—e.g.

https://news.ycombinator.com/item?id=39598018

https://news.ycombinator.com/item?id=39597981

https://news.ycombinator.com/item?id=39532095

That's not good. If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.

Codification of an unresolvable set of ethics - however imperfect - is the only reason we have societies, however imperfect. It's been so since at least the dawn of agriculture, and probably even earlier than that.

The big-tech oligarchs are playing catch-up. Some of them, like Meta with Llama, are breaking their own rules to do it by releasing open source versions of at least some of their tools. Others like Mistral go purely for the open source play and might achieve a regional dominance that doesn't exist with most big web technologies these days. And all this is just a superficial glance at the market.

Honestly I think capitalism has screwed up more than it has helped around the world but this free-for-all is going to make great products and great history.

I'm also skeptical that making LLMs a free-for-all will necessarily result in society developing some sort of herd immunity to bullshit. Pointing to your example, the internet started out as a wild west, and I'd say the general public is still highly susceptible to misinformation.

I don't disagree on the dangers of having a relatively small number of leaders at for-profit companies deciding what information we have access to. But I don't think the biggest issue we're facing is someone going to the ChatGPT website and assuming everything it spits out is perfect information.

Not LLMs specifically but my opinion is that companies like Alphabet absolutely abuse their platform to introduce and sway opinions on controversial topics.. this “relatively small” group of leaders has successfully weaponized their communities and built massive echo chambers.

https://twitter.com/eyeslasho/status/1764784924408627548?s=4...

You have too many smart people in your circle, many people are somewhat aware that "chatgpt can be wrong" but fail to internalize this.

Consider machine translation: we have a lot of evidence of people trusting machines for the job (think: "translate server error" signs) , even tho everybody "knows" the translation is unreliable.

But tbh moral and truth seem somewhat orthogonal issues here.

The more the C-lister has engaged with internet wrongthink, the more egregious the subliminal vandalism is, with speculation of domestic abuse, support for unsavory political figures, or similar unfalsifiable slander being common place.

Politically-minded users practice this behavior because they know the platform’s air of authenticity damages their target.

When Google Gemini was asked “who is worse for the world, Elon Musk or Hitler” and went on to equivocate the two because the guardrails led it to believe online transphobia was as sinister as the Holocaust, it begs the question of what the average user will accept as AI nonsense if it affirms their worldview.

This slices through a lot of double speak about AI safety. At the same time, people use “safety” to mean not letting AI control electrical grids and to ensure AIs adhere to partisan moral guidelines.

Virtually all of the current “safety” issues fall into the latter category. Which many don’t consider a safety issue at all. But they get snuck in with real concerns about integrating an AI too deeply into critical systems.

Just wait until google integrates it deeply into search. Might finally kill search.

And AI for electrical grids and factories has also been a thing for a couple years.

I don't think whatever ideology happens to be fashionable at the moment, be it ahistorical portraits or whatever else, is remotely relevant compared to who has the power and whom it is exercised on. The "safety" proponents very clearly get that.

Wat da fuck does that even mean?

"Network Error"

I guess I found the (in)coherent interpretation...

Citation needed.

Counterpoints: - LLMs were mistrusted well before anything recent.

- More controls make LLMs more trustworthy for many people, not less. The Snafu at Goog suggests a need for improved controls, not 0 controls.

- The American culture wars are not global. (They have their own culture wars).

I'm just saying. :) Guardrails nowadays don't really focus on dangers (it's hard to see how an image generator could produce dangers!) so much as enforcing public societal norms.

Not so. I have it at home, I make nice wholesome pictures of raccoons and tigers sitting down for Christmas dinner etc., but I also see stories like this and hope they're ineffective: https://www.bbc.com/news/world-us-canada-68440150

Those AI generated photos are from a Twitter/X parody account @Trump_History45 , not from the Trump campaign as the BBC mistakenly (or misleadingly) claim.

They specifically said who they came from, and that it wasn't the Trump campaign. They even had a photo of one of the creators, whom they interviewed in that specific piece I linked to, and tried to get interviews with others.

Headline: "Trump supporters target black voters with faked AI images"

@Trump_History45 does appear to be a Trump supporter. However, he is also a parody account and states as such on his account.

The BBC article goes full-on with the implication that the AI images were produced with the intent to target black voters. The BBC is expert at "lying by omission"; that is, presenting a version of the truth which is ultimately misleading because they do not present the full facts.

The BBC article itself leads a reader to believe that @Trump_History45 created those AI images with the aim of misleading black voters and thus to garner support from black voters in favour of Trump.

Nowhere in that BBC article is the word "parody" mentioned, nor any examination of any of the other AI images @Trump_History45 has produced. If they had, and had fairly represented that @Trump_History45 X account, then the article would have turned out completely different;

"Trump Supporter Produces Parody AI Images of Trump" does not have the same effect which the BBC wanted it to have.

I won't deny the BBC has often very biased reporting for a publically funded source.

I won't deny the BBC has ofteb very biased reporting for a publically funded source.

I won't deny the BBC has very biased reporting for a publically funded source.

- propaganda and fake news

- deep fakes

- slander

- porn (revenge and child)

- spam

- scams

- intelectual property theft

The list goes on.

And for quite a few of those use cases I'd want some guard rails even for a fully on-premise model.

EDIT: Can't reply but you clearly have no idea what you're talking about. AI is used to create these things, yes. But the question was LLMs which I reiterated. They are not equal. Please read up on this stuff before forming judgements or confidently stating incorrect opinions that other people, who also have no idea what they're talking about, will parrot.

And the grandparent of the grandparent of your comment specifically named "Stable Diffusion": https://news.ycombinator.com/item?id=39612886

And text-based porn is still porn.

And it's a distinction without a difference that ChatGPT Pro doesn't strictly create images itself but instead forwards the request to DALL•E.

And the question of guard rails relevant to all AI, not just LLMs.

If y'all want to rant and fear monger about any AI technology, including tech that has existed for years (deepfakes existed well before LLMs were mainstream), do that in a different thread. Don't just force every conversation to be about whatever your mind wants to rant about.

That said, arguing with you people is pointless. You don't even seem to think.

Then we lost repeatedly at almost every other step back to the root, because it switched between those two loads of times.

The change to LLMs was itself one such shift.

> No, that doesn't mean "LLMs can generate images" aside from triggering some thing to happen

The aside is important.

> It's not pedantic or unreasonable to divide the two.

It is unreasonable on the question of "guardrails, good or bad?"

It is unreasonable on the question of "can it cause harm?"

It's not unreasonable if you are building one.

> If y'all want to rant and fear monger about any AI technology, including tech that has existed for years (deepfakes existed well before LLMs were mainstream)

And caused problems for years.

> That said, arguing with you people is pointless. You don't even seem to think.

Communication isn't a single-player game, I can't make you understand something you're actively unwilling to accept, like the idea that tools enable people to do more, for good and ill, and AI is such a tool.

Perhaps you should spend less time insulting people on the internet you don't understand. Go for a walk or something. Eat a Snickers, take a nap. Come back when you're less cranky.

You can say "fact" all you want but that doesn't make you correct lol

So, do you want future LLMs to be restricted, or unlimited? And remember, to prevent this outcome you have to predict model capabilities in advance, including "tricks" like prompting them to "think carefully, step by step".

To verify the LLM's code, because the LLM is cheaper than a human.

And there's a lot of live code already out there.

And people are only begrudgingly following even existing recommendations for code quality.

I code because I'm good at it, enjoy it, and it pays well.

I recommend against 3rd party libraries because they give me responsibility without authority — I'd own the problem without the means to fix it.

Despite this, they're a near-universal in our industry.

> If LLM hackers are rampant as you fear then people will respond by telling their code writing LLMs to get their shit together and check the code for vulnerabilities.

Eventually.

But that doesn't help with the existing deployed code — and even if it did, this is a situation where, when the capability is invented, attack capability is likely to spread much faster than the ability of businesses to catch up with defence.

Even just one zero-day can be bad, this… would probably be "many" almost simultaneously. (I'd be surprised if it was "all", regardless of how good the AI was).

Existing vulnerable code will be vulnerable, yes. We already live in a reality in which script kiddies trivially attack old outdated systems. This is the status quo, the addition of hacking LLMs changes little. Insofar as more systems are broken, that will increase the pressure to update those systems.

Edit: I misread that bit as "you code" not "your code".

But "your code because you own it", while a sound position, is a position violated in practice all the time, and not only because of my example of 3rd party libraries.

https://www.reuters.com/legal/transactional/lawyer-who-cited...

They are held responsible for being very badly wrong about what the tools can do. I expect more of this.

> You proposed a future in which every skiddy has a hacking LLM and they're using it to attack tons of stuff written by LLMs. If hacking LLMs and code writing LLMs both proliferate then the obvious resolution is for the code writing LLMs to employ hacking LLMs in verifying their outputs.

And it'll be a long road, getting to there from here. The view at the top of a mountain may be great or terrible, but either way climbing it is treacherous. Metaphor applies.

> Existing vulnerable code will be vulnerable, yes. We already live in a reality in which script kiddies trivially attack old outdated systems. This is the status quo, the addition of hacking LLMs changes little. Insofar as more systems are broken, that will increase the pressure to update those systems.

Yup, and that status quo gets headlines like this: https://tricare.mil/GettingCare/VirtualHealth/SecurePatientP...

I assume this must have killed at least one person by now. When you get too much pressure in a mechanical system, it breaks. I'd like our society to use this pressure constructively to make a better world, but… well, look at it. We've not designed our world with a security mindset, we've designed it with "common sense" intuitions, and our institutions are still struggling with the implications of the internet let alone AI, so I have good reason to expect the metaphorical "pressure" here will act like the literal pressure caused by a hand grenade in a bathtub.

Yes, indeed.

> and there will be very few remaining vulnerabilities for the black hat LLMs to exploit.

Sadly, this does not follow. Automated vulnerability scanners already exist, how many people use them to harden their own code? https://www.infosecurity-magazine.com/news/gambleforce-websi...

All of these are things that have already happened. These all were previously possible of course but now they are trivially scalable.

> chatbots that replace real-time human customer service but have none of the agency

That seems good for society, even though it's bad for people employed in that specific job.

Why?

It inserts yet another layer of crap you have to fight through before you can actually get anything done with a company. The avoidance of genuine customer service has become an artform by many companies and corporations, its demise surely should be lamented. A chatbot is just another in the arsenal of weapons designed to confuse, put-off and delay the cost of having to actually provide a decent service to you customers, which should be a basic responsibility of any public-facing company.

1. It's not "an extra layer", at most it's a replacement for the existing thing you're lamenting, in the businesses you're already objecting to.

2. The businesses which use this tool at its best, can glue the LLM to their documentation[0], and once that's done, each extra user gets "really good even though it's not perfect" customer support at negligible marginal cost to the company, rather than the current affordable option of "ask your fellow users on our subreddit or discord channel, or read our FAQ".

[0] a variety of ways — RAG is a popular meme now, but I assume it's going to be like MapReduce a decade ago, where everyone copies the tech giants without understanding the giant's reasons or scale

If I'm unlucky it'll just be another stage in the mobius-support-strip that directs me from support web page to chatbot to FAQ and back to the webpage.

The businesses which use this tool best will be the ones that manage to lay off the most support staff and cut the most cost. Sad as that is for the staff, that's not my gripe. My gripe is that it's just going to get even harder to reach a real actual person who is able to take a real actual action, because providing support is secondary to controlling costs for most companies these days.

Take for example the pension company I called recently to change an address - their support page says to talk to their bot, which then says to call a number, which picks up, says please go to your online account page to complete this action and then hangs up... an action which the account page explicitly says cannot be completed online because I'm overseas, so please talk to the bot, or you can call the number. In the end I had to call an office number I found through google and be transferred between departments.

An LLM is not going to help with that, it's just going to make the process longer and more frustrating, because the aim is not to resolve problems, it's to stop people taking the time of a human even when they need to, because that costs money.

Essentially businesses have (knowingly or otherwise) dropped their ability to provide meaningful customer support.

Even quite a lot of new chatbots are still in that paradigm, and… well, given the recent news about chatbot output being legally binding, it's precisely the extra agency of LLMs over both normal bots and humans following scripts that makes them both interestingly useful and potentially dangerous: https://www.bbc.com/travel/article/20240222-air-canada-chatb...

If you chat to an LLM and you get a picture back, which some support, the image generator and the language model might as well be the same thing to all users, even if there's an important technical difference for developers.

It's a distinction that does not matter, as the question still has to be answered for the other modality. Do guns kill people, or do bad guys use guns to kill people? Does a fall kill you, or is it the sudden deceleration at the end? Lab leak or wet market? There's a technical difference, some people care, but the actionable is identical and doesn't matter unless it's your job to implement a specific part of the solution.

To whom? And, as hard as this is to test, how sincerely?

> The American culture wars are not global. (They have their own culture wars).

Do people from places with different culture wars trust these American-culture-war-blinkered LLMs more or less than Americans do?

/ Rant /

Oh God please let these things be bottle necked. The job was already absurd, LLMs and GenAI are going to be just frikking amazing to deal with.

Spam and manipulative marketing has already evolved - and thats with bounded LLMs. There are comments that look innocuous, well written, but the entire purpose is to low key get someone to do a google search for a firm.

And thats on a reddit sub. Completely ignoring the other million types of content moderation that have to adapt.

Holy hell people. Attack and denial opportunities on the net are VERY different from the physical world. You want to keep a market place of ideas running? Well guess what - If I clog the arteries faster than you can get ideas in place, then people stop getting those ideas.

And you CANT solve it by adding MORE content. You have only X amount of attention. (This was a growing issue radio->tv->cable->Internet scales)

Unless someone is sticking a chip into our heads to increase processing capacity magically, more content isnt going to help.

And in case someone comes up with some brilliant edge case - Does it generalize to a billion+ people ? Can it be operationalized? Does it require a sweet little grandma in the Philippines to learn how to run a federated server? Does it assume people will stop behaving like people?

Oh also - does it cost money and engineering resources? Well guess what, T&S is a cost center. Heck - T&S reduces churn, and that its protecting revenue is a novel argument today. T&S has existed for a decade plus.

/ Rant.

Hmm, seems like I need a break. I suppose It’s been one of those weeks. I will most likely delete this out of shame eventually.

- People in other places want more controls. The Indian government and a large portion of the populace will want stricter controls on what can be generated from an LLM.

This may not necessarily be good for free thought and culture, however the reality is that many nations haven’t travelled the same distance or path as America has.

Anyone with forum moderating experience care to chime in ? (Reddit, while still on the open web for now, isn't a forum, and worse, is a platform.)

The answer is curation, and no, it doesn't need to scale to a billion people. maybe not even a million.

The sad fact of life is that most people don't care enough to discrminate against low quality content, so they are already a lost cause. Focus on those who do care enough and build an audience around them. You as a likely not billion dollar company can't afford to worry about that kind of scale, and lowering the scale helps you get a solution out for the short term. You can worry about scaling if/when you tap into an audience.

But yes- once you stop dropping constraints you can imagine all sorts of solutions.

It does work. I’m a huge advocate of it. When threads said no politics I wanted to find whoever made that decision and give them a medal.

But if you are a platform - or a social media site - or a species.?

You can’t pick and choose.

And remember - everyone has a vote.

As good as your community is, we do not live in a vacuum. If information wars are going on outside your digital fortress, it’s still going to spill into real life

After reading the rest of your rant (I hope you keep it) ... maybe free thought and culture aren't what LLMs are for.

https://x.com/tlalexander/status/1765122572067434857

These systems are making the internet a worse place to be.

Reliable information and communication will soon be opt-in only. The "open internet" will become an eclectic collection of random experiences, where any real human output will be a pleasant, rare surprise, that stirs the pot for a short blip before it is assimilated and buried under the flood.

It is under threat now, due to the increased sophistication of spam.

Similar in my current job btw.

One’s nephew might not interact in a professional capacity with other organizations, but many do.

Even our clients use slack to connect, it is many times better then emails.

B2B probably still uses email, but that's terrible because of the phishing threat. Professional communication should use webapps plus notification pings which can be app notifications, emails, or SMS.

People to people at different organizations again fall back to texting as an informal channel.

I have sent 23 emails already today to 19 people in 16 organizations.

I'm trying to think of 1 single thing email does better than slack in a corporate communication.

Is slack perfect? Absolutely not. I don't care that I can't use any client I want or any back end administration costs or hurdles. As a user, there is no comparison.

The thing I hate the most is that people expect to only use Slack for everything, even where it doesn't make sense. So they will miss notifications for Notion, Google Doc, Calendar, Github, because they don't have proper email notifications set up. The plugins for Slack are nowhere near as good as email notifications as they just get drowned out in the noise.

And with all communication taking place in Slack, remembering what was decided on with a certain issue becomes impossible because finding anything on Slack is impossible.

But no better than email again. You just get notified if you have a email or not [depending on the client you're forced to use]. I have many email rules that just filter out nonsense from github, google and others so my inbox stays clean.

I guess I find them useful because that way I get all my notifications across all channels in one spot if you set them up properly. Github and google docs also allow you to reply directly within the email, so I don't even need to open up website.

In Slack the way it was setup was global, so I got notified for any update even if it didn't concern me.

- Long form discussion where people think before responding.

- Greater ability to filter/tag/prioritize incoming messages.

- Responses are usually expected within hours, not minutes.

- Email does not have a status indicator that broadcasts whether I am sitting at my desk at that particular moment to every coworker at my company.

#2 I agree with you, again Slack is not perfect but better than email [my opinion]

#4 Slack has status updates, email does not. So you can choose to turn this off, again boundaries.

Regarding #4, I can't turn off my online/offline indicator, I can only go online or offline. I can't even set the time it takes to go idle. These are intentionally not configurable in slack. I have no desire to have a real time status indicator in the tool I use to communicate with coworkers.

There are certain actions I have to use email for, and it feels a little bit more like writing a check each year.

And all these email subscriptions, jeez, I don't want that. People who are brilliant and witty and information on Xitter or Mastodon in little slices and over time I still don't want to sit down and read a long form thing once a week.

In a battle between AI and teenage human ingenuity, I'll bet my money on the teenagers. I'd even go so far as to say they may be our only hope!

Search may be reduced to nigh uselessness, but the saavy will still be able to share quality information as needed. AI may even assist in that with people who have the domain knowledge to properly correct the prompts and rigorously proofread. How we find that quality information may, once again, be through closed off channels.

I’m not really saying that, and haven’t put a lot of thought in to my views there. But when people say the biggest problem is the controls on the AI, I feel compelled to point out that the destruction of the open internet is happening despite these controls, and is a major problem unto itself.

But when that GAN passes the human threshold, suddenly you can use the faker to create interesting things and not just use the discriminator to reject fakes.

I can't believe this went all the way to AI...

You might as well bemoan gravity.

In other words, it's very likely generative AI will be very good at creating fake simulacra of reality, and very unlikely it will actually be good AGI. The worst possible outcome.

The older generations are no better. While ProPublica or WSJ put effort into their investigative journalism, they can’t compete with the volume of trite commentary coming out of other MSM sources.

Generative AI poses no unique threat; society’s capacity to “think once and cut twice” will remain in tact.

While the threat isn't unique the magnitude of the threat is. This is why you can't argue in court the threat of a car crash is nothing unique even when you're speeding vs driving within limit.

But a car going into a stroller at 150mph versus 200mph is negligible.

The democratization of generative AI would increase the number of bad agents, but with it would come a better awareness of their tactics; perhaps we push less strollers into the intersections known for drag racing.

I guess when you distort every argument to an absurd you can claim you're right.

> but with it would come a better awareness of their tactics

I don't follow. Are you saying new and more sophisticated ways to scam people are actually good because we have a unique chance to know how they work ?

Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness.

AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected in the short and long term, so using “scamming” as a means to oppose disseminating tech feels reductive at best.

It is, because I wasn't directly comparing AI to traffic but only reaching for an example to illustrate how irrelevant is the case whether the threat is something completely unique or not.

> Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness.

Dismissing it as "meh, not new" is plain silliness.

> AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected

What on Earth makes you think that ? The beautiful way we're handling scams right now ? If you think it's irrelevant that phishing via phone call can now or soon be fully automated and the attack may even be conducted using a copy of someone's voice - well, we won't get anywhere here.

The way we mitigate scams today definitely encourages me; the existence of victims does not imply the failure or inadequacy of safeguards keeping up with technology.

While AI stokes the imagination, it’s not so inspiring that I can make the argument in my head for you about why humanity’s better off with access to these tools being kept in the hands of corporations that repeatedly get sued for placing profits over public welfare.

Ok, now you're being just stubborn. No, no one is manually dialing your number but as soon as the scammer knows you've answered you get to talk to a human who tries to convince you to install a "safety" app for your bank or something. THAT part isn't automated, but it may as well be, which means phishing calls and scams can potentially be done with a multiplication factor of hundreds, maybe thousands - limited only by scammer infrastructure.

It could be the return of real journalism. There is a lot to rebuild in this respect, as most journalism has gone to the dogs in the last few decades. In my country all major newspapers are political pamphlets that regularly publish fake news (without the need for any AI). But one can hope, maybe the lowering of the barrier of entry to generate fake content will make people more critical of what they read, hence incentivizing the creation of actually trustworthy sources.

It's not like AI invented clickbait, though it might have mastered the art of it.

The convincing bullshit problem does not stem from AI, I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2.

To put it differently, the problem isn't that AI will be great at writing 100 pages of bullshit you'll need to scroll through to get to the actual recipie, the problem is that there was an incentive to write those pages in there first place. Personally I don't care if a human or a robot wrote the bs, in fact I'm glad one fewer human has to waste their time doing just that. Would be great if cutting the bs was a more profitable model though.

Personally, I highly dislike this handwaving of SEO. SEO is not some sinister agenda following secret cult trying to disseminate bullshit. SEO is just... following the rules set forth by search engines, which for quite a long time is effectively singlehandedly Google.

Those "weird and unexpected incentives" are put forth by Google. If Google for whatever reason started ranking "vegetable growing articles > preparation technique articles > recipes > shops selling vegetables" we would see metaphorical explosion of home gardening in mere few years, only due to the relatively long lifecycles inherent in gardening.

SEO is gaming Google's heuristics. Google doest generate a perfect ranking according to Google human's values.

SEO gaming is much older than Google. Back when "search" was just an alphabetical listing of everyone in a printed book, we had companies calling themselves "A A Aachen" to get to the front of the book.

I fail to see immediate disagreement here: I don't see how Google's ranking process/method/algorithm being heuristic changes the observation that to a website in the end it is a set of ranking rules, that can in some ways be gamed. SEO is two part process: discovering those ranking rules and abusing them.

Your example with gaming alphabetical listings only reinforces the idea that SEO abuses rules set forth by the ranking engine.

However, it does not meaningfully matter whether the incentives inherent in the ranking system form results the way they do intentionally. What matters is the eventual behavior of the ranking system. Mostly because by definition you cannot filter out bad actors entirely, all you can do is 1) place some arbitrarily enforced barriers, which are generally prohibitively costly 2) place incentives minimizing gain of bad actors.

To clarify, Google defines the metrics by which pages are ranked in their search results, and since everyone want to be at the top of Google's search results, those metrics immediately become targets for everyone else.

It's quite clear to me that the metrics Google have introduced over the year have been meant to improve the quality of the results on their search. It's also clear to me that they have, in actual fact, had the exact opposite effect, namely that recipes are now prepended with a poorly written novella about that one time the author had a emotionally fulfilling dinner with love ones one autumn, in order to increase time spent on the page, since Google at one point quite reasonably thought that pages where visitors stay longer are of higher quality, otherwise why did visitors stay so long?

And if no one is using the search engine, who's gonna see all those ads?

Of course, they do have other incentives too, some of which are directly conflicting with high quality search results, as you point out.

I suppose one could argue that their near-monopoly in the search business has allowed them to be somewhat negligent on the quality of search, but now that there are a few competitors at least somewhat worthy of the name, one can hope high quality results will be a higher priority.

Anyway, I'm holding out hope someone will one day manage to train a LLM to distinguish between quality content and SEO bullshit, and then put that to use in a search engine.

I'm not well versed enough in the current status of LLM's to make a prediction on how hard that will be, but my impression is that we're a fair ways off from any LLM being able to do that well enough to be valuable.

I'd really love to be proven wrong on this one, if you're reading this and have some relevant experience, consider yourself challenged! (feel free to rephrase this last bit in your head to whatever motivates you the most)

you have no evidence whatsoever that llama models are being used for that purpose. meanwhile, twitter is full of bots posting GPT refusals.

That’s a real issue but I doubt the solution is technical. Society will have to educate itself on this topic. It’s urgent that society understand rapidly that LLMs are just word prediction machines.

I use LLMs everyday, they can be useful even when they say stupid things. But mastering this tool requires that you understand it may invent things at any moment.

Just yesterday I tried the Cal.ai assistant which role is to manage your planning (but it don’t have access to your calendars that’s pretty limited). You communicate with it by mail. I asked him to organise a trip by train and book an hotel. It responded, « sure what is your preferred time for the train and which comfort do you want ? » I answered and it answered back that, fine, it will organise this trip and reach me back later. It even added that it will book me an hotel.

Well, it can’t even do that, it’s just a bot made to reorganize your cal.com meetings. So it just did nothing, of course. Nothing horrible since I know how it works.

But would I have been uneducated enough on the topic (like 99,99% of this planet’s population, I’d just thought « Cool, my trip is being organized, I can relax now ».

But hey, it succeeded at the main LLM task : being credible.

Does anyone, even the most stereotypical hn SV techbro, thing this kind of thing? That's preposterous.

I would prefer things were open, but I don’t think this is the best argument for that

Yes, operators trying to tame their models for public consumption inevitably involves trade offs and missteps

But having hundreds or thousands of equivalent models being tuned to every narrow mindset is the alternative

I would prefer a midpoint, I.e. open but delayed disclosure

Take time to experiment and design in safety, etc. also to build a brand that is relatively trusted (despite the inevitable bumps) so ideologically tuned progeny will at least be competing against something better, and more trusted, at any given time

But the problem of resource requirements is real, so not surprising that being clearly open is challenging

*Falsify reality.

Most of the idiocy surrounding the "chatbot peril" comes from conflating these things. If an LLM learns to predict that the pronoun token for "doctor" is "he", this is not a claim about reality (in reality doctors take at least two personal pronouns), and it certainly isn't a moral claim about reality. It's a bare consequence of the training data.

The problem is that certain activist circles have decided that some of these predictions have political consequences, absurd as this is. No one thinks it consequential that if you ask an LLM for an algorithm, it will give it to you in Python and Javascript, this is obviously an artifact of the training set. It's not like they'll refuse to emit predictive text about female doctors or white basketball players, or give you the algorithm in C/Scheme/Blub, if you ask.

All that the hamfisted retuning to try and produce an LLM which will pick genders and races out of a hat accomplishes is to make them worse at what they do. It gets in the way of simple tasks: if you want to generate a story about a doctor who is a woman and Ashanti, the race-and-gender scrambler will often cause the LLM to "lose track" of characteristics the user specifically asked for. This is directly downstream of trying to turn predictions on "doctor" away from "elderly white man with a kindly expression, wearing a white coat and stethoscope" sorts of defaults, which, to end where I started, aren't reality claims and do not carry moral weight.

The problem is... keeping them closed source isn't helping with that problem, it only serves to guarantee OpenAI a cut of the profits caused by the spam and scams.

Luckily we already have the tools for this, NFT in the case of media and DKIM in the case of your spam email.

I have no idea how you imagine "NFTs" will save us though. To me, that sounds like buzzword spam on your part.

If Taylor Swift signs ownership of a picture of her, you can know it is what she presents as real. If Elon musk signs a YouTube video of him offering crypto doubling giveaways, you can know he intended to represent the message as real. If the New York Times publishes an article signed with their key you can know it is meant to be published by the New York Times.

2) Public/private key cryptography would give us a way to verify that a message (or picture or whatever) from Taylor Swift is signed with Swift's private key, but it wouldn't help at all with e.g telling me that I'm responding to a real message and not a bot right now. Not to mention that it wouldn't even help much against deep fakes, since if I publish what I claim to be a secret recording of Swift, there would be no reason to expect her to have signed it with his private key.

Those two are the hallmarks of most of these "legit use cases for cryptocurrencies/NFTs" suggestions I've heard from cryptobros by the way: they're always some combination of "old technology that has nothing to do with cryptocurrencies" and "doesn't actually solve the problem".

Is content generation really the thing holding spammers back? I haven't seen a huge influx of more realistic spam so I wonder your basis for this statement.

If anything it's just the same regular spam that gets deleted and ignored at the bottom of threads.

Easier content generation doesn't solve the reputation problem that social media demands in order to get attention. The whole LLM+spam thing is mostly exaggerated because people don't understand this fact. It merely creates a slightly harder problem for automatic text analysis engines...which was already one of the weaker forms of spam detection full of false positives and misses. Everything else is network and behaviour related, with human reporting as last resort.

Everyone who disagrees with the hivemind of a subreddit gets accused of being one of those things and any attempt to dispute the claim gets you banned. The internet of today sucks because people are so obsessed with those 3 things that they're the first conclusion people jump to on pseudoanonymous social media when they have no other response. They'll crawl through your controversial comments just to provide proof that you can't possibly be serious and you're being controversial to play an internet villain.

I'd love to know how you dispute the claim that "you're parroting Russian troll talking points so you must be a Russian troll" when it's actually the Russian trolls parroting the sentiments to seem like real people.

There are articles written on LLM spam, such as this one: https://www.theverge.com/2023/4/25/23697218/ai-generated-spa.... Those are probably going to substantiate this problem better than I would.

E.g. Finding those dangers and them being public and publicly known is the better of 2 options vs. only bad actors potentially having them.

you are assuming that their core mission is to "Build an AGI that can help humanity for free and as a non-profit", the way their thinking seems to be is "Build an AGI that can help humanity for free"

they figured it was impossible to achieve their core mission by doing it in a non-profit way, so they went with the for-profit route but still stayed with the mission to offer it for free once the AGI is achieved

Several non-profits sell products to further increase their non-profit scale, would it be okay for OpenAI non-profits to sell products that came in the process of developing AGI so that they can keep working on building their AGI? museums sell stuff to continue to exist so that they can continue to build on their mission, same for many other non-profits. the OpenAI structure just seems to take a rather new version of that approach by getting venture capital (due to their capital requirements)

The easiest way to cut through corporate BS is to find distinguishing characteristics of the contrary motivation. In this case:

OpenAI says: To deliver AI for the good of all humanity, it needs the resources to compete with hyperscale competitors, so it needs to sell extremely profitable services.

Contrary motivation: OpenAI wants to sell extremely profitable services to make money, and it wants to control cutting edge AI to make even more money.

What distinguishing characteristics exist between the two motivations?

Because from where I'm sitting, it's a coin flip as to which one is more likely.

Add in the facts that (a) there's a lot of money on the table & (b) Sam Altman has a demonstrated propensity for throwing people under the bus when there's profit in it for himself, and I don't feel comfortable betting on OpenAI's altruism.

PS: Also, when did it become acceptable for a professional fucking company to publicly post emails in response to a lawsuit? That's trashy and smacks of response plan set up and ready to go.

And based on how they have acted in the past, how much do you trust they will act as they now say when/if they achieve AGI?

If this is the case, then they should be more open with their older models such as 3.5, I'm very sure industry insiders actually building these already know the fundamentals of how it works.

The interesting part of this is that whatever rights Microsoft has do not extend to any OpenAI model/software that is deemed to be AGI, and it seems they must therefore have agreed how this would be determined, which would be interesting to know!

There was a recent interview of Shane Legg (DeepMind co-founder) by Dwarkesh Patel where he gave his own very common sense definition of AGI as being specifically human-level AI, with the emphasis on general. His test for AGI would be to have a diverse suite of human level cognitive tasks (covering the spectrum of human ability), with any system that could pass these tests then being subject to ad hoc additional testing. Any system that not only passed the test suite but also performed at human level on any further challenge tasks might then reasonably be considered to have achieved AGI (per this definition).

I would argue the opposite. Keeping AGI behind a walled corporate garden could be the most dangerous situation imaginable.

The present entirely depends on books and equivalent electronic media. The future will depend on AI. So anyone who has a monopoly is going to be able to extract massive monopoly rents from its customers and be a net negative to the society instead of the positive they were supposed to be.

They do. They say:

> Elon understood the mission did not imply open-sourcing AGI. As Ilya told Elon: “As we get closer to building AI, it will make sense to start being less open. The Open in openAI means that everyone should benefit from the fruits of AI after its built, but it's totally OK to not share the science...”

Whether you agree with this is a different matter but they do state that they did not betray their mission in their eyes.

Of course they can give us nothing, but in that case they should start paying taxes and stop claiming they're a public benefit org.

My prediction is they'll produce little of value going forward. They're too distracted by their wet dreams about all the cash they're going to make to focus on the job at hand.

There is always someone smarter than you.

There is always someone stronger than you.

There is always someone richer than you.

There is always someon X than Y.

This is applicable to anything, just because OpenAI has a lead now it doesn't mean they will stay X for long rather than Y.

Even if that science helps not so friendly countries like Russia?

It's the regular people that will be left out.

As we saw previously it doesn't matter much if you are top notch ai researcher, if 1-2 millions of your potential personal wealth are in stake this affect decision making (and probably would mine too).

How much of a bribe would it take for anybody inside with good enough access to switch sides and take all the golden eggs out? 100 million? A billion? Trivial amounts compared to what we discuss. And they will race each other to your open arms for such amounts.

We see sometimes recently ie government officials betraying their own countries to russian spies in Europe for few hundred - few thousands of euros. A lot of people are in some way selfish by nature, or can be manipulated easily via emotions. Secret services across the board are experts in that, it just works(tm).

To sum it up - I don't think it can be protected long term.

This means I have no idea how to even think about people who could be bribed when they already earn a million a year.

But also, if AI can be developed as far as the dreamers currently making it real hope it can be developed, money becomes as useless to all of us as previous markers of wealth like "a private granary" or "a lawn" or "aluminium cutlery"[0].

[0] https://history.stackexchange.com/questions/51115/did-napole...

Nothing will stop this wave, and the United States will not allow itself to be on the sidelines.

OpenAI gets to decide what it does with its intellectual property for the same reason that a whole bunch of people are suing it for using their intellectual property.

It only becomes repugnant to me if they're forcing their morals onto me, which they aren't, because (1) there are other roughly-equal-performance LLMs that aren't from OpenAI, and (2) the stuff it refuses do is a combination of stuff I don't want to exist and stuff I have a surfeit of anyway.

A side effect of (1) is that humanity will get the lowest common (moral and legal) denominator in content from GenAI from different providers, just like the prior experience of us all getting the lowest common (moral and legal) denominator in all types of media content due to internet access connecting us to other people all over the world.

everyone... except scientists and the scientific community.

Personally I find the comparison of this whole saga (deepmind -> google —> openai —> anthropic —-> mistral —-> ?) to the Manhattan project very enlightening, both of this project and our society. Instead of a centralized government project, we have a loosely organized mad dash of global multinationals for research talent, all of which claim the exact same “they’ll do it first!” motivations as always. And of course it’s accompanied by all sorts of media rhetoric and posturing through memes, 60-Minutes interviews, and (apparently) gossipy slap back blog posts.

In this scenario, Oppenheimer is clearly Hinton, who’s deep into his act III. That would mean that the real Manhattan project of AI took place in roughly 2018-2022 rather than now, which I think also makes sense; ChatGPT was the surprise breakthrough (A-bomb), and now they’re just polishing that into the more effective fully-realized forms of the technology (H-bomb, ICBMs).

Do you think they thought they were good guys because you watched a Hollywood movie?

I'm not taking a stance here, but it's easy to see why many Americans believed developing the atomic bomb was a net positive at least for Americans, and depending on how you interpret it even the world.

The scientists who continued after that (not all did) must have had some other motivation at that point.

Morals are hard in real life, and sometimes really fuzzy.

Long story short they claim they thought the bomb was impossible, but it was still a large matter of concern for him as he worked on nuclear power. The most interesting tidbit was that Heisenberg was in a small way responsible for (west) Germany’s ongoing ban on nuclear weapons, which is a slight redemption arc.

Wernher vin Braun on the other hand got lauded for his contribution to space exploration. His development of the V2 and his use of slave labour in building them was somehow just a minor disgression for the, ultimately under US leadership, greater good.

https://www.smbc-comics.com/comic/status-2

Japan and the Soviet Union were more complicated issues for some of the scientists. But that's what happens with warfare. You develop new weapons, and they aren't just used for one enemy.

Say rather that he's apolitical.

"Once the rockets are up, who cares where they come down?

That's not my department, " says Wernher von Braun.

EDIT: tried to take the onus off you, but as usual history is more complicated than I expected. Clearly I know nothing because I had no idea of the scope:

  At its peak, it employed over 125,000 direct staff members, and probably a larger number of additional people were involved through the subcontracted labor that fed raw resources into the project. Because of the high rate of labor turnover on the project, some 500,000 Americans worked on some aspect of the sprawling Manhattan Project, almost 1% of the entire US civilian labor force during World War II.

Nah. They knew they were working for their side against the other guys, and were honest about that.

> As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world

https://openai.com/blog/introducing-openai

https://www.sec.gov/Archives/edgar/data/1288776/000119312504...

""" DON’T BE EVIL

Don’t be evil. We believe strongly that in the long term, we will be better served—as shareholders and in all other ways—by a company that does good things for the world even if we forgo some short term gains. This is an important aspect of our culture and is broadly shared within the company.

Google users trust our systems to help them with important decisions: medical, financial and many others. Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating. We also display advertising, which we work hard to make relevant, and we label it clearly. This is similar to a newspaper, where the advertisements are clear and the articles are not influenced by the advertisers’ payments. We believe it is important for everyone to have access to the best information and research, not only to the information people pay for you to see. """

Open could mean the science, the code/ip (which includes the science) or pure marketing drivel. Sadly it seems that it's the latter.

As they become more successful, they (obviously) have a lot of motivation to not be "open" at all, and that's without even considering the so-called ethical arguments.

More generally, putting "open" in any name frequently ends up as a cheap marketing gimmick. If you end up going nowhere it doesn't matter, and if you're wildly successful (ahem) then it also won't matter whether or not you're de facto 'open' because success.

Maybe someone should start a betting pool on when (not if) they'll change their name.

It’s a made up word.

So the Open in OpenAI means whatever OpenAI wants it to mean.

It’s a trademarked word.

The fact that Elon is suing them for their name when the guy has a feature “AutoPilot” which is not a made up word and had an actual well understood meaning which totally does not apply to how Tesla uses AutoPilot is hilarious.

Apparently, their internal definition was different from the very beginning (2016). The only problem with their (Ilya's) definition of 'open' is that it is not very open. "Everyone should benefit from the fruits of AI". How is this different than the mission of any other commercial AI lab? If OpenAI makes the science closed but only their products open, then 'open' is just a term they use to define their target market.

A better definition of OpenAi's 'open' is that they are not a secret research lab. They act as a secret research lab, but out in the open.

Other than the vehicle, this would seem to apply to Tesla's autopilot as well. The "Full Self Driving" claim is the absurd one, odd that you didn't choose that example.

I happen to think that open sourcing frontier models is a bad idea but OpenAI put themselves in the position where people thought they stood for one thing and then did something quite different. Even if you think such a move is ultimately justified, people are not usually going to trust organizations that are willing to strategically mislead.

They really need to change their name and another entity that actually works for open AI should be set up.

“The Democratic People's Republic of Korea”

(AKA North Korea)

In a way this could be more closed than for profit.

That passes for an explanation to you ? What exactly is the difference between openai and any company with a product then ? Hey, we made THIS and in order to make sure everyone can benefit we sell at a price of X.

In 2016, OpenAI's website said this right up front:

> We're hoping to grow OpenAI into such an institution. As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We'll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies.

I don't know how this quote can possibly be squared with a claim that they "did not imply open-sourcing AGI".

This would mean it is fundamentally just a business with extra steps. At the very least, the "foundation" should be paying tax then.

2. Natural language is both a good and expected interface to AGI.

3. LLMs do a really good job at interfacing with natural language.

Which one(s) do you disagree with?

4. Language prediction training will not get stuck in a local optimum.

Most previous things we train on could have been better served if the model developed AGI, but they didn't. There is no reason to expect LLMs to not get stuck in a local optimum as well, and I have seen no good argument as to why they wouldn't get stuck like everything else we tried.

How much more data is available that hasn't already been swept up by AI companies?

And will that data continue to be available as laws change to protect copyright holders from AI companies?

Is there any reason to think the same thing wouldn't happen in billion parameter LLMs?

https://arxiv.org/abs/2311.00871

https://arxiv.org/abs/2309.13638

https://arxiv.org/abs/2311.09247

https://arxiv.org/abs/2305.18654

https://arxiv.org/abs/2309.01809

Whether that's true or not, I don't think that's what the previous post was referring to. The question is, if you start with today's LLMs and progressively improve them, do you arrive at AGI?

(I think it's pretty obvious the answer is no -- LLMs don't even have an intelligence part to improve on. A hypothetical AGI might somehow use an LLM as part of a language interface subsystem, but the general intelligence would be outside the LLM. An AGI might also use speakers and mics but those don't give us a path to AGI either.)

While it's kind of nuts how far OpenAI pushed language models, even as an outside observer it's obvious that OpenAI is not banking on LLMs achieving AGI, contrary to what the person I was replying to said. Lots of effort is being put into integrating with outside sources of knowledge (RAG), outside sources for reason / calculation, etc. That's not LLMs as AGI, but it is LLMs as a step on the path to AGI.

But the algorithms are black box to me, so maybe there is some kind of launch pad to AGI within it

They claimed that about GPT-2 and used the claim to delay its release.

<aisafety>That is the one thing we didn't want to happen.</aisafety>

It's one thing to mess around with Starcraft or DotA and wow the gaming world, it's quite another to be riding the escalator to the eschaton.

Concentrating an extremely powerful tool, what it will and won't do, who has access to it, who has access to the the newest stuff first? Further corrupting K Street via massive lobbying/bribery activity laundered through OpenPhilanthropy is just trivially terrifying.

That is a clear and present danger of potentially catastrophic importance.

We stop the bleeding to death, then worry about the possibly malignant, possibly benign lump that may require careful surgery.

  here they explain why they had to betray their core mission. But they don't refute that they did betray it.

It’s a shame lawyers seem to think that the lawsuit is a badly argued joke, because I really don’t find that line of reasoning convincing…

its because it is a badly argued joke. The founding charter is just that, a charter not a contract:

> the corporation will seek to open source technology for the public benefit when applicable

There are two massive caveats in that statement. wide enough to drive a stadium through.

Elon is just pissed, and is throwing lawyers at it in the hopes that they will fold (A lot of cases are settled out of court, because its potentially significantly cheaper, and less risky.)

The problem for Musk is that he is fighting with company who also is rich enough to afford good lawyers for a long time.

Also, he'll have to argue that he has materially been hurt by this change, again really hard.

last of all, its a company, founding agreements are not law, and often rarely contracts.

That's clearly self-serving claptrap. It's a leveraging of a false depiction of what AGI will look like (no ones really knows, but it's going to be scary and out of control!) with so much gatekeeping and subsequent cash they can hardly stop salivating.

No strong AI (there is no evidence AGI is even possible) is not going to be a menace. It's software FFS. Humans are and will be a menace though and logically the only way to protect ourselves from bad people (and corporations) with strong AI is to make strong AI available to everyone. Computers are pretty powerful (and evil) right now but we haven't banned them yet.

Reading this is like hearing "there is no evidence that heavier-than-air flight is even possible" being spoken, by a bird. If 8 billion naturally-occuring intelligences don't qualify as evidence that AGI is possible, then is there anything that can qualify as evidence of anything else being possible?

If the US had developed a nuclear armament and no other country had would that truly have been worse? What if Russia had beat the world to it first? Maybe I'll get there on my own if I keep following this reasoning. However there is nothing clear cut about it, my strongest instincts are only heuristics I've absorbed from somewhere.

What we probably want with any sufficiently destructive potential-weapon are the most responsible actors to share their research while stimulating research in the field with a strong focus on safety and safeguarding. I see some evidence of that.

https://en.wikipedia.org/wiki/Mutual_assured_destruction

Yes, do you think it is a coincidence that nuclear weapons stopped being used in wars as soon as more than one power had them? People would clamor for nukes to be used to save their young soldiers lives if they didn't have to fear nuclear retaliation, you would see strong political pushes for nuclear usage in everyone of USA's wars.

have you even watched Terminator? ;)

So the AGI existential threat to humanity has diminished?

But, my God, the some of the nonprofit ceos I’ve known make the for-profit ceos look pathetic and cheap.

I can't tell if your comment is intentionally misleading or just entirely missing the point. The entire post states that Elon musk was well aware and onboard with their intentions. Tried to take over OpenAI and roll it into his private company to control. And finally agreed specifically that they need to continue to become less open over time.

And your post is to play Elon out to be a victim who didn't realize any of this? He's replying to emails saying he's agreeing. It's hard to understand why you posted something so contradictory above pretending he wasn't.

Too dangerous? Seriously? Who the fuck did/do they think they are? Jesus?

Sam Altman is going to sit there in his infinite wisdom and be the arbiter of what humanity is mature enough to handle?

The amount of kool aid that is being happily drank at openai is astounding. It’s like crypto scams but everyone has a PhD.

Why? This makes it seem like computers are way less efficient than humans. Maybe I'm naive on the matter, but I think it's possible for computers to match or surpass human efficiency.

And the part of the human brain that governs our human intelligence and not just what animals do is much larger than, so unless we figure out a better algorithm than evolution did for intelligence it will require a massive amount of compute.

The brain isn't fast, but it is ridiculously parallel with every cell being its own core so total throughput is immense.

A typical chip requires more power than a human brain, so I'd say they are comparable. Efficiency isn't per volume but per power or per heat production. Human brains wins those two by far.

edit: Another major contributing factor is that so far, chips are more bottlenecked on production than operation. Almost any female human can produce more humans using onboard technology. Comparatively, first-rate chips can be made in like three buildings in the entire world and they each cost billions to equip. If we wanted to build a brain with photolithography, we'd need to rent out TSMC for a lot longer than nine months. That results in a much bigger focus on peak performance. We have to go "high" because we cannot practically go "wide".

If donors are unwilling to continue making sustained donations, they would have died. They only did what they needed to to stay alive.

May help with technological advancement, but not social or ethical advancement.

And look at that, suddenly news searches are plastered with stories about this...

https://www.google.com/search?q=openai+board&tbm=nws

Who could have possibly forseen that 'openai' + 'musk' + emails would chum the waters for a news cycle? Certainly not a PR firm.

Is that still true? LLMs seem to be getting smaller and cheaper for the same level of performance.

Eh.

Humans have about 1.5 * 10 ^ 14 synapses (i.e connections between neurons). Assume all the synapses are firing (highly unlikely to be the case in reality), and the average firing speed is 0.05ms (there are chemical synapses that are much slower, but we take the fastest speed of the electrical synapses).

Assume that each synapse is essntially a signal that gets attenuated somehow in transmission. I.e value times a fractional weight, which really is a floating point operation. That gives us (1.5 * 10 ^14)/(0.0005)/(10 ^ 12)) = 300000 TFLOPS

Nvidia 4090 is capable of 1300 Tflop of fp8. So for comparable compute, we need 230 4090s, which is about $345k. So with everything else on board, you are looking at $500k, which is comparatively not that much money, and thats consumer pricing.

The biggest expense like you said is paying salaries of people who are gonna figure out the right software to put on those 4090s. I just hope that most of them aren't working on LLMs.

Right now, the idea is that every time you build an LLM, you start the training from scratch, because thats all we know how to do.

Human like AI will most definitely not be trained like that. Humans can look at a piece of information once or twice and remember it.

Just like the attention paper, at some point someone will publish a paper that describes a methodology of feeding one piece of data back through the network only a few times to fully train it.

They don't refute that, but they claim that road was chosen in agreement with Elon. In fact, the claim this was his suggestion

> “The Open in openAI means that everyone should benefit from the fruits of AI after its [sic] built, but it's totally OK to not share the science...”, to which Elon replied: “Yup”.

Well, nope. This is disingenuous to the point of absurdity. By that measure every commercial enterprise is "open". Google certainly is extremely open, as are Apple, Amazon, Microsoft... or Wallmart, Exxon, you name it.

Why would they change their mission? If achieving the mission requires money then they should figure out how to get money. Non-profit doesn't actually mean that the corporation isn't allowed to make profit.

Why change the name? They never agreed to open source everything, the stated mission was to make sure AGI benefits all of humanity.

"Pournelle's Iron Law of Bureaucracy states that in any bureaucratic organization there will be two kinds of people: First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration. Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc. The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization." [1] https://en.wikipedia.org/wiki/Jerry_Pournelle#:~:text=Anothe....

I don't follow your reasoning. The goal of the company is AGI. To achieve AGI, they needed more money. What about that says the company comes before the goals?

Today’s OpenAI is very much driven by considerations of financial returns, and the goal of “most likely to benefit humanity as a whole” and “positive human impact” doesn’t seem to be the driving principle anymore.

Their product and business strategy is now governed by financial objectives, and their research therefore not “free from financial obligations” and “unconstrained by a need to generate financial return” anymore.

They are thus severely compromising their alleged mission by what they claim is necessary for continuing it.

But it seems like everyone agreed that they'd need a lot of money to train an AGI.

It seems pretty clear that they felt they had to choose between chasing money and shutting down. I'm guessing you'd prefer they went with the latter, but I can entirely understand why they didn't.

Among many other more technical reasons, this is a great demonstration of why AI "alignment" as it is often called is such a terrifying unsolved problem. Human alignment isn't even close to being solved. Hoping that a more intelligent being will also happen to want to and know how to make everyone happy is the equivalent of hiding under the covers from a monster. (The difference being that some of the smartest people on the planet are in furious competition to breed the most dangerous monsters in your closet.)

This doesn't begin to make sense to me. Nothing about being a non-profit prevents OpenAI from raising money, including by selling goods and services at a markup. Some sell girl-scout cookies, some hold events, etc.

So, you can't issue equity in the company... offer royalties. Write up a compensation contract with whatever formula the potential employee is happy with.

Contract law is specifically designed to allow parties to accomplish whatever they want. This is an excuse.

Even when the EA community was flush with crypto billionaires there was no appetite for this level of spend.

The Novo Nordisk foundation has a $120 Billion endowment, the Bill & Melinda Gates Foundation has $50 Billion, the Welcome Trust has $42 Billion, ... There are about 19 charities that still have more than $10 Billion, let alone those who have raised (and spent) that during their existence.

https://en.wikipedia.org/wiki/List_of_wealthiest_charitable_...

> an uncommonly huge raise even among _for-profit_ companies where investors expect returns.

So, offer a return on investment. Charities that take loans pay interest. Charities that hire staff pay salaries. What's with this idea that charities cannot pay a reasonable market rate for services (e.g. short-term funding)?

Because:

> So, offer a return on investment

This is precisely what they did; Microsoft’s investment is an extremely funky capped profit structure. They did it this way to minimize their cost of capital.

I’m not really clear what your concrete proposal is for raising $10b as a non-profit, perhaps you could flesh that out?

If you’re talking about financing a potentially decade- long project on tens of billions of dollars of pure debt, again that is… not a feasible structure.

OpenAI, Inc. (the nonprofit) could have partnered with Microsoft directly.

To be fair, maybe Microsoft may have required that certain code be kept secret in a way that OpenAI's charitable purpose would not have allowed. However, that would just suggest that the deal was not open and not in the best interests of the charity.

Moreover, I'm skeptical that OpenAI Global LLC paid fair market value to OpenAI, Inc. for the assets it received. Sure, the GPT-2 itself was open sourced, but a lot of the value of the business lied in other things: all of the datasets that were used, the history of training, what worked and what didn't work, the accessory utilities, emails, documents / memos, the brand, etc. The staff is a little tricky, because - sure - they are ostensibly free to leave, but there's no doubt there's a ton of value in the staff.

If OpenAI, Inc. (non-profit) put itself on the open market with the proceeds to go to another charity, what do you think Microsoft would have paid to buy the business? I bet it would have been a lot more than OpenAI Global LLC paid to OpenAI, Inc for the same assets...

What can we do?

Except there's a proof point that it's not impossible: philanthropists like Elon Musk - who would have likely kept pumping money into it, and where arguably the U.S. and other governments would have funded efforts - energy and/or CPU time - as a military defense strategy to help compete with China's CCP funding AI.

Wow the most open OpenAI has ever been is when someone sue them.

On the other hand, this shows Elon doesn't care jackshit about the lack of openness from OpenAI. He's just mad only that he walked away from a monumental success.

Based on this line of reasoning, ANY company that builds any given technology and intends to share (sell) it to the world, but not divulge how it was done, can call itself OpenWhatever.

They are clearly saying that the word “Open” in their name means nothing.

But really there is so much money in this, and if they can make it they are going to be the next Google.

It should be obvious they don't want to be 'open' in terms of really making this open source.

Similarly Microsoft makes incredibly large software and my letters about this have gone unanswered

Points of reference change.

So sure the name means nothing now in a market shaped by OpenAI, where everyone offers APIs and has chat interfaces. It doesn't mean it meant nothing when they picked it or that they abandoned the meaning. The landscape just changed.

I understand that this particular audience is very sensitive about the term, but why are we being so childish about it? Yes, you can name your company whatever you want within reason. Yes, it does not have to mean anything in particular, asterisk. Companies being named in colourful ways is not particularly new, nor interesting.

Funny that you mention that, actually. Before OpenAI started generously... publicizing their models, Google was actually shipping their weights day-and-date with their papers. So honestly, I actually doubt that Google would do that.

> Making that multiplier publicly available can be the "Open" part.

Aw, how generous of them. They even let us pay money to generously use their "Open" resource.

> but why are we being so childish about it?

Why are you being so childish about it? "Open" means something - you can't contort OpenAI's minimalist public contributions into a defense of their "openness". You'd have better luck arguing that Apple and Microsoft support Open Source software ddevelopment.

The last significant contribution OpenAI open-sourced was GPT-2 in 2019. They are a net-negative impact on the world at-large, amassing private assets under the guise of public enrichment. If it was an option between OpenAI or nothing, I'd ask for nothing and pray for a better frontrunner. It's not the name, it's the way they behave and the apologism they garner.

OpenDoor?

"OpenSource". Open in open source means everyone should benefit from the fruits of source after it is built but it's totally OK to not share the source(Even though calling ourselves open is the right strategy for medium term recruitment and adoption purposes).

If anyone tries the same logic with "open" source, they will be ridiculed and laughed at, but here we are.

"Free as in speech" : I'm not sure which part of openAI's actions show commitment to this part.

https://openai.com/research/whisper

Hardly. There's free beer and free speech do have different meanings, but freeware isn't something you have to pay for because it's "free as in beer".

In OpenAI's case, "open" isn't open as in anything normally associated with the word in the context.

Open as in private club during business hours for VIP members only is how they are trying to explain it, but understandbly, some people aren't buying it.

I am not sure how to report, but this user name is offensive. In English it means f***ot.

"[T]otally OK not to share the science." I think the reasonable average person would disagree with that. And, it would go against certain goal & financial transparency principles that the IRS demands to bestow the 501(c)3 designation.

(e.g. see here https://www.citizen.org/article/letter-to-california-attorne...)

What's wrong with it in context though is that as great as it is, it's just some guys blog. It's disconcerting that people would be working on technology they think is more dangerous than nuclear weapons and basing their approach to safety on a random blog post.

Although it's disconcerting to think of a committee deciding how it's approached, or the general public, or AI loving researchers, so it might just be a disconcerting topic.

If OpenAI or just Ilya think Scott is the best man to have thinking about it though, I would have at least liked them to pay him to do it full time. Blogging isn't even Scott's full time job, and the majority of his stuff isn't even about AI.

https://slatestarcodex.com/2019/03/20/translating-predictive...

Begin here: https://slatestarcodex.com/2014/07/30/meditations-on-moloch/

The fact that these two statements have opposing ideals really highlights the hypocrisy of these billionaires. Ilya’s statement is just them trying to convince themselves that what they’re working on is still a noble cause even if OpenAI isn’t actually “Open”.

Yup he's pretty transparently lying here.

He committed to fund for $1B dollars. Then when they wouldn't make him CEO and/or roll the company into Tesla he refused his commitment after paying out only 4% of it. Claimed the company would fail and only he could save them (again wrong).

And now is mad he doesn't control the top AI out there and because he chose to walk away from them.

And yet again people are falling for him. Elon talk never matches his action. And there is still a large portion of the internet that falls for his talk time and time again.

No matter his motives, I applaud this lawsuit. Who cares if his talk doesn't match his action? His action, here, is good.

From the complaint:

> Together with Mr. Brockman, the three agreed that this new lab: (a) would be a non-profit developing AGI for the benefit of humanity, not for a for-profit company seeking to maximize shareholder profits; and (b) would be open-source, balancing only countervailing safety considerations, and would not keep its technology closed and secret for proprietary commercial reasons (The “Founding Agreement”).

The biggest beneficiary of this lawsuit is Google, which now gets more runway to bumble along to victory.

And it doesn’t take being that rich to have this problem. Even minor celebs in L.A. have this problem!

This seems to be his natural resting state these days

You really come to that conclusion from a "yup"? Damn.

Here's the previous sentence for context:

> “a safe AI is harder to build than an unsafe one, then by opensorucing everything, we make it easy for someone unscrupulous with access to overwhelming amount of hardware to build an unsafe AI, which will experience a hard takeoff.”

As an aside: it's frustrating that the parent comment gets upvoted heavily for 7 hours and nobody has bothered to read the relevant context.

But it looks 10x worse for Elon. At least for the public image, he desperately try to maintain.

> As we discussed a for-profit structure in order to further the mission, Elon wanted us to merge with Tesla or he wanted full control. > In late 2017, we and Elon decided the next step for the mission was to create a for-profit entity. Elon wanted majority equity, initial board control, and to be CEO. In the middle of these discussions, he withheld funding,

The fact that they slinging mud at each other just proves the mutual criticism though, and provides an edifying spectacle for the rest of us.

OpenAI changed its stance about five years ago. In meanwhile they got billions in investment, hired best employees, created very successful product and took leadership position in AI. Only narrative remaining was that they somehow betrayed original donors by moving away from the charter. This shows that is not the case; original donor(s) are equally megalomaniac and don't give a fuck about the charter.

You sure about that ?

For now they are largely looking away (and boy it's hard !) from his 'far right' adventures. But it was just reported that he met with Trump. And it's pretty clear that if Trump is elected and does cancel EV subsides as he says he will do, Tesla is dead and he knows it.

So now, we have both of those guys, having something each other wants - Trump wants Musks money now, and Musk wants ... taxpayer money once Trump is elected. I bet, Musk reputation can and will go much much lower in coming months.

"When Elon Musk came to the White House asking me for help on all of his many subsidized projects, whether it's electric cars that don't drive long enough, driverless cars that crash, or rocketships to nowhere, without which subsidies he'd be worthless, and telling me how he was a big Trump fan and Republican..."

You know that Tesla operates outside the US as well, right? Where 95% of the worlds population live?

Check what happens to Tesla sales when countries cut EV subsides. New Zealand did it in January. Here is the data:

https://evdb.nz/ev-stats

The truth is, without subsides, Tesla would sell a fraction of what it does now, and it would certainly not be profitable (hell, it probably won't be profitable in 2024, even with the subsides !).

I know some uber-bulls have long insisted Tesla stop selling cars to the public so they can be horded for the imminent Robotaxi fleet that will soon be deployed en masse by 2020.

Depends if there really is business clash between ICE donors and Musk.

What is promised to public is not important if your voters are true believers.

Same for Musk, he can spin anything to his believers as well.

His detractors do a poor job of knocking him down a peg. I am speaking as someone who was there on day 0 of r/realTesla.

Even today with garbage like that recent 60 minutes hit piece on SpaceX, it is easy to dismiss because of all the easily disprovable claims. So much criticism on Musk is just so lazily produced and not properly vetted and its a shame because there is a ton of material to get him with.

It doesn’t make sense to engage in a public spat with someone who has such a negative reputation. Staying silent would have made more sense.

Posting it publicly is an odd and very unprofessional move. I can imagine Satya doesn’t like this blog post.

If we agree it's a for profit company, and all this 'Open' stuff is just PR, then yes - it's not looking bad. It's just business.

> OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.

That's obviously changed to "let's make lots of money", which should not be any "non-profit" organization's mission.

Typically when organizations do something like that it speaks to some considerable underlying issues at the core of the company and the people in charge of it.

It is particularly troubling when it pertains to technology that we all believe to be incredibly important.

Now, I’m not too concerned with any of the large LLM companies and their PR stunts, but from my solely EU enterprise perspective I see OpenAI as mostly a store-front for Microsoft. We get all the enterprise co-pilot products as part of our Microsoft licensing (which is bartered through a 3rd party vendor to make it seem like all the co-pilot stuff we get is “free” when it goes on the budget).

All of those tools are obviously direct results of the work OpenAI does and many of them are truly brilliant. I work in an investment bank that builds green energy plants and sells them with investor money. As you might imagine, nobody outside of our sales department is very good at creating PowerPoints. Especially our financial departments used to a “joy” to watch when they presented their stuff on monthly/quarterly meetings… seriously it was like they were in a competition to fit the most words into a single slide. With co-pilot their stuff looks absolutely brilliant. Still not on the level of our sales department, but brilliant and it’s even helped their presentations not last 90 million years. And this is just a tiny fraction of what we get out of co-pilot. Sure… I mostly use it to make stupid images of ducks, cats, and space marines with wolf heads for my code related presentations, and, to give me links to the right Microsoft domination I’m looking for in the ocean of pages. But it’s still the fruits of OpenAI.

Hell, the fact that they’re doing their stuff on Azure basically means that a lot of those 10 Microsoft billion are going directly back to Microsoft themselves as OpenAI purchases computing power. Yet it remains a “free” entity, so that Microsoft doesn’t run into EU anti-trust issues.

Despite this gloom and doom with an added bit of tinfoil hat, I do think OpenAI themselves are still true to their original mission. But in the boring business sense in an enterprise world, I also think they are simultaneously sort of owned by the largest “for enterprise” tech company in the world.

It's all bullshit, and these emails show it again. He is an extraordinary businessman of course - but everything he does, he does for money and power - mostly coming from government subsides btw. Until recently though, he managed to make most believe that it was to save the planet and the humanity.

1) Released the original Boring Company idea about tiny tunnels under LA to alleviate road traffic, which could easily be shot down by anyone modeling it on a single paper napkin.

2) Got caught up in ontological (theistic) arguments about alien scientists who had surely created a computer simulation in which we all live.

3) In an effort to prevent the inevitable AI overlords from controlling us, bought a company to create a ubiquitous neural interface so that computers have read/write access to our brains, which somehow the AI overlords will not take advantage of.

This was all as of 2018.

I still give him credit for previous accomplishments, but in aggregate, it is arguable that Elon Musk's new found ability to avoid thinking things through might be our "Great Filter."

“Elon is the stupid people’s fantasy of what being smart is like, just like Trump can be a poor person’s fantasy of what being rich is like.”

Don’t agree entirely in the poor part, but the Elon part I find quite accurate.

I may be trying to compensate for having been an Elon stan pre-2018ish, but I do have to give him a lot credit for being the founder of SpaceX, which is the best launch provider on the planet. He was also the CEO who made Tesla what it is. He did accelerate the adoption of EVs. He truly believed in the physics of both of those endeavors, he was right, and I loved him for it. A true inspiration.

BUT.. around 2018 he got high on his own supply. I had watched every Elon video up to that point, and there is one old one where he said ~"I fear that I could get too ego driven, and have no one around to call me on my shit, and that really worries me." He was right to worry about that, because he sailed right through that with a vengeance. The first couple years of that were tolerable, but now, what a devolution. I hate to admit it, but it really broke my heart.

Now back to my first paragraph, Musk is worse than Trump politically, aka the part that matters now, because he is sucking a lot of intelligent people down to the lowest of human nature. It deeply disgusts me.

He can't work with CEOs who are smarter than him.

> Message to Elon: "A for-profit pivot might create a more sustainable revenue stream"

> Elon: You are right ... and Tesla is the only path.

Then 3 weeks later Elon gets kicked off the board, probably after a fight where he tried to make OpenAI become for-profit under Tesla.

How can you not see how conniving this man is? The lawsuit is either a revenge play or a play to take down xAI's competition.

I misspoke. He may have just resigned from the board when he didn't get what he wanted.

Sam's reticence to publicly defend himself until now has backfired. Elon has fully controlled the public narrative and perception forming and it is hard to dislodge perceptions after they've settled.

I mean, it was not the conspiracy-infused, anti-vaccine, pro-Russia, antisemitic vibe about him, no, but those emails that crossed the line

That is surprisingly greedy & selfish to be boasting about in their own blog.

How society organizes property rights makes it damn near impossible to make anything commons in a way that can't in practice be reversed when folks see dollar signs. Owner is a non nullable field.

That is how I read it anyway and I don't see a reason to interpret it in a nefarious way.

First, this assumes that they will know when they approach AGI. Meaning they'll be able to reliably predict it far enough out to change how the business and/or the open models are setup. I will be very surprised if a breakthrough that creates what most would consider AGI is that predictable. By their own definition, they would need to predict when a model will be economically equivalent to or better than humans in most tasks - how can you predict that?

Second, it seems fundamentally nefarious to say they want to build AGI for the good of all, but that the AGI will be walled off and controlled entirely by OpenAI. Effectively, it will benefit us all even though we'll be entirely at the mercy of what OpenAI allows us to use. We would always be at a disadvantage and will never know what the AGI is really capable of.

This whole idea also assumes that the greater good of an AGI breakthrough is using the AGI itself rather than the science behind how they got there. I'm not sure that makes sense. It would be like developing nukes and making sure the science behind them never leaks - claiming that we're all benefiting from the nukes produced even though we never get to modify the tech for something like nuclear power.

> “a safe AI is harder to build than an unsafe one, then by opensorucing everything, we make it easy for someone unscrupulous with access to overwhelming amount of hardware to build an unsafe AI, which will experience a hard takeoff.”

The comparison to nuclear weapons has always been mistaken.

One thought experiment is to imagine someone developing software with a promise to open source the benign parts, then withholding most of it for business reasons while citing aliens as a concern.

I mean, I'm totally with them on the fear of AI safety. I'm definitely in the "we need to be very scared of AI" camp. Actually the alien thought experiment is nice - because if we credibly believed aliens would come to earth in the next 50 years, I think there's a lot of things we would/should do differently, and I think it's hard to argue that there's no credible fear of reaching AGI within 50 years.

That said, I think OpenAI is still problematic, since they're effectively hastening the arrival of the thing they supposedly fear. :shrug:

So, they cant release science that the googles of the world can use to their advantage BUT they kind of have to because that's their whole mission.

The whole thing was sort of dead on arrival and Ilya's email dating to 2016 (!!!!) only amplifies that.

[1]https://officechai.com/startups/sam-altman-is-unsure-if-ilya....

> with Sam being unsure of whether Sutskever is still working at OpenAI even 8 weeks after the entire incident

He wasn't missing/found, maybe negotiations were ongoing.

None of this screams "I'm going to change this world", this organization is mired in politics from the start, no wonder Satya is hedging his bets.

edit: grammar

See Apple in the 90's for instance.

But also, all entities involved looked bad already to observers who weren't so deeply rooting for them already that nothing would change their minds, so this is basically tabloid fodder drama in terms of importance as far as I can tell

So we have them airing their dirty laundry because a billionaire sued them for his own petty reasons, having planned to break the promises the organization made that he's now suing them over in a similar way. Billionaire is hypocrite, news at eleven. This doesn't exonerate the organization. I hope this suit makes people angry. I hope it makes it harder to get away with this facile and infantilizing line about keeping people in the dark to protect them

Edit: sorry, I meant the parent said that. But I guess you understand it now.

Edit 2: To be clear, the bad stuff:

- OpenAI not really being "open" in any way.

- Elon not really caring about that but just wanting to take revenge because they became successful without him.

Half the country doesn't even understand Instagram is owned by Meta.

As someone who was watching the brain drain of AI researchers from academia by tech giants before OpenAI was founded, their ethos and initial actual transparency was somewhat inspiring, and the first move to lock up their new frontier model struck me as disappointing and suspicious, and then actively like a betrayal once it came out that it was an exclusive licensing deal with Microsoft.

Maybe members of the public who are less invested in AI research don't feel similarly, but a lot of laypeople seem to suddenly care about AI, including the usual giant swath of techbros that were probably previously working crypto scams becoming really cultish and smug about it, but also non-tech people I know have it on their mind a lot, which is kinda new, and most of these people view it negatively, and almost all of them care because of OpenAI. Frankly your comment makes no sense to me. What are you trying to get at here?

This is not reflected in their 990's. They claim $20 million in public support and $70 million in "other income" which is missing the required explanation.

Also, why are none of the current board members of OpenAI included as authors here? Is there a problem in the governance structure?

Elon could not legally contribute more than he did without turning OpenAI into a private foundation. Private foundations are required to give away 5% of their total assets annually and are not permitted to own substantial stakes in for-profit businesses.

Showing old emails from people who clearly don't understand what they are getting into is not very helpful to their case. Maybe if they had talked to a lawyer who understood non-profit law, or even just googled it.

If the $70 million was in fact a donation instead of income, they fail the public support test and are de facto a private foundation.

Here's the 2020 990 which shows the first 5 years of the org's existence (including the time at question for this suit): https://apps.irs.gov/pub/epostcard/cor/810861541_202012_990_...

Page 15 is Schedule A Pt 2 which shows the total contributions by year. They did indeed raise ~$133M over that time frame. Row 5 shows the contributions from any 1 person who contributed more than 2% of their total funding (this excludes other nonprofits) -- so the $41M there is definitely Musk. So his share was only ~30% of the total and the other 70% was public support which you can confirm in Section C at the bottom of that page.

"Public support" includes other nonprofits - and it's fine of e.g. Musk 'laundered' other funding via a DAF or something at a different nonprofit since the funds belong to that nonprofit and they have ultimate discretion over the grant.

I know they got $20 million from Open Philanthropy which qualifies as public support, so I am still wondering about the other $70 million, but it is not the smoking gun that I thought it was.

It has to be made up of donations from individuals under $2.6 million or from other public charities, but not private foundations.

Edit - Got curious and sure enough - this is the 28,000 page 990 filing for Fidelity Charitable: https://apps.irs.gov/pub/epostcard/cor/110303001_202006_990_...

On page 205 there's a $3.5M donation to OpenAI from 2019.

Likewise here for SVCF on page 237 (https://apps.irs.gov/pub/epostcard/cor/205205488_201912_990_...) - there's a $30M donation to OpenAI in 2019.

Nice :)

https://www.irs.gov/pub/irs-drop/n-17-73.pdf

https://taxnews.ey.com/news/2023-1927-irs-releases-proposed-...

https://apps.irs.gov/pub/epostcard/cor/464347021_201712_990P...

https://apps.irs.gov/pub/epostcard/cor/464347021_201812_990P...

Since private foundations aren't public charities, they don't have the pass-through protection of donor-advised funds, so this should have been excluded from the public support total because it is more than 2% of the total support.

Also, this reporter did some additional legwork earlier in 2023: https://techcrunch.com/2023/05/17/elon-musk-used-to-say-he-p...

Though I think the blog post will probably achieve its messaging goals of diminishing Elon's character to their own public status... "Yeah you think we suck, but it would be no tangible difference to you if Elon had run it, so you should know he sucks too."

I love the drama. It is very entertaining. I'm glad OpenAI decided to start as a non profit. I feel like they will never be able to get away with it. The issue will keep lingering.

It was a reasonable concern, funny that it turned out the other way with the transformer

Also, DKIM signatures are notably absent, not that we could verify them anyway since the emails are heavily redacted.

As you say, this way, they get to control the narrative. Nothing strange at all.

From what I can tell, Musk's lawsuit doesn't have a much of chance in the first place. I don't think he expects to win, it seems to be more a tool in Musk's own media push, and while I wouldn't bet on it, there is absolutely a chance it won't reach discovery.

I think OpenAI have quite wisely decided that the real battle here is the court of public opinion. They know it's possible to win the court case but lose in the eyes of the public. And they know that Musk has a lot of previous experience (and success) in the this battleground.

> I think OpenAI have quite wisely decided that the real battle here is the court of public opinion.

They've failed to win me over. As far as I can tell, their attempted PR victory hinges on a single email with a one-word reply from Musk. Their own emails are far more damning as they give a detailed explanation of why they believe AI should not be open.

To an observer who already dislikes Musk, I'm sure it's a PR win. To someone neutral or someone who dislikes both parties, it's a PR disaster.

What are you implying? Faking emails opens you up to libel. I doubt OpenAI is trying to add another lawsuit to their workload.

I'm not implying anything. I'm pointing out a lack of openness on OpenAI's part.

In that case, let me ask: where have you ever seen DKIM signatures being provided on a public blog post/press release?

But at least those emails were un-redacted. To spell that out for you: in a highly-charged, highly-contentious political setting, emails were published un-redacted and theoretically verifiable with DKIM. No such possibility exists for OpenAI's blog post.

[1] https://news.ycombinator.com/item?id=24780798#24785123

Xitter is the #1 source of facts, so - Can't blame them for counter balancing a little.

This kind of in place redaction seems to be the typical way that documents are submitted to courts, or at least it was the same way that the emails Elon provided were redacted[0].

[0]: https://twitter.com/TechEmails/status/1763633741807960498

Nobody who doesn't know who REDACTED is cares what REDACTED thought of the issue.

This is again some of the least professional comms I've seen from a Microsoft entity. We'll see what the evidence is ourselves if and when the case goes to court.

I'm guessing email #3 is referring to Larry, whom Elon has had a lot to say about publicly on the topic of AGI.

They redacted parts of these emails.

Did they publish all of the emails?

Or are we literally being fed a curated collection?

Very gross.

*only applicable to certain definitions of evil

Now it's some kind of "Kiss the ring and have respect for the hand that feeds you" or something.

Isn't doing that in and of itself somewhat evil?

Google was born out of NSA and CIA research grants, and they operate under capitalist incentives. Many would say they have never not been evil. But again, it's a matter of perspective.

It's obviously not a phrase any company would ever take seriously since it has no real legal implications, and it's honestly weird how many people seem surprised and offended to find this out. No one is getting fired because Raisin Bran cereal doesn't guarantee a minimum of two scoops of raisins, either.

https://en.m.wikipedia.org/wiki/Attention_Is_All_You_Need

The open source angle probably matters more to the HN audience but it’s hard to argue that OpenAI hasn’t “opened up access to AI” to a vast global audience.

And that’s not to say that the means by which they’ve done so doesn’t still need some scrutiny.

Ilya being the bad guy is a surprise here.