News for hackers. Stuff that matters.
Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @07:49AM (#37800924)
> partnership to bring greater focus and urgency to the creation of artificial general intelligence (AGI)

Why is this urgent?

Re:John Carmack and Rich Sutton partner to accelerate develo... by chasd00 on Saturday October 07, 2023 @08:39AM (#37801231)
> partnership to bring greater focus and urgency to the creation of artificial general intelligence (AGI)

>> Why is this urgent?

that's an understatement. The rush to AGI reminds me of the rush to human cloning.

Re:John Carmack and Rich Sutton partner to accelerate develo... by throwaway10965 on Saturday October 07, 2023 @08:03AM (#37800995)
They're not saying it's urgent; they want to bring urgency - convince people to move faster.

It seemed appropriate to ask AI about the meaning of the phrase:

The phrase "bring urgency to something" means to inject a sense of importance and immediate attention to a specific issue, project, or situation. The aim is to motivate people to prioritize the task at hand and to act more quickly than they might otherwise.

Re:John Carmack and Rich Sutton partner to accelerate develo... by falcor84 on Saturday October 07, 2023 @08:57AM (#37801348)
When someone wants to "bring urgency", this generally implies that it's already quite urgent to them, so I understood this as that question.
Re:John Carmack and Rich Sutton partner to accelerate develo... by Skyy93 on Saturday October 07, 2023 @07:53AM (#37800942)
Perhaps there is some belief in this that an AGI can save us from our biggest problems? Energy Crisis, Environmental Pollution, Waging Wars.

But I agree with you ... it is not really urgent since we know answers for the most of the problems but do not like the solution.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @07:55AM (#37800954)
A “solution” that isn’t viable nor palatable isn’t a true solution; it is often merely a hope that people are not people.
Re:John Carmack and Rich Sutton partner to accelerate develo... by Skyy93 on Saturday October 07, 2023 @07:57AM (#37800965)
Than there will be never be a solution to some problems.
Re:John Carmack and Rich Sutton partner to accelerate develo... by tharne on Saturday October 07, 2023 @09:46AM (#37801659)
You’re presupposing every problem has a nice solution. This is almost certainly false
Re:John Carmack and Rich Sutton partner to accelerate develo... by ang_cire on Saturday October 07, 2023 @11:03AM (#37802282)
Any solution is a nice solution depending on your frame of reference. There is no such thing as objectively good or bad without defining the acceptable and unacceptable effects.
Re:John Carmack and Rich Sutton partner to accelerate develo... by nine_k on Saturday October 07, 2023 @02:24PM (#37804214)
There is an implied frame of reference that requires the continued existence of humans.

Any solution that violates this boundary condition is "objectively bad", that is, cultures that follow this solution disappear and thus lose the evolution game.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Monday October 09, 2023 @11:05AM (#37821212)
I take your point but...

1. There are a set of things that are nearly objectively true across nearly all cultures. For example, random killing for selfish purposes is not ok.

2. I'm not a moral relativist, in two senses. First, the "weak" sense: I would prefer to live in some moral systems more than others. Second, in "nearly-strong" sense: some moral systems are 'nearly' objectively better than others (see point above). (Note: my definitions are imprecise; still working on how to square them with existing philosophical writings.)

3. Even for people that evaluate the morality of something according to its effects (consequentialists), there is considerable room to debate the relevant time frame.

Re:John Carmack and Rich Sutton partner to accelerate develo... by Qwertious on Saturday October 07, 2023 @11:14AM (#37802394)
Lots of problems could be at least severely mitigated if you could throw literally millions of bodies at the problem - if AGI could deliver a $50 chip that could substitute for a human, then could shove each one into a roomba wired to an arm, and you basically have an infinite free workforce.
Re:John Carmack and Rich Sutton partner to accelerate develo... by gretch on Saturday October 07, 2023 @01:23PM (#37803600)
Yeah except, how do you manufacture them? And distribute them to where the work needs to be? And what type of fuel do they use - we don’t have infinite free energy for an infinite free workforce.
Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Monday October 09, 2023 @10:58AM (#37821130)
> You’re presupposing every problem has a nice solution.

Not at all. Where are you getting that?

Re:John Carmack and Rich Sutton partner to accelerate develo... by lttlrck on Saturday October 07, 2023 @10:06AM (#37801810)
Viability is a given. But palatability? That's like giving up and saying there are no solutions to hard problems.
Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Monday October 09, 2023 @10:58AM (#37821129)
I take your point. Palatability can change.

> That's like giving up and saying there are no solutions to hard problems.

Not at all. I'm simply defining a solution as one that has some probability of traction.

Re:John Carmack and Rich Sutton partner to accelerate develo... by garblegarble on Saturday October 07, 2023 @08:45AM (#37801264)
>we know answers for the most of the problems but do not like the solution.

What an absolutely wonderful insight put extremely succinctly, thank you!

Re:John Carmack and Rich Sutton partner to accelerate develo... by sschueller on Saturday October 07, 2023 @09:43AM (#37801640)
A true AGI would determine the only solution to our crisis is the elimination of the human race...
Re:John Carmack and Rich Sutton partner to accelerate develo... by dartos on Saturday October 07, 2023 @09:58AM (#37801740)
Are you just basing that idea off of sci fi novels?
Re:John Carmack and Rich Sutton partner to accelerate develo... by charlie0 on Saturday October 07, 2023 @10:36AM (#37802050)
It would probably be the simplest solution to a naive super intelligent agi. Just take a look at how humanity dealt with dangerous animals. They are mostly no longer around unless in a zoo.
Re:John Carmack and Rich Sutton partner to accelerate develo... by esrauch on Saturday October 07, 2023 @11:15AM (#37802411)
Humans don't put ants in zoos because the ants overconsume their localized habitat though. A naive super intelligent agi has no reason to care if global warming makes the planet 150 degrees F on average, if it's self-interested it would surely be able to be "ok" on a planet under those conditions.
Re:John Carmack and Rich Sutton partner to accelerate develo... by charlie0 on Saturday October 07, 2023 @11:24AM (#37802502)
Sure, but at some point it will be asked to solve those problems and if it's not properly built, could easily arrive at that conclusion from a problem solving perspective. Self interest doesn't have to come into play at all here.
Re:John Carmack and Rich Sutton partner to accelerate develo... by AuryGlenz on Saturday October 07, 2023 @09:03PM (#37806938)
If it’s tasked with solving those problems it can also be done with constraints such as “without killing humans.”
Re:John Carmack and Rich Sutton partner to accelerate develo... by hskalin on Monday October 09, 2023 @12:09AM (#37816859)
Not if you add a constraint that ensures survival of humanity.
Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @08:16AM (#37801067)
Without taking a particular position on how the probabilities shake out, I suggest critically reading “What We Owe the Future” by William MacAskill and:or other long-term writings that use statistical thinking over a wide range of future scenarios.
Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @09:25AM (#37801526)
Creator of "Centre for Effective Altruism"? Longtermism? No thanks. I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others. I mean, living unethically today in order to affect future lives positively is not for me.
Re:John Carmack and Rich Sutton partner to accelerate develo... by NateEag on Saturday October 07, 2023 @09:55AM (#37801711)
> I mean, living unethically today in order to affect future lives positively is not for me.

Though it is an almost-inescapable conclusion of being a utilitarian and significantly smarter than average.

If you can defraud millions and put that money to work saving lives and buying many more QALYs than the defrauded would have otherwise had, you have a moral obligation to do so.

...there are many ways to make utilitarianism fall apart, but that one's my current favorite.

Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @10:09AM (#37801838)
> If you can defraud millions and put that money to work saving lives and buying many more QALYs than the defrauded would have otherwise had, you have a moral obligation to do so

So defrauding millions to accrue QALYs (for whom?) is morally acceptable?

> Though it is an almost-inescapable conclusion of being a utilitarian and significantly smarter than average.

A modicum of spirituality or religion will quickly quash this. So you have to add atheism to the requirement. Or at least non-spirituality or belief in materialism (I'm not a philosopher so may be using that term incorrectly).

Re:John Carmack and Rich Sutton partner to accelerate develo... by NateEag on Saturday October 07, 2023 @02:09PM (#37804085)
> So you have to add atheism to the requirement.

Yep, you're absolutely right - I forgot to specify that.

Most utilitarians don't believe in transcendence, in my experience - not believing in it is usually why they're utilitarian (trying to salvage any argument for behavioral standards other than "I like it when other people do what I like").

Re:John Carmack and Rich Sutton partner to accelerate develo... by NateEag on Saturday October 07, 2023 @02:15PM (#37804139)
> So defrauding millions to accrue QALYs (for whom?) is morally acceptable?

Under most utilitarian theories of ethics, I think it's morally required, if you can show the fraud really does yield a significantly higher QALY count.

The best out I can see to argue the fraud is not required is to show that another course of action will yield even more QALYs. Then you'd be obligated you do that instead.

That, of course, comes to another fundamental flaws of utilitarianism, namely the belief that you can know what the results of your actions will be with any meaningful completeness or certitude.

You can't.

Without that it's all just excuses that justify whatever you want to do.

Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @04:09PM (#37805099)
> namely the belief that you can know what the results of your actions will be with any meaningful completeness or certitude. You can't.

In parallel threads, a subscriber to this belief mentions probabilities. As long as the probability of an outcome is likely (based on variables and formulas who defines, I don’t know), the means justify the ends. So it’s even worse than what you describe: certitude is not even required, only a likely probability of certitude.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @11:21AM (#37802480)
> I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others.

Such a claim is completely illogical. How can a philosophy be "debunked" by one example of a fraudster?

Please, slow down. Read and write critically.

Sure, one can distort and misuse almost any moral philosophy, such as utilitarianism or any other. History is filled with examples.

You may disagree with particular assessments, conclusions, or even the basic tenets of longtermism. Argue against the philosophy if you like.

Pick an existential risk, such as nuclear proliferation or artificial intelligence. Study the risks and the benefits. But according to what framework? Please do some legwork about _how_ you plan on evaluating various possible future scenarios. This is is hardly controversial.

My recommendation is simple: read MacAskill's book. It is a valuable and different framework than a vast majority of people have really considered. These ideas are not widely percolating yet. You don't need to completely agree with it. If you are thinking person, you probably won't.

Now, if you think I'm missing a key perspective or set of facts or reasoning, please let me know. I'll change my assessment based on good evidence.

Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @12:38PM (#37803184)
> Or are you talking about something else?

I'm talking about immoral or questionable behavior today to justify a possible future. It's like going back in time to kill Hitler's grandfather to prevent Hitler's birth. You have no idea of unintended consequences, and you are intentionally causing harm now for a future completely out of your control (although you deceive yourself into thinking it is in your control by taking action today).

> Pick an existential risk, such as nuclear proliferation or artificial intelligence. Study the risks and the benefits.

I lived through the Cold War. I don't need this exercise because, the bottom line is, no one knows what existential risk - if any - will happen. The worst of the Cold War never happened, yet many of us thought it would. Likewise, you might choose to focus on AI as your existential risk, go through all the sacrifices and immoral behaviors that longtermism leads you to do in order to mitigate AI, and then a giant meteor collides with Earth. Oops! You chose the wrong existential risk. And meanwhile, you compromised your morals and right behavior in order to mitigate the wrong existential risk. No thanks, not for me.

> My recommendation is simple: read MacAskill's book

No. I might read about 1000 books in my lifetime (85 reading years * 12 per year). You don't get to choose one of them. But I do appreciate your passion and evangalism.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @01:56PM (#37803967)
> no one knows what existential risk - if any - will happen.

We live in a world of uncertainty. We can and must make probabilistic assessments. Don't pretend otherwise by using the classic "appeal to ignorance" fallacy.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @02:01PM (#37804015)
>> Or are you talking about something else?

> I'm talking about immoral or questionable behavior today to justify a possible future.

You've outlined a form of deontological ethics, seems to me. You don't subscribe to consequentialism, that fine. But hopefully you recognize there are many moral theories.

> It's like going back in time to kill Hitler's grandfather to prevent Hitler's birth.

This is a strawman. (A really loaded one at that.)

Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @03:14PM (#37804609)
> You've outlined a form of deontological ethics,

I had to look this up. You've given a modern label to an ancient concept sometimes known as the Golden Rule. I'm not sure why.

>> It's like going back in time to kill Hitler's grandfather to prevent Hitler's birth.

> This is a strawman. (A really loaded one at that.)

It's not intended that way, and I'm posing it with honesty. If it's too loaded, then replace Hitler with, I don't know, <insert_bad_guy_here>. It doesn't matter who the bad guy is; the point is: you kill an innocent person (grandfather_of_bad_guy) in order to mold the future in a way that you think will save more, other innocent lives. Correct?

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @06:10PM (#37805964)
> I had to look this up.

Great! Philosophers have explored the ideas of "inherent goodness" versus "goodness because of the effects" for thousands of years. People face these decisions everyday, whether they realize it or not.

> You've given a modern label to an ancient concept sometimes known as the Golden Rule. I'm not sure why.

First, I didn't discover the idea of deontological ethics (DE). From Brittanica:

> In deontological ethics an action is considered morally good because of some characteristic of the action itself, not because the product of the action is good. Deontological ethics holds that at least some acts are morally obligatory regardless of their consequences for human welfare. Descriptive of such ethics are such expressions as “Duty for duty’s sake,” “Virtue is its own reward,” and “Let justice be done though the heavens fall.”

Second, the Golden Rule is not the same thing as DE.

Consider this:

> Like most key tenets of ethics, the golden rule shows two major sides: one promoting fairness and individual entitlement, conceived as reciprocity; the other promoting helpfulness and generosity to the end of social welfare. Both the Kantian and Utilitarian traditions focus on only one side, furthering the great distinctions in philosophical ethics—the deontology-teleology and justice-benevolence distinctions. For the general theory project, this one-sidedness is purposeful, a research tool for reductive explanation. > > - https://iep.utm.edu/goldrule/

If you want to follow the Golden Rule, you need to think about consequences! Hence probability over future scenarios.

You _really_ need to branch out more from Hacker News. Go read some philosophy!

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @06:40PM (#37806138)
[delayed]
Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @02:11PM (#37804110)
> I lived through the Cold War. I don't need this exercise because, the bottom line is, no one knows what existential risk - if any - will happen. The worst of the Cold War never happened, yet many of us thought it would. Likewise, you might choose to focus on AI as your existential risk, go through all the sacrifices and immoral behaviors that longtermism leads you to do in order to mitigate AI, and then a giant meteor collides with Earth. Oops! You chose the wrong existential risk. And meanwhile, you compromised your morals and right behavior in order to mitigate the wrong existential risk. No thanks, not for me.

Your claim seems to boil down to two things: First, people sometimes make very incorrect predictions, so you've stopped worrying about them. Second, instead, you focus on "doing the right thing" based on your ethical criteria (which, as I say in sibling comment, seems to be aligned with deontological ethics -- i.e. the ends don't justify the means.)

I'm not going to argue with the second (your choice of ethics) here. But your first point is terribly flawed; it ignores probability and decision-making under uncertainty.

Sure, you have to make assessments about what to believe. Strictly speaking, as a society, it is irrational to stop paying attention and responding to the most important things, even if any one of them might be unlikely.

Personally, you might find it necessary or useful to tune out this stuff. I get that. I'd guess this happened: you have stopped _caring_ for various reasons (perhaps information overload, sensationalization of the news, more pressing matters right in front of you) and your brain has constructed a worldview that _rationalizes_ not caring. Totally human, I get it. But this isn't reasoning nor logic.

I may not have your years of experience, but I'm no spring chicken either. Age doesn't always bring wisdom, unfortunately. People learn the wrong lessons. We've learned many. I have a keen interest in the Cold War. If you study the history, you'll recognize that our path 'through' it was no means assured. We are still living with the downstream effects of it, including huge nuclear arsenals and proxy wars.

Your comment above is naive with regards to history. It paints a simplistic and convenient narrative with the benefit of hindsight. You've ignored one key aspect of history, which is to understand it _in context_. Using hindsight exclusively isn't the only nor best way to learn from history.

Even subscribing to your sense of ethics, there is a certain level of responsibility to act rationally in order to achieve moral ends in a moral way. To completely ignore probability, which what your comments have suggested so far, is immoral.

Please, remember this: using poor logic and FUD to sully a serious book (based on some vague implied connection to a crook) is not ok.

Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @03:20PM (#37804656)
> Your claim seems to boil down to two things: First, people sometimes make very incorrect predictions, so you've stopped worrying about them.

Didn't say I stopped worrying about them: global warming worries me. Ubiquitous microplastics worry me. Heavy metals in our food supply & PFAS in our water worries me.

> Second, instead, you focus on "doing the right thing" based on your ethical criteria (which, as I say in sibling comment, seems to be aligned with deontological ethics -- i.e. the ends don't justify the means.)

To me, the means almost never justify the end. And I understand you see it differently.

> using poor logic

I'm not sure what you deem poor logic, but I would point out that most of human life is based on emotion, not logic. In any case, I enjoy our debate and would be glad to continue in email if you like. See my profile for my email - I did not find yours.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Monday October 09, 2023 @10:09AM (#37820611)
I'd appreciate if you could be mindful about the context around a quotation.

> Please, remember this: using poor logic and FUD to sully a serious book (based on some vague implied connection to a crook) is not ok.

I wrote this because of your original comment about [William MacAskill]: "Creator of "Centre for Effective Altruism"? Longtermism? No thanks. I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others. I mean, living unethically today in order to affect future lives positively is not for me."

I'm not going to elaborate in detail about what I've already written, but your comment suggests that Sam Bankman's reputation "debunks" a book (which happens to consist of some moral philosophy, some history, and some application of the ideas to future scenarios). This is poor logic. I hope you would recognize and own this mistake.

Also, yes, I have enjoyed discussing. I have strived to not make any personal attacks, and I apologize if I'm come across as rude.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Monday October 09, 2023 @10:14AM (#37820657)
>> Your claim seems to boil down to two things: First, people sometimes make very incorrect predictions, so you've stopped worrying about them.

> Didn't say I stopped worrying about them: global warming worries me. Ubiquitous microplastics worry me. Heavy metals in our food supply & PFAS in our water worries me.

On what basis do you worry about them? Why worry about some more than others? (I know my answers, but I'm interested in how you do it.)

Do you factor in the probability of their occurrence? I'd expect you do, this is sort of inevitable for people, though people aren't typically very well calibrated in probabilities, especially on the upper (near 100%) and lower ends (near 0%).

I'm trying to understand what I perceive as your pushback against moral reasoning that involves probabilities over future scenarios.

Here is my rough overall take at this point: given your ethical foundations (DE or close to it) and life experience (hard to know, often not driven by logic and reason -- such is the human condition) you bristle against consequentialism and longtermism in particular. I get it, but I don't think this is a good place to end up, even when I factor in your core beliefs. I 'chalk it up' to the high cost of mental reorgnization. My hypothesis is this: the 'wedge' examples that people use against consequentialism have had an outsized effect on how you view it. I don't detect any antipathy towards logic or probabilistic reasoning in general, but I also don't perceive a relentless drive towards it.

Re:John Carmack and Rich Sutton partner to accelerate develo... by xpe on Saturday October 07, 2023 @11:48AM (#37802697)
I don't get to meet the people who spend time on HN. But I'm curious: who doesn't see the value in _critically_ reading long-term thinking over a wide range of future scenarios?

Too busy coding? Reading about something for _your_ career? In service of _your_ family? Trying to get ahead? Yeah, I get it. But tell me: isn't thinking long-term worth something akin to a few hours a month?

You don't need to be doom-and-gloom about it. Sure, get stuff done. Be in the moment. All good.

Wouldn't it be nice to have some confidence that we're setting up future generations to have the possibility to at least have what we do, if not better?

P.S. FWIW, the morality of valuing future generations is _not_ properly addressed by most moral philosophies.

Re:John Carmack and Rich Sutton partner to accelerate develo... by TedDoesntTalk on Saturday October 07, 2023 @12:43PM (#37803234)
> isn't thinking long-term worth something akin to a few hours a month?

> Wouldn't it be nice to have some confidence that we're setting up future generations to have the possibility to at least have what we do

This presupposes that you have some control of the long-term. You don't, any more than the flap of a butterfly's wings in Melbourne affects the velocity of the wind in NYC several weeks later.

Re:John Carmack and Rich Sutton partner to accelerate develo... by aik on Saturday October 07, 2023 @01:42PM (#37803797)
Tell that to the millions of individual humans that have advanced humanity in some way or another for the past tens of thousands of years, through some discovery or invention or inspiring groups of people to do something great.
Re:John Carmack and Rich Sutton partner to accelerate develo... by layer8 on Saturday October 07, 2023 @10:02AM (#37801773)
It’s not. They are saying that they want to make it (appear) urgent.