News for hackers. Stuff that matters.
Re:7 Weeks AI Alignment Curriculum by empiko on Sunday March 05, 2023 @01:12PM (#35032282)
I don't think they are taken that seriously. They are writing their op-eds and fiery papers, but the field marches forward nevertheless. But I agree that the whole fields seems like ivory-tower types trying to hijack the latest hottest technology an become the decision makers based on their "exprertise". AI as a field always attracted all kinds of weird individuals, probably because it is such a popular and influential concept. If we would work with their ethics, we would never allow computers to reach general population, because it is simply too dangerous and a lot of "harm" can be done. Their use will be heavily monitored and a license or an assessment would be required to use some of the functions, e.g., connect to the Internet.
Re:7 Weeks AI Alignment Curriculum by kanzure on Sunday March 05, 2023 @01:27PM (#35032423)
We need something more like the proactionary principle instead of the precautionary principle: https://web.archive.org/web/20120203050245/http://www.maxmor...
Re:7 Weeks AI Alignment Curriculum by mitthrowaway2 on Sunday March 05, 2023 @01:30PM (#35032453)
I'm amazed to see how common this view is on HN. You'd think the rapid progress made in AI would give more validity and attention to these concerns raised about risk and alignment.
Re:7 Weeks AI Alignment Curriculum by dantheman on Sunday March 05, 2023 @02:41PM (#35033144)
The people doing this work are mediocre at best, if they could push the state of the art they'd being do that. It's a bunch of second handers trying to control those who actually understand what is going on.
Re:7 Weeks AI Alignment Curriculum by mitthrowaway2 on Sunday March 05, 2023 @03:26PM (#35033603)
Can you point to any evidence or examples of this? I don't find it a convincing counterargument at all.
Re:7 Weeks AI Alignment Curriculum by random314 on Sunday March 05, 2023 @01:38PM (#35032547)
Yes. The AGI teacher is a recent graduate with a Masters in physics. None of the material in this so called "course" is helpful with actual AI research or industry work.

AI "alignment" is a made up field ride with crackpots and charlatans.

Re:7 Weeks AI Alignment Curriculum by jacquesthibs on Sunday March 05, 2023 @01:49PM (#35032648)
The creator of the course was a research engineer at DeepMind and now works at OpenAI. Several people at those orgs have acted as facilitators for the course since its creation.

The purpose of the course is to give people an introduction to AI risk so that they can start getting into the field. Some who take it have been ML engineers for a while and some are just learning about the field.

The expectation was never to consider yourself a full-fledged alignment researcher once you finish this course, but instead to get people started. Many people will spend a bunch of time doing things like implementing GPT from scratch and replicate papers after this to try to get an internship at one of the big orgs. And then do other things to slowly make their way to producing great research. Sometimes ya’ll just want to hate on this site.

Re:7 Weeks AI Alignment Curriculum by random314 on Sunday March 05, 2023 @04:54PM (#35034438)
I am talking about the people conducting the course - Bluedot impact. The cofounder and DRI for AI safety is who I am talking about

https://www.linkedin.com/mwlite/profile/in/jamie-bernardi/

The researcher who has apparently created the syllabus is also a fresh PhD with 3 years of work experience.

Re:7 Weeks AI Alignment Curriculum by mitthrowaway2 on Sunday March 05, 2023 @05:08PM (#35034602)
So, what are they wrong about?
Re:7 Weeks AI Alignment Curriculum by random314 on Sunday March 05, 2023 @05:23PM (#35034780)
If you like to pay for an AI ML course being taught by someone whose entire ML education is 6 months of independent self teaching (as per his own LinkedIn) go ahead, be my guest.
Re:7 Weeks AI Alignment Curriculum by ChaseT on Sunday March 05, 2023 @07:40PM (#35036028)
The course is free and funded by Open Philanthropy.

It's not taught by James solely, but there's weekly discussion groups on readings chosen by a mix of alums from OpenAI and Oxford's Future Of Humanity Institute.

Re:7 Weeks AI Alignment Curriculum by mitthrowaway2 on Sunday March 05, 2023 @01:57PM (#35032718)
I mean, I'm not sure I'd require people who study and advocate for nuclear disarmament to also be publishing papers on how to improve bomb yields in order to treat them as credible. I'd probably expect them to study game theory and geopolitics.

"None of the material in this nuclear disarmament course even covers how to build a modern hypersonic missile!"

Re:7 Weeks AI Alignment Curriculum by Sharlin on Sunday March 05, 2023 @02:37PM (#35033111)
Don't be silly. Computer proliferation is not an existential risk.
Re:7 Weeks AI Alignment Curriculum by 13years on Sunday March 05, 2023 @04:05PM (#35033915)
Nuclear risk is a contained problem. Access is extremely limited and controlled by low tech hardware locks. The technology is understood, we know what makes bombs explode and it is predictable by physics.

AI is the opposite of all these characteristics. Its access is nearly everyone. The technology is not understood and not predictable. It is a nuclear warhead that thinks for which we don't know what might make it explode.

Re:7 Weeks AI Alignment Curriculum by Sharlin on Sunday March 05, 2023 @04:54PM (#35034446)
Yes, although I wouldn't go as far as to claim that current AI technology has any sort of "accessible to all" path to AGI. Accessible to irresponsible actors with lots of money? Possibly, and that's the real risk. But at least it's not an absurd comparison unlike the GP's.
Re:7 Weeks AI Alignment Curriculum by 13years on Sunday March 05, 2023 @05:29PM (#35034848)
Unlimited power makes everyone irresponsible actors. Nvidia already stated they expect a million fold processing power in 10 years for AI.
Re:7 Weeks AI Alignment Curriculum by empiko on Tuesday March 14, 2023 @11:05AM (#35152653)
> Computer proliferation is not an existential risk.

AI is created and run on computers so AGI risk must by definition be a subset of computer proliferation risk