News for hackers. Stuff that matters.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by stupidcar on Tuesday April 13, 2021 @01:01PM (#26795392)
AI safety seems like one of those topics that trips up even smart people such as Hawkins, because it intuitively seems like it has very obvious solutions. And for each objection raised there's usually a "so you just..." continuation, and an objection to that, until you're talking about things like instrumental convergence and you're in the territory of trying to reason in quite a complex way about the behaviour of systems that don't exist yet, and so the temptation is to dismiss it all as theoretical hand-wringing.

Personally, while I'm not on board with the board with the direst predictions of the super-intelligence pessimist crowd, I have become more and more convinced that goal misalignment is going to be a significant problem, and that while it might not doom the species, it's something that all AI researchers like Hawkins need to start paying close attention to now.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by fighterpilot on Tuesday April 13, 2021 @01:42PM (#26796139)
I also think people get too caught up on the expected time-frame.

The large majority of active AI researchers think that AGI will happen at some point in the (sub-1000 year) future.

When exactly isn't a very interesting question, relatively speaking.

We're going to have to deal with AGI eventually, and whether it's going to do what we want is not something that can be theoretically predicted from the armchair.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by fossuser on Tuesday April 13, 2021 @01:49PM (#26796252)
Yeah - and people are famously bad about predicting these events: https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no...

If it is something that's a hundred plus years out then we'll probably need whatever tech develops in the mean time to help, but since it's hard to know that seems reasonable for people to be working on it now?

It's also possible to figure things out before the necessary tech is possible (I think a lot of CS papers in the 60s became more interesting later when the hardware caught up to be more useful, arguably the recent NN stuff falls into this category too).

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by ketzo on Tuesday April 13, 2021 @03:19PM (#26797648)
Just wanna say I absolutely loved reading that blog post, thanks for the link.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by fighterpilot on Wednesday April 14, 2021 @02:12AM (#26803526)
Check out the blog post in the comment that's parallel to /u/fossuser above. It's also great.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by walleeee on Tuesday April 13, 2021 @02:12PM (#26796609)
> If it is something that's a hundred plus years out then we'll probably need whatever tech develops in the mean time to help

Circular justification of technological development is the reason unfriendly AGI is a threat in the first place (and also the reason we are unlikely to see it realized, imo; the internal combustion engine for instance poses existential risks not only to people but to the possibility of machine intelligence).

Technology is not a monolith, forms can and do preclude other forms

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by rakhodorkovsky on Wednesday April 14, 2021 @01:00AM (#26803108)
It's the critical question. The time frame is critical for reasoning about alignment and what to do now.

If you expect AGI to happen tomorrow, you throw up your hands in despair or exhilaration, if you expect it in a hundred years, you might also throw up your hands, but it's a different picture; a hundred years of development will see the development of new technologies transformative in their own right.

This excellent blog post hones in on this question: https://fantasticanachronism.com/2021/03/23/two-paths-to-the...

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by michaelbarton on Tuesday April 13, 2021 @08:34PM (#26801529)
I have thought that we might have a good a grasp on an existential problem slime AI safety, as earlier hominids did on the problem of preventing a fitter out-competing species dominating the landscape.

It might have a couple of things in common: a vague sense of danger without a specific grasp on containing it using out rudimentary tools. And once the cat is out of the bag it's near impossible to get back in.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by ilaksh on Tuesday April 13, 2021 @01:59PM (#26796428)
To me it's obvious that if we create something that is really like a digital animal or digital person then we will lose control pretty soon. Because animals are intended to be adaptive in the extreme and 100% autonomous, survive and reproduce.

But I still think we can create fairly general purpose systems without those animal-like characteristics such as full autonomy.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by jamesrcole on Tuesday April 13, 2021 @03:39PM (#26797962)
What if they’re sandboxed in some fashion? There’s zoos and jails and they’re reasonably effective at “keeping them in”.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by emiliobumachar on Tuesday April 13, 2021 @04:01PM (#26798328)
That's been conjectured under the name "AI Box".

https://en.wikipedia.org/wiki/AI_box

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by tshaddox on Tuesday April 13, 2021 @04:16PM (#26798607)
Do you anticipate goal misalignment being a more significant problem for AIs than it already is for humans? If so, why? And either way, why would we need to approach goal alignment differently than we do with humans?
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by jimbokun on Tuesday April 13, 2021 @04:28PM (#26798803)
It's more significant for AIs because we expect them to become super human, thus with potentially unlimited potential for disaster.

I would say we have been dealing with goal alignment problems with humans for most of human history.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by tshaddox on Tuesday April 13, 2021 @05:03PM (#26799274)
Why would we expect them to become super human? I would expect AIs to be able to use the latest technology and weapons, and also to develop new and better technology and weapons, and to exclude other intelligences from using said technology and weapons, but note that this is already true for humans.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by pmichaud on Tuesday April 13, 2021 @05:13PM (#26799399)
The basic answer is that unlike humans, AIs will be able to recursively self improve (in principle).
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by tshaddox on Tuesday April 13, 2021 @06:30PM (#26800253)
I don’t really buy that. Humans also improve their own abilities using technology, and I don’t see any reason to expect that technological advancements made by AIs won’t be available to humans as well. Yes, an AI group that is hostile to a human group may want to develop technology and keep it to themselves, but again, that’s already the case with different human groups (and tends to apply most prominently to our most destructive technologies).
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by TeMPOraL on Wednesday April 14, 2021 @03:31AM (#26804070)
There's an important difference: AI will be constructed using our engineering methods, not nature's.

The way humans improve their mental abilities is quite inefficient. You can boil it down to three main methods:

- Altering the chemical balance of our bodies. Exercise, diet, drugs. In its precision and scope of effect, it's not that different from beating a machine with a hammer until it improves. There's so much you can do this way, because the brain is a highly optimized system, and a part of a highly optimized system of the body. Change any parameter at random, and you're likely to make things worse[0].

- Learning. I.e. dumping information and doing repetitive rain dances, until the brain picks on the pattern we're trying to internalize.

- Outsourcing. Building external tools for thought. This is speaking, writing, language, notations, abstractions; this is TODO lists and schedules and spreadsheets; it's also listening and reading and society - because our biggest "second brain" is other people. That last trick is what let us dominate this planet.

Now take an AI constructed in silico. If it reaches close to human-level of cognition, it can already do learning and outsourcing (sans society, initially). But what it can also do is:

- Precision hardware improvement. If it's running on anything that came out of human factory, that hardware can be redesigned, improved directly, at component level. Unlike with human brain, there are people (or later, AIs) that understand how the substrate work. The factory itself can be improved too, to create even better hardware.

- Precision software improvement. Even if the AI was made accidentally, from some completely opaque ML model, by definition we know much more about even the blackest of our algorithmic boxes than we know about our brain. Core algorithms can be optimized, improved. More software constructs can be added at the IO boundaries.

Imagine how more effective you'd be if, on top of all that you are and do, you could put your TODO list, calendar and scientific calculator in your head, as well as store verbatim every book you've read, in a searchable format. Humans can't do that, we have to keep these things external and RPC through our eyes and hands. An IQ 100 human-level AI could easily make these things run in itself, or on a co-processor, with direct interface to itself - the equivalent of gaining new senses. Going by human standards, this could easily give it a boost to apparent IQ 200.

And then it could do it again, and again, and again, compounding its capabilities at every step. That's the "sudden takeoff" people are worried about.

> I don’t see any reason to expect that technological advancements made by AIs won’t be available to humans as well

They may be available, but they won't be useful. We'll always be second-hand citizens (until we figure out BCI), because the AI will be able to plug the technology directly to itself, while we'll have to interface with it through our senses and bodies. It's a difference between a process running a subprocess on the same machine with local IPC, vs. running it over a network on a machine on another side of the planet, via a very low-bandwidth API. Performance differs by many orders of magnitude.

--

[0] - See the Algernon argument - https://www.gwern.net/Drug-heuristics#algernon-argument.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by tshaddox on Wednesday April 14, 2021 @11:34AM (#26808524)
You seem to be making the assumption that we will be able to create general AI significantly before we (or the general AI) will invent general human brain-computer interfaces. While that might be possible, it seems unlikely to me.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by throwaway81523 on Wednesday April 14, 2021 @02:10AM (#26803516)
Hawkins worries about AI accidentally going nuts and destroying stuff. He ignores that it's an existential threat for the same reason nuclear weapons are: not because of potential accidents, but because of potential on-purposes. Do you really want Bezos / Musk / Biden / Putin / your favorite baddie being the first to get control of these things? It doesn't seem wise.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by MR4D on Tuesday April 13, 2021 @01:55PM (#26796372)
> AI safety....

My personal thought is that since humanity currently can't manage the I(non-A) safety, that we'll fumble through this as well.

As long as they can't replicate, we'll probably be ok, but once that changes, we're probably toast.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by gnzoidberg on Tuesday April 13, 2021 @02:14PM (#26796639)
Thus, eventually we'll be toast.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by xzvf on Tuesday April 13, 2021 @01:14PM (#26795605)
This is not something programmers and other wild HN dwellers are accustomed to hearing, but there is a very strong case to suggest that intelligence, the universal kind, is not possible without empathy. Worry about alignment in machines, not in human-likes.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by fossuser on Tuesday April 13, 2021 @01:34PM (#26795980)
I think this is objectively wrong even on a human level. I could see some part of it if you're using empathy in a general enough sense to only mean modeling other minds without caring about their goals except as a way to pursue your own (which doesn't sound like what you're saying). It sounds more like you're putting intelligence in some reference class where you're just stating it's not intelligence until it's already aligned with humans (which is not helpful).

For some reason people tend to think that general intelligence would generate all these other positive human-like qualities, but a lot of those are not super well aligned even in humans and they are tied to our multi-billion year evolutionary history selecting for certain things.

This is basically the orthogonality thesis which I found pretty compelling: https://www.lesswrong.com/tag/orthogonality-thesis - the AGI crowd has a lot of really good writing on this stuff and they've thought a lot about it. If it's something you're curious about it's worth reading the current stuff.

Some other relevant essays:

https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden...

https://www.lesswrong.com/posts/mMBTPTjRbsrqbSkZE/sorting-pe...

This talk is also a decent introduction: https://www.youtube.com/watch?v=EUjc1WuyPT8

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by xzvf on Tuesday April 13, 2021 @01:52PM (#26796308)
When we get intelligent machines they will become beings. Contrary to SV corporate-speak, you don’t align beings. You don’t align your neighbors, you don’t align people from other countries. You can’t hardwire them to follow certain principles, that’s the whole point of beings, otherwise you’re back at automated machines. All of these lesswrong posts sound so technical and philosophical and so on but in the end they all really ask 'How would you control superheroes and supervillains?' and that is not a very interesting question to me.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by fossuser on Tuesday April 13, 2021 @02:08PM (#26796554)
You do align people all the time - what is culture, story telling, arguing over what's good and bad, law? Discussion and changing people's minds? Persuasion and sharing ideas?

> "All of these lesswrong posts sound so technical and philosophical and so on but in the end they all really ask 'How would you control superheroes and supervillains?'"

They explicitly don't ask that because if you get to that point and you don't have them aligned with human goals, you're fucked. The purpose is to understand how to align an AGI's goals to humanity before they reach that level.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by xzvf on Tuesday April 13, 2021 @02:20PM (#26796721)
Then instead of superheroes think about countries or groups. How do you align Iran? How do you align the NRA? How’s culture, storytelling, arguing, etc. working for us so far? The point is that there is no simple recipe for "alignment", it will be continuous work and discourse, just as it is today with humanity. We’re talking about minds here, not machines that follow exact orders. How do you change minds?
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by fossuser on Tuesday April 13, 2021 @02:47PM (#26797153)
> How’s culture, storytelling, arguing, etc. working for us so far?

Pretty well really (despite many obvious problems) - humanity has done and is doing great things. We also have a huge advantage though that the alignment has a shared evolutionary history so it really doesn't vary that much (most humans have a shared intuition on most things because of that, we also perceive the world very similarly). For the specific examples of countries, international incentives via trade have done a lot and things are a lot better than they have been historically.

> We’re talking about minds here, not machines that follow exact orders. How do you change minds?

We agree more than you probably think? You can change minds though and you can teach people to think critically and try to take an empirical approach to learning things in addition to built in intuition (which can be helpful, but is often flawed). Similarly there are probably ways to train artificial minds that lead to positive results.

> The point is that there is no simple recipe for "alignment"

I agree - I doubt it's simple (seems clear that it is definitely not simple), but like there are strategies to teach people how to think better, there are probably strategies to build an AGI such that it's aligned with human interests (at least that's the hope). If alignment is impossible then whatever initial conditions set an AGI's goal could lead to pretty bad outcomes - not by malevolence, but just by chance: https://www.lesswrong.com/tag/paperclip-maximizer

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by coldtea on Tuesday April 13, 2021 @04:57PM (#26799190)
>Then instead of superheroes think about countries or groups. How do you align Iran?

One could start by not toppling their legimate democratic leader in the 50s, not imposing a dictatorship afterwards, and not sponsoring their neighbors to go to war with them.

Also avoiding subsequent decades of sanctions, insults, condescention, using their neighbors against them, and direct attacks and threats towards them would go a long way towards "aligning" them...

Finally, respecting their culture and sovereignity, and doing business with them, would really take this alignment to the next level...

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by _greim_ on Tuesday April 13, 2021 @01:20PM (#26795738)
This is why people talk about optimization instead of intelligence, since you side-step the baggage that comes with the word "intelligence". E.g. an optimizer doesn't need to be universal to be a problem, whether it's optimizing for social media addictiveness or paperclip manufacturing.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by xzvf on Tuesday April 13, 2021 @01:27PM (#26795823)
But optimization is decidedly not intelligence. We’ve known this for decades and have clear proofs this is the case. This is just a collective dance of burying our heads in the sand. I’ll quote this guy called Turing: “If a machine is expected to be infallible, it cannot also be intelligent”.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by _greim_ on Tuesday April 13, 2021 @02:09PM (#26796560)
> But optimization is decidedly not intelligence.

I don't think anyone is making that claim. That's why the distinction is useful.

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by DougMerritt on Tuesday April 13, 2021 @01:43PM (#26796157)
Optimization is not usually a synonym for intelligence, despite some individual's beliefs, but it can be an effective substitute at times -- chess programs, for instance, play world class chess via optimization rather than via intelligence-as-seen-in-humans.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by yaacov on Tuesday April 13, 2021 @01:30PM (#26795903)
I think you’re using the word “intelligence” to mean something entirely different from what the AI alignment crowd is worried about.
Re:Book Review: “A Thousand Brains” by Jeff Hawkins by luma on Tuesday April 13, 2021 @04:48PM (#26799065)
If there's a strong case, maybe you could put it forward for us?

You've made a heck of an assertion there...

Re:Book Review: “A Thousand Brains” by Jeff Hawkins by roywiggins on Tuesday April 13, 2021 @01:55PM (#26796369)
If that were true, sociopaths would be dumb as rocks. But, some sociopaths are actually pretty smart and don't demonstrate the sort of empathy you'd want in an AGI.