News for hackers. Stuff that matters.
Re:Maryland research could improve the AI task of sensorimot... by giardini on Thursday May 16, 2019 @10:09AM (#19929193)
Related older easy-to-read article, paper and seminar lecture:

1. "Binary holographic reduced representations for SWI-Prolog" by Jocelyn Ireson-Paine:

http://www.j-paine.org/hrr.html

2. "Dual Role of Analogy in the Design of a Cognitive Computer" by Pentti Kanerva, in Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences. Workshop. Sofia, Bulgaria, July 17-20, 1998:

http://faculty.cs.tamu.edu/choe/mirror/kanerva.ANALOGY98-kan...

3. Pentti Kanerva lectures at Stanford on "Computing with High-Dimensional Vectors":

https://www.youtube.com/watch?v=zUCoxhExe0o

4. Additional information, including the slides, for Kanerva's lecture are available at:

http://web.stanford.edu/class/ee380/Abstracts/171025.html

Re:Maryland research could improve the AI task of sensorimot... by evrydayhustling on Thursday May 16, 2019 @09:51AM (#19929028)
This is probably a "publishing in the press" translation issue, but the notion that neural networks have no memory is dead wrong. DNNs encode incredibly specific histories of training data in their weights, which is both a privacy/security risk [1] and a phenomena that is actively being utilized in many dnn architectures. Overall, this has a "brand the framework before proving exceptional capabilities" feel.

[1] https://arxiv.org/abs/1802.08232

Re:Maryland research could improve the AI task of sensorimot... by gnode on Thursday May 16, 2019 @10:21AM (#19929303)
I don't think that's what they meant. I interpreted it to mean: neural networks are not trained in a continuous online manner.

That said, there are neural network architectures which have state, such as recurrent neural networks, time-delay neural networks, and long short-term memory. However, the state is used for covering problem domains with a temporal nature, rather than for reflective learning. It's typically reset between different inputs.

Re:Maryland research could improve the AI task of sensorimot... by sgillen on Thursday May 16, 2019 @04:44PM (#19933004)
There are lots of examples of networks being trained online, I see it all the time in robotic control.
Re:Maryland research could improve the AI task of sensorimot... by taneq on Thursday May 16, 2019 @09:15AM (#19928703)
These HDVs (hyperdimensional binary vectors) sound a lot like the SDRs (sparse distributed representations) that Numenta talks about with their HTMs (hierarchical temporal memory).

Wow, what a lot of TLAs.

Re:Maryland research could improve the AI task of sensorimot... by andbberger on Thursday May 16, 2019 @09:34AM (#19928864)
That is probably not a coincidence. Pentti Kanerva, who does a lot of work on hyperdimensional computing, is at the redwood center for theory neuro which Hawkins (numenta) helped found.
Re:Maryland research could improve the AI task of sensorimot... by evgen on Thursday May 16, 2019 @10:13AM (#19929222)
Kanerva is also the author of the (vastly underrated IMHO) book Sparse Distributed Memory from the late 80s that set the stage for a lot of this and predates Numenta by a couple of decades...
Re:Maryland research could improve the AI task of sensorimot... by taneq on Thursday May 16, 2019 @10:42PM (#19935764)
Huh, thanks for the reference. I have heard elsewhere that the one issue with Numenta's work is that they aren't particularly proactive about citing prior art, so it's hard to know (as a newcomer) what parts are their innovations and what parts are incremental improvements on existing techniques.
Re:Maryland research could improve the AI task of sensorimot... by plutonorm on Thursday May 16, 2019 @09:17AM (#19928716)
Hyperdimensional binary vectors??

What like:

010101010111010101010111011001

So they are encoding perceptions and actions into the same vector space. Cool. But the marketing is just gibberish.

Re:Maryland research could improve the AI task of sensorimot... by jbotz on Thursday May 16, 2019 @09:59AM (#19929098)
Not really. "hyperdimensional binary vector" may sound rather buzzwordish, but it has a fairly precise definition in this context, and it may indeed be highly significant for the future of neural computing. You can learn more about it with the code (and paper) here: https://github.com/alexanderganderson/bnn_vis
Re:Maryland research could improve the AI task of sensorimot... by plutonorm on Thursday May 16, 2019 @01:36PM (#19931032)
Err. Just read the abstract. It's a binary vector. Nothing more. That these high dimensional binary vectors behave similarly enough to real vectors for the purposes of training binary neural networks in no way justifies the name "hyperdenional binary vector". It's a vector of Booleans.
Re:Maryland research could improve the AI task of sensorimot... by sgillen on Thursday May 16, 2019 @05:29PM (#19933484)
You are technically correct, it's just that when the vector of booleans gets long enough they start to exhibit certain properties that "normal" binary vectors don't have. Because of this the authors use the term "hyper dimensional vector" to let other professional researchers quickly know what they are talking about. Here's a decent blog post about some of those properties: http://gigasquidsoftware.com/blog/2016/02/06/why-hyperdimens...

I kind of agree that "hyper-dimensional" sounds like a dumb buzzword, and maybe it is. But I do think we should have a word for vectors that are big enough that they fall in this regime.

Re:Maryland research could improve the AI task of sensorimot... by plutonorm on Friday May 17, 2019 @12:47PM (#19941304)
I'm slightly more convinced. But it does seem a pretty silly name. Also I have read about these counter intuitive properties of large vectors in other places without this specific name having come up. But, yeah, interesting link, thanks :)
Re:Maryland research could improve the AI task of sensorimot... by gre on Thursday May 16, 2019 @09:20AM (#19928743)
Not posting your machine learning paper on arxiv.org is a red flag.
Re:Maryland research could improve the AI task of sensorimot... by andbberger on Thursday May 16, 2019 @09:35AM (#19928869)
What?? arxiv is a _preprint_ server
Re:Maryland research could improve the AI task of sensorimot... by hsdfhsdal on Thursday May 16, 2019 @10:30AM (#19929370)
What about vixra.org?
Re:Maryland research could improve the AI task of sensorimot... by p1esk on Thursday May 16, 2019 @11:33AM (#19929929)
For papers so bad that even arxiv refuses to publish them? Sure, why not...
Re:Maryland research could improve the AI task of sensorimot... by billconan on Thursday May 16, 2019 @09:51AM (#19929022)
I do not understand the point of this paper. Say, deep reinforcement learning also encodes perception/state into a large vector. Combining perceptions into a large vector doesn’t seem to contribute much?
Re:Maryland research could improve the AI task of sensorimot... by gnode on Thursday May 16, 2019 @10:38AM (#19929437)
The article explains so little, so it's hard to know.

Neural networks generally don't learn by themselves, they require some kind of optimisation outside of their operation. It seems to me that they're saying this is more of a cognitive architecture which learns continuously by memorisation.

Re:Maryland research could improve the AI task of sensorimot... by tentboy on Thursday May 16, 2019 @02:22PM (#19931468)
Unfortunately I have very little experience with AI or ML. I did however have the lead contact on this paper (Yiannis Aloimonos) as a professor for computer vision my senior year at Maryland.

He spent most of the lectures rambling about his research, including this, rather than the course topic. I learned the course material on my own but still enjoyed listening to him.

Re:Maryland research could improve the AI task of sensorimot... by mklyons on Thursday May 16, 2019 @03:22PM (#19932145)
Yiannis is a fantastic professor, probably my favorite during undergrad at Maryland.
Re:Maryland research could improve the AI task of sensorimot... by vcavallo on Thursday May 16, 2019 @09:18AM (#19928730)
this seems like a pretty significant step towards a sort of “holistic” agent whose past informs its present and future in a more human-relatable way.
Re:Maryland research could improve the AI task of sensorimot... by cs702 on Thursday May 16, 2019 @09:17AM (#19928718)
This is a very click-baity headline. I'm tempted to flag it!

Has anyone here seen the actual paper?

Re:Maryland research could improve the AI task of sensorimot... by nabla9 on Thursday May 16, 2019 @09:31AM (#19928833)
Re:Maryland research could improve the AI task of sensorimot... by p1esk on Thursday May 16, 2019 @09:57AM (#19929080)
Closed access journal for machine learning in 2019? No thanks
Re:Maryland research could improve the AI task of sensorimot... by valgor on Thursday May 16, 2019 @10:01AM (#19929111)
The paper is linked in the article if can be bothered to read what you are threatening to flag...
Re:Maryland research could improve the AI task of sensorimot... by cs702 on Thursday May 16, 2019 @01:41PM (#19931082)
FYI, someone (probably a moderator) changed the headline after I wrote my comment. I'm OK with the new headline.
Re:Maryland research could improve the AI task of sensorimot... by vcavallo on Thursday May 16, 2019 @09:19AM (#19928735)
have you read the content of the link? i wouldn’t flag it
Re:Maryland research could improve the AI task of sensorimot... by cs702 on Thursday May 16, 2019 @01:39PM (#19931065)
FYI, someone (probably a moderator) changed the headline after I wrote my comment. I'm OK with the new headline.