News for hackers. Stuff that matters.
Re:In mobile, 62.7% of computation energy cost is spent on d... by bshanks on Tuesday November 26, 2019 @02:28PM (#21641721)
One way in which the brain differs from conventional computers is that neurons appear to serve as both CPU and memory storage. Rather than having a few fast CPUs and a large bank of dedicated memory with a few busses (leading to "the von Neumann bottleneck" of data transfer between memory and CPU), the brain has many slow neurons each of which has many connections with other neurons.

So, in-memory computing may be more brain-like.

The brain appears to be very good at massive concurrency with low energy consumption, at the expense of slow serial computation and a high error rate.

Re:In mobile, 62.7% of computation energy cost is spent on d... by ggreer on Wednesday November 27, 2019 @03:44PM (#21651918)
The reason brains work the way they do is because evolution hill climbed to a solution, not because that solution is globally optimal. Compared to silicon, neurons are absurdly slow. Axons transmit information at 100 meters per second. Electronics transmit information 3,000,000 times faster. When your switching speed is 100Hz, the only way to get anything done in a reasonable amount of time is to be massively parallel. That's why brains are the way they are.

Other points of comparison: Animal wings (birds, insects, bats) combine both lift and thrust, but we don't fly around in ornithopters. And while animal legs combine support and power for locomotion, we don't drive around in vehicles with legs. Wheels are far more efficient, but they're not something that evolution can discover easily.

Re:In mobile, 62.7% of computation energy cost is spent on d... by kbenson on Wednesday November 27, 2019 @05:53PM (#21652972)
> Wheels are far more efficient

Wheels are far more efficient if you have a very specific type of terrain to deal with. It's not immediately obvious to me that a wheel is more efficient overall on naturally occurring terrain. For example, wheels don't work well on steep and rough terrain, or in water (as opposed to under water). The trade-off might be better stated as wheels are very efficient for very select terrain types, and that efficiency tapers off fairly sharply for some terrains, reaching 0% in some cases, while legs are less efficient overall, but achieve some level of usefulness on almost all terrains (making them a better choice for organisms that have disparate terrain types to deal with).

Similarly, a computer may be much faster in some types of operations, but we can't even get it to do some stuff that's fairly trivial for many animal brains, so that may also be a trade-off in efficiency somewhere (possibly in an area we don't sufficiently understand yet).

Re:In mobile, 62.7% of computation energy cost is spent on d... by perl4ever on Wednesday November 27, 2019 @06:18PM (#21653159)
Possibly walking recaptures energy that a simple wheel mechanism wouldn't? I vaguely think I read something about how humans are fairly efficient at recycling the energy produced as you stride and gravity pulls you down.
Re:In mobile, 62.7% of computation energy cost is spent on d... by clairity on Wednesday November 27, 2019 @06:47PM (#21653334)
yes, tendons, ligaments and even muscles are used to recapture energy as animals locomote. about 30% for walking is what i seem to remember from my grad school days.

ggreer above shows some real hubris and a deep ignorance of the ingeniousness of biological systems compared to human-created ones. a brain (and nervous system) is miraculous in its ability to gather, assess, store and discard ambiguous and contradictory information at astonishing rates.

Re:In mobile, 62.7% of computation energy cost is spent on d... by Gibbon1 on Wednesday November 27, 2019 @06:20PM (#21653165)
> fairly efficient at recycling the energy produced as you stride and gravity pulls you down

Ever walk on dry loose sand at the beach? Plod plod plod.

Re:In mobile, 62.7% of computation energy cost is spent on d... by perl4ever on Wednesday November 27, 2019 @06:29PM (#21653223)
So are you saying that's the exception that highlights the rule, or are you saying humans evolved to walk on beaches?

Cars that aren't normally driven on sand have issues too.

Re:In mobile, 62.7% of computation energy cost is spent on d... by Gibbon1 on Wednesday November 27, 2019 @08:47PM (#21654005)
I'm saying one doesn't notice until it doesn't work, then you notice.
Re:In mobile, 62.7% of computation energy cost is spent on d... by perl4ever on Wednesday November 27, 2019 @06:24PM (#21653192)
I would take exception to the idea that evolution can't/didn't discover wheels.

https://en.wikipedia.org/wiki/ATP_synthase

Everything from E. Coli, to yeast, to plants, to humans, is based on rotating molecular motors.

Now, it is true that we don't see macroscopic wheels on living organisms. But a person or animal is composed of trillions of cells. By analogy, we don't see the millions of vehicles in a nation composing an entity that has wheels made up of vehicles.

Re:In mobile, 62.7% of computation energy cost is spent on d... by perl4ever on Wednesday November 27, 2019 @06:27PM (#21653209)
"About half of all known bacteria have at least one flagellum, indicating that rotation may in fact be the most common form of locomotion in living systems"

https://upload.wikimedia.org/wikipedia/commons/3/3c/Physical...

Re:In mobile, 62.7% of computation energy cost is spent on d... by __s on Wednesday November 27, 2019 @03:54PM (#21652027)
There's a few animals that have evolved wheels. Much harder to evolve an axle
Re:In mobile, 62.7% of computation energy cost is spent on d... by mikorym on Wednesday November 27, 2019 @04:41AM (#21646509)
From a mathematical view on complexity, I don't see exactly the difference between memory and computation. I wonder what would be suitable definitions for the two concepts.
Re:In mobile, 62.7% of computation energy cost is spent on d... by arcticbull on Wednesday November 27, 2019 @12:30PM (#21649912)
This is actually reflected in modern FPGA devices. To your point, there's no material difference between "computation" and "retrieval" (they're just two different ways of mapping a set of inputs onto a set of outputs). FPGAs [1] are basically giant arrays of truth tables. These are implemented as look-up tables -- RAM -- paired with some dedicated, optimized, high-complexity logic elements like multipliers.

Could you implement a 16x16 bit multiplier (yielding a 32-bit result) as a RAM or ROM, I mean, totally. It would be a 32 bit address space where each entry is also 32 bits. That would take 16GB. You'd have the fastest multiplier in the West, though, net of propagation delay.

[1] From a combinational logic perspective.

Re:In mobile, 62.7% of computation energy cost is spent on d... by wongarsu on Wednesday November 27, 2019 @08:23AM (#21647579)
Memory is the current state, computation is the permutation of state (your step function that takes you from one state to the next).
Re:In mobile, 62.7% of computation energy cost is spent on d... by mikorym on Wednesday November 27, 2019 @08:25AM (#21647596)
This is assuming you have an automaton, I would guess? But what if you approach it via something like entropy?

Edit: Doesn't that step function encode all of memory implicitly?

Re:In mobile, 62.7% of computation energy cost is spent on d... by gpm on Wednesday November 27, 2019 @10:24AM (#21648584)
I have no clue what you mean by approaching computation from the point of view of entropy, entropy is a measure of the degree of disorder in a system, how is that a method of performing computation?

The step function does not encode memory. The step function is a function that takes memory state 1 as input, and outputs memory state 2 as output. The step function itself doesn't change, it stores no information.

In physical terms, the step function is the CPU, minus the caches, registers and other modifiable state. Memory is the disk and ram and all the CPUs caches and registers. You can't tell me anything about what the input to the CPU is, or what it is actually doing (other than that it is capable of running arbitrary x64 assembly), without looking at the state.

Re:In mobile, 62.7% of computation energy cost is spent on d... by mikorym on Thursday November 28, 2019 @12:39AM (#21654858)
A function does store information. The common definition of a function is the set (R,(X,Y)) where R is the graph of the function (consisting of the pairs) and X and Y are domain and codomain, respectively.

Computation can be seen as some act of carrying out logic. And in turn, logic can be approached via entropy. There are a lot of people working in QM, computation and entropy. But the simpler definition is just via partitions (and you especially don't have to insist on it being in the world of QM either). [1]

[1] http://www.ellerman.org/intro-logical-entropy/

Re:In mobile, 62.7% of computation energy cost is spent on d... by gpm on Wednesday December 04, 2019 @10:44AM (#21703113)
The selection of the function to use contains information. Once a function is given (e.g. the function specifying the x64 ISA), the function stores no information, it is immutable.

(Sorry about replying so late, I know it's unlikely you will see this, oh well)

Re:In mobile, 62.7% of computation energy cost is spent on d... by sfink on Wednesday November 27, 2019 @10:36AM (#21648707)
kolmogorov complexity converts the combination of program and data to entropy, fwiw
Re:In mobile, 62.7% of computation energy cost is spent on d... by tel on Wednesday November 27, 2019 @08:38AM (#21647687)
If computation is the process of spending effort to receive information, then memory is partially paid-down computation. There are lots of ways to blur these lines.
Re:In mobile, 62.7% of computation energy cost is spent on d... by shele on Wednesday November 27, 2019 @05:38AM (#21646801)
You can make sense of the differences abstractly considering the band, the state register and the instruction table of a Turing machine.
Re:In mobile, 62.7% of computation energy cost is spent on d... by jacquesm on Wednesday November 27, 2019 @08:02AM (#21647464)
A Turing machine is just one of many ways in which you could make a computing device. It is one of the easiest to mechanize and program for, which is why we use them but there is absolutely no reason to believe it is the only way of doing things. GP was getting at a more basic form of computation than the specific one performed by Turing machines. From a mathematical perspective the tape, the registers and instruction tables are all implementation details.
Re:In mobile, 62.7% of computation energy cost is spent on d... by shele on Wednesday November 27, 2019 @10:07AM (#21648428)
> absolutely no reason to believe it is the only way of doing things

Well, I did not suggests that.

> From a mathematical perspective the tape, the registers and instruction tables are all implementation details.

No, they are used to define concepts such as space complexity for Turing machines and there is something to learn from Turing machines, that's why people are studying them.

Re:In mobile, 62.7% of computation energy cost is spent on d... by pdpi on Wednesday November 27, 2019 @11:50AM (#21649498)
You can think of the Church Turing thesis as providing a formalism for how the tape, registers and instruction tables are all implementation details.
Re:In mobile, 62.7% of computation energy cost is spent on d... by mikorym on Wednesday November 27, 2019 @08:26AM (#21647604)
So the band and state register is memory? And the instruction table is computation?
Re:In mobile, 62.7% of computation energy cost is spent on d... by gpm on Wednesday November 27, 2019 @10:25AM (#21648606)
The instruction table is a function. Computation is the act of applying that function to the state.

Edit:

Grammatically "computation" is "The act of computing" where computing is a verb.

"Memory" is a noun, and not one about an act. It's more like saying "clothing".

Re:In mobile, 62.7% of computation energy cost is spent on d... by mikorym on Thursday November 28, 2019 @12:33AM (#21654837)
This kind of touches on my original point.

A function can be seen both as the set of all pairs of the function, or as an oracle that you feed input to and receive output. The first is to me the full memory space, the second is maybe computation.

Re:In mobile, 62.7% of computation energy cost is spent on d... by eggie on Wednesday November 27, 2019 @04:46AM (#21646530)
> neurons appear to serve as both CPU and memory storage

I'm not sure this is really certain. But that uncertainty is more because it's hard to prove that there isn't another means of recording memory than something we know.

There has been speculation that there might be some genetic or biopolymer based system in the hippocampus, but I'm not familiar with anything more than speculation on that topic.

Re:In mobile, 62.7% of computation energy cost is spent on d... by MayeulC on Wednesday November 27, 2019 @06:01AM (#21646905)
While there might be other mechanisms at play, plasticity is a form of memory: If each synapse can store weights, read them, and change them, it could be seen as a memory.

For me, an analogy that is quite adequate in computer science would be LUTs in an FPGA: they are memory+compute units, and can be used as both.

To go a bit further, any memory could indeed be considered a computation unit, or vice-versa: consider a results cache, for instance. The difference I see between memory and computation is that memory accesses are, if not instantaneous, at least constant-time. If you want more precision, this can be computed: the necessary memory footprint is extended trough time as well as space (if you only use a results LUT, you'd have to make it bigger to gain precision).

Re:In mobile, 62.7% of computation energy cost is spent on d... by polyterative on Wednesday November 27, 2019 @05:41AM (#21646812)
Can it be that the neural structure itself is the memory? The weak/strong relationships maybe form the memories themselves, reconsructing a vivid image like seeds for procedural algorithms.
Re:In mobile, 62.7% of computation energy cost is spent on d... by polyterative on Wednesday November 27, 2019 @06:06AM (#21646932)
This still doesn't explain our sense of time in memories. We always remember when other than what, even if some knowledge has no time positioning
Re:In mobile, 62.7% of computation energy cost is spent on d... by eggie on Wednesday November 27, 2019 @04:48AM (#21646538)
It seems unlikely to me that biological systems wouldn't exploit digital (e.g. genomic/DNA/RNA) kinds of memory storage, given that the materials are there and the energy costs of maintaining and accessing that kind of storage can be fantastically low. Keeping memory alive in neural circuits would seem much more difficult. Surely that's happening to a large degree though. I'm just highlighting the possibility of an alternative basis for certain kinds of memory in the brain.
Re:In mobile, 62.7% of computation energy cost is spent on d... by HPsquared on Wednesday November 27, 2019 @06:18AM (#21646985)
That method of storage doesn't even need to be in the brain - could be anywhere!
Re:In mobile, 62.7% of computation energy cost is spent on d... by tehjoker on Wednesday November 27, 2019 @10:28AM (#21648632)
High error rate?! I said precisely moving several complex muscles to vocalize a concept that requires simulating not only how I perceive it but how both the directed receiver and other listeners will as well while focusing my eyes rapidly on an area the size of a quarter.
Re:In mobile, 62.7% of computation energy cost is spent on d... by LeifCarrotson on Wednesday November 27, 2019 @11:18AM (#21649138)
Try doing that a couple times in a row, recording the audio and video. Can you do it the same way twice, with a measure of "same-ness" that can be trivially computed? No, humans cannot do that.

Error rate in the context of computing does not need to mean invalid, just different. Our astonishingly high abilities to compensate for variations in input data do not result from an ability to produce output data repeatably.

Re:In mobile, 62.7% of computation energy cost is spent on d... by tehjoker on Wednesday November 27, 2019 @10:58PM (#21654478)
I don't think directly comparing humans and computers are that useful as they are "designed" for different tasks.
Re:In mobile, 62.7% of computation energy cost is spent on d... by SilasX on Wednesday November 27, 2019 @10:48AM (#21648818)
Could it be feasible to use the "brainlike" model for practical hardware and reap benefits from avoiding the vN bottleneck?

My intuition is that such a system would be much harder to reason about -- and therefore harder for compilers to emit efficient machine code for -- but I'm assuming someone here knows the topic pretty well?

(Before you give me "the lecture", yes, I'm aware that in general it's not a good idea to simply mimic the kludges that evolution came up with.)

Re:In mobile, 62.7% of computation energy cost is spent on d... by lm28469 on Wednesday November 27, 2019 @11:27AM (#21649244)
Unless we made some big advances I'm not aware of, as far as I know we really don't know much about how the brain works. I doubt anyone can answer your question.
Re:In mobile, 62.7% of computation energy cost is spent on d... by SilasX on Wednesday November 27, 2019 @11:30AM (#21649288)
My question doesn't depend on knowing exactly how the brain works; it's a question about mixing memory and CPU, which happens to be similar to (how we believe) the brain works.
Re:In mobile, 62.7% of computation energy cost is spent on d... by lm28469 on Wednesday November 27, 2019 @11:38AM (#21649357)
> which happens to be similar to (how we believe) the brain works.

Can you link sources supporting that ?

Re:In mobile, 62.7% of computation energy cost is spent on d... by derefr on Wednesday November 27, 2019 @11:50AM (#21649490)
The claim you quoted is actually a null hypothesis—the negation (that the brain separates memory and processing) would require us to posit new types of as-yet-undiscovered neurochemical interactions (which may certainly exist, but we can't say!), and thus the burden of evidence would fall on proving that claim. Or, un-negated, for this claim, the burden of evidence is on disproving it.
Re:In mobile, 62.7% of computation energy cost is spent on d... by lm28469 on Wednesday November 27, 2019 @12:09PM (#21649686)
If there is nothing supporting the claim there is nothing to disprove though. I'm genuinely curious about sources because I've never heard of that.

As far as I know any comparison between a computer and a brain is flawed from the get go.

edit: and to be clear my initial comment was just hinting that we can't develop a "brainlike" computer architecture because we simply don't know how the brain works at all.

Re:In mobile, 62.7% of computation energy cost is spent on d... by TheFiend7 on Wednesday November 27, 2019 @12:40PM (#21650037)
I don't think you're wrong per se. But people aren't "claiming" the brain works this way. People are simply stating that maybe the brain functions like this and we could adopt similar designs based on our perceived impressions of how the brain may function.

We could totally botch it though and end up with an even worse computer and turns out it works nothing like the brain. Who knows. But right now many people "believe" the brain may function in the manner described.

Re:In mobile, 62.7% of computation energy cost is spent on d... by derefr on Wednesday November 27, 2019 @12:40PM (#21650026)
To be clear, what I'm saying here is that this claim maps into our model of neurology as the equivalent of a claim like "the earth isn't a triangle." Whether or not it's supported by any evidence, what it actually is is a positively-phrased restatement of part of our set of axiomatic priors.

We know that the Earth can only be one shape; we know there are an infinite variety of shapes that something can be; and so our priors contain a set of all the claims like "the Earth is potato-shaped" and "the Earth is a doughnut" all with extremely low probability, before we encounter any evidence at all, just because the probability-mass has to get spread out among all those infinitely-many claims.

Assuming continued lack of evidence either way, a claim like "the Earth is not [one particular shape]", then, doesn't require argumentative support to be taken as a default assumption (as you might in e.g. the opening of a journal paper.) The probability of it being any particular shape started very low, and we've never encountered any evidence to raise that probability, so it's stayed very low.

(Yes, that even applies to the specific claim that "the Earth is not an oblate spheroid." If we never encountered any evidence to suggest that claim, then it'd have just as low a default confidence as any of the other claims it competes with.)

For claims with no evidence either for or against them, the analytical priors derived from the facts about the classes of claims to which the claim belongs, determine where the burden of proof lays. Low-confidence priors? Burden to prove. High-confidence priors? Burden to disprove.

In this case, we already know that neurons can do several things, and AFAIK we've never encountered any evidence of neurons having specialized functionality, or any evidence that neurons don't have specialized functionality. Our tools just aren't up to telling us whether they do or not. But, because one claim actually factors out to several claims (neuron specialization → lots of different ways neurons could specialize) while the other claim doesn't (neuron generality → just neuron generality), the probability-mass ends up on the neuron-generality side. (This is another way to state Occam's Razor.)

Mind you, this might be entirely down to our inability to study neuronal dynamics in vivo in fine-enough detail. In this case, our lack of evidence doesn't imply a lack of facts to be found, because we have no evidence for or against this hypothesis. Instead, it just determines what our model should be in the absence of such evidence, until such time as we can gather evidence that does directly prove or disprove the specific hypothesis.

Or, to put that another way: if humans only ever studied bees from a distance, the default hypothesis should be that all bees do all bee jobs. The burden of proof is on the claim that bees specialize. Later, when we get up-close to a beehive, we'd learn that bees do specialize. But that doesn't mean that we were incorrect to believe the opposite before. Both our belief before the evidence, and our belief after the evidence, were the "correct" belief given our knowledge.

Re:In mobile, 62.7% of computation energy cost is spent on d... by SilasX on Wednesday November 27, 2019 @12:33PM (#21649953)
I was just going off the claims at the top of the thread:

https://news.ycombinator.com/item?id=21641721

I have no special expertise otherwise; if you want more substantiation or wish to dispute that point, you could post a reply where the claim was originally introduced.

Re:In mobile, 62.7% of computation energy cost is spent on d... by amelius on Wednesday November 27, 2019 @07:15AM (#21647223)
Yes but in a computer every processing unit can be used for widely different tasks. In contrast, the neurons in a brain are used for the same task over and over again.
Re:In mobile, 62.7% of computation energy cost is spent on d... by mojuba on Wednesday November 27, 2019 @07:48AM (#21647387)
But you could as well say, the same part of computer's memory where the execution binary is stored is used for the same thing again and again.

I don't think these parallels between the computer and the brain are valid though. The brain is a highly parallel but slow machine, we have very fast and not-so-parallel computers (just a few cores). I think a lot of what the brain does can be expressed on computers without changing the architecture much, i.e. you don't need to compare the architectures, just the outcomes of computation.

Re:In mobile, 62.7% of computation energy cost is spent on d... by TheFiend7 on Wednesday November 27, 2019 @01:25PM (#21650472)
But surely we can improve modern computing devices architecture by taking a look at the computing advantages the brain takes in order to achieve its desired tasks.

Don't forget humans are one of the most expensive resources and nothing can really replace them for even a vast majority of non-physical jobs. So if we could build computing devices that potentially solve some of these problems, needless to say, the productivity boosts would be plentiful.

Secondly, we aren't even close to being able to model anything relatively close to the computational capabilities of the human brain because we don't even understand the human brain. So your comments on highly parallel but slow and fast but no so parallel don't make a lot of sense.

For example take MapBox's new vision SDK. It's able to perform semi-decent feature extraction on the road while people drive via a camera. Well guess what I would absolute stomp the vision SDK on accuracy for every feature it thinks it identified, not only that, I am capable of identifying an order of magnitude more features than it can, not only that, but I'm able to identify new features on the fly and even guess with greater accuracy what they are.

So yeah, there are a plethora of functional yields that we have yet to achieve with some of the most powerful computers in the world that are achieved by the human brain every day. Which could be indicative of maybe both a resource, but also an architecture problem.

Re:In mobile, 62.7% of computation energy cost is spent on d... by mojuba on Wednesday November 27, 2019 @04:51PM (#21652513)
> I would absolute stomp the vision SDK on accuracy

I suspect our recognition abilities may in fact be worse than that of a good SDK but we have the advantage of a general real world experience. Bit of a simplified example, in order to recognize a dog you should have seen a lot of various animals, have a basic knowledge of anatomy, animal behaviour etc. A lot of the times recognizing the context helps too: e.g. something on a lead ahead of a walking human in the street, likely a dog - you need only a very quick confirmation to say it's definitely a dog, etc.

I also think the brain does a lot of tree searches with optimizations (which are never perfect), and it's where the brain's parallel architecture proves to be beneficial.

Intuitively though, modern computers are still not powerful enough to perform the same tasks albeit mostly sequentially. I believe we'll get there and I think AGI in a simplified virtual/gaming environment is the best place to test our approaches.

Re:In mobile, 62.7% of computation energy cost is spent on d... by didericis on Wednesday November 27, 2019 @07:33AM (#21647316)
> the neurons in a brain are used for the same task over and over again

How does this square with the fact that people can get large portions of their brain scooped out and still retain many of their previous abilities?

This is a naive assumption, but if there were purely a 1 to 1 correspondence between a neuron and any given task, I’d assume you wouldn’t be able to recover from massive brain damage. But it seems like people can retrain the functional parts of their brain after damage to other parts to pick up the slack somehow.

Re:In mobile, 62.7% of computation energy cost is spent on d... by KoftaBob on Wednesday November 27, 2019 @09:19AM (#21647998)
Super fascinating, any recommendations on where I can do further reading on this?
Re:In mobile, 62.7% of computation energy cost is spent on d... by Richard_East on Wednesday November 27, 2019 @09:31AM (#21648083)
Low energy consumption checks out - a brain uses only 12 watts.
Re:In mobile, 62.7% of computation energy cost is spent on d... by lm28469 on Wednesday November 27, 2019 @11:25AM (#21649217)
You have to feed and train it for the first 15+ years before it's any good though.
Re:In mobile, 62.7% of computation energy cost is spent on d... by TheFiend7 on Wednesday November 27, 2019 @12:43PM (#21650062)
Brutal truth, that's also assuming 100% success rate. Lots of brains turn out to be shitty brains after 15+ years too...
Re:In mobile, 62.7% of computation energy cost is spent on d... by 8bitsrule on Wednesday November 27, 2019 @05:03PM (#21652624)
"Insanity -- a perfectly rational adjustment to an insane world." — R. D. Laing
Re:In mobile, 62.7% of computation energy cost is spent on d... by anticensor on Wednesday November 27, 2019 @04:19PM (#21652229)
Your brain uses 12 watts at rest and 120 watts under stress.
Re:In mobile, 62.7% of computation energy cost is spent on d... by varjag on Wednesday November 27, 2019 @04:24AM (#21646450)
There is no CPU in brain, some analogies don't go very far.