bl
In constructing my incorrect answer in the grand-parent comment, my though process was being guided by the way Matlab/numpy treats these items (and I think I'm on solid ground that Matlab/numpy treat them differently because mathematicians consider them differently). The built-in functions operate very differently (if they work at all) for
size(A) = (m,1)
and size(A) = (m,n≠1)
So there may be 1000 numbers floating in the ether, but conceptually they're not the same. Multiplying a 1000x1 vector by a 1xp vector has a completely different result than multiplying one thousand 1x1 vectors by that same 1xp vector.Although, only many hours later do I realize that the original submission title might've been wordplay on the phrase "a picture is worth a thousand words", so my brain is not reliable today. I shall refrain from spewing more-likely-than-not incorrect statements concerning linear algebra.
Imagine three, 4-by-1 vectors, each "one-dimensional". Twelve total scalars, each vector with four rows and one column. Arrange these three vectors side by side and merge them into a single 4-by-3 matrix. This matrix is "two-dimensional".
Now, let's imagine five such matrices, each 4-by-3. Stack the five matrices one on top of the other. We currently have a 4x3x5 matrix. This matrix, which contains 60 scalars, is "3-dimensional".
Repeat a similar exercise 997 more times and you have a 1000-dimensional matrix.
Compare that matrix to this: 1000 of our original 4-by-1 vectors arranged side by side, which gives a 4x1000 matrix, which is simply a "two-dimensional" matrix with 4000 elements.
Believe it or not, I paused for half a heartbeat while typing fixed and triangulation while I was typing and thought to myself "Should I go double-check this on some hard-to-access reference, say Wikipedia? Nah. Better blurt this out before I head out the door for the day".
I ought to be careful how I phrase my gripes so that they convey my intention: keeping discussions accurate about subjects which I know a bit. Poor metaphors seem to aggravate me more than most.
Thanks for setting me straight.
Another item of note: During introductory neuroscience class, the idea was "Hippocampus, learning & memory. Hippocampus, learning & memory. Hippocampus, learning & memory." Perhaps not so brutally, and I may have missed some subtleties. I found it quite refreshing to learn that the hippocampus was involved so deeply in a task that wasn't so starkly "This arbitrary experimental task is aversive; I, mouse, must avoid it." Hippocampus...is there anything it can't do? <Leonard Nimoy: "The answer is yes.">
Following is a neuroscientist's get-off-my-lawn rant
or
A Comment Wherein `bl Gripes about Stretched Tech Analogies Concerning Neuroscience.
A GPS device receives signals from a set of beacons with well-established, fixed locations (i.e., geostationary satellites). Knowing the beacons' fixed locations, the device is able to triangulate its own position. A GPS device can go to a completely unfamiliar location and work exactly as well as when its at a location its been to hundreds of times.
One could conceivably think of a mammalian visual system surmounted on a position-encoded cranium as equivalent to the GPS beacon system. But hippocampal place cells would have absolutely no contribution to navigating an unfamiliar environment because they had not been "trained".
But let's do a thought experiment and see if we can reasonably dispense with the need for experiment itself. The article (via the researchers' statements) zoomed past a detail: cortical surface area is much more indicative of neural processing capability than gross brain volume. In many contexts, a neuroscientist might use "brain size" as shorthand for cortical surface area. Also consider that more "advanced" mammals tend to have more convoluted cortexes, thus larger cortical surface areas. So it's quite possible for a large mammal's (whale's or elephant's) brain to be volumetrically larger than a human's, but to have relatively smaller surface area because it is less convoluted.
In the event that we could actually accomplish such a comparative study as you propose, we'd probably find that "tethering" does not monotonically increase with surface area. Then we'd determine that the authors' hypothesis is overly simplified.
I merely want to avoid prematurely narrowing the range of functions that are possible. If we, for the moment, think of the neural computation of a single neuron as a neural network, then the spike/no-spike decision would be in the last layer and a whole host of linear/non-linear (some not necessarily boolean) functions could be implemented by the dendrites. And some single neuron processing we already know behaves in a non-boolean manner.
Be aware, just because arbitrarily powerful logic could be constructed solely out of boolean components (I don't even know if this is true. Isn't this kinda what is going on in an FPGA?) doesn't mean that neural hardware is purposed the same way. They may very well may be analog, at least for some computations.
And to speak to your second paragraph, I should declare my personal biases. As a dendritic physiologist, I wasn't much interested in whole-cell firing characteristics, but in the dendrite's sub-threshold behavior.
How do a smattering of synaptic inputs, each with varying strengths, interact within the complex electrophysiological scaffolding provided by a branched dendrite layered with non-uniform, non-linear ion channel distributions?
So my perspective is somewhat inverted: To me, neuron firing is the implementation detail! <smilie face>
It is overly simplified to say that for the processing to be useful that it has to share the result. If a particular part of the dendritic sub-tree was stimulated enough, it could bring the neuron into a particular electrophysiological state that succeeding synaptic input would cause a wholly different computation to occur. Thus, you can see the importance of timing discussed by the sister comment.
That's my outlook on the structure-function link between dendritic morphology and dendritic information processing, with the modification that I'd not restrict it to boolean functions. There are very many more types of functions, linear and non-linear, that can conceivably be built out of neuronal dendrites.
And I like the nuance of your second paragraph. There are all sorts of wacky, complex calculations one can image being possible, but any one neuron may implement a subset. Now, across a few hundred billion neurons in a mammalian nervous system...
You're spot on with regard to timing, too. All this "information processing" with branched dendrites + non-linear ion channels are greatly expanded with a timing component.
In some cases, depending on the actual geometry of the dendrite and the particular complement of voltage-activated ion channels, the voltage change as a result of neurotransmitter release might lead to quite a distributed depolarization even without triggering a dendritic action potential.
Conversely, an action potential initiated in the dendrites doesn't necessarily faithfully propagate to the cell body (soma). This is also dependent on the local geometry and ion channel distribution. Dendritic action potentials are not all-or-nothing events like those of the axon.
To answer your question: Smith, et al., did observe dendritic action potentials (spikes) by measuring a proxy: calcium influx indicated by a fluorescent dye that changes efficiency when bound to calcium. This calcium influx, and by extension, the dendritic spike, is what was spatially-restricted. The authors are extrapolating information processing from the spatially-restricted dendritic spike.
Active dendritic information processing has, for several decades, been theorized and modeled. The combination of two-photon microscopy and more "classical" electrophysiology techniques (like patch clamping used in this article) is finally opening the theories to experimentation.
[Not to be too critical, but this paper is far from the first to experimentally investigate dendritic information processing. I, personally, am glad some segment of HN is interested in neural computation.]
Previous Hacker News discussion of Mr. Park's original piece: http://news.ycombinator.com/item?id=2753343
For example, the Purkinje neurons of the cerebellum:
http://www.gladstone.ucsf.edu/wp/2009/06/knowledgegrows/
Note how the number of branches increases to sample more of the volume.
All my statements are based on my understanding of mammalian learning and memory. But I think you hit on the key with "Perhaps her neurons are different".
Indeed, invertebrate neurons are wildly different from those of mammals. In fact, if you are accustomed to looking at mammalian neurons [1], invertebrate neurons can look positively alien. For example, check out the Lobula Giant Movement Detector (LGMD) neuron of the locust [2] and other insects [3].
A) The scale is different: The thickness of some of its branches are about the size of the cell body on a mammalian neuron.
B) The organization is different: The dendritic arbor is divided into nearly independent subfields with very independent functions.
C) The behavior is different: The spike output patterns of an LGMD would be distinguishable to a first-year neuroscience student. And the output connections are extremely strong, pretty much one-to-one.
Add it all together, and this one neuron does the job of at least a few dozen mammalian neurons. How many, exactly, is difficult to tell. Not every insect neuron is as fantastical as the LGMD, but I would say that "600000" value ought to be scaled by some number greater than five. Given that, one could say our spider friend has the equivalent of several million mammalian neurons.
Raw neuron count is merely the crudest of measures of neural processing capability. How sophisticated the processing nodes are (i.e., the neurons) and how they are wired together (i.e., the network topology) are way more critical.
[1] Note that http://en.wikipedia.org/wiki/Neuron depicts exclusively mammalian neurons.
[2] Locust version: http://jn.physiology.org/content/97/1/159/F1.large.jpg ; figure from this article: http://jn.physiology.org/content/97/1/159.full
[3] Fly version, top row; rat (i.e., representative mammalian neurons) for comparison, bottom row: http://c431376.r76.cf2.rackcdn.com/995/fnsys-03-017/image_m/... ; figure from this article: http://www.frontiersin.org/systems_neuroscience/10.3389/neur...
Indeed. It appears more akin to the situation we have with classical and quantum mechanics: classical (Newtonian) mechanics fairly accurately describe the behavior of macroscopic objects at relatively slow velocities. We acknowledge that the system is not valid outside of that range. But since that range encompasses the bulk of our everyday, practical experience, classical mechanics are exceedingly useful.
Classical optics are not suddenly outdated with these discoveries. Snell's law and the lens-maker's formula are just as relevant as they were yesterday. We just need to add a few more terms to the equation if we etch a gradient of nano-scale resonators to the surface of our optical element. (Boy, do I feel like Geordi in Star Trek reading that last sentence aloud.)
Negative index of refraction materials are not quite new: wikipedia indicates that they are already being used in commercially-available products (http://en.wikipedia.org/wiki/Negative_index_metamaterials). For me, the most exciting application is creating a lens that circumvents the diffraction limit that limits optical imaging resolution. Right now, the most sophisticated, expensive microscope objective lenses can just barely resolve sub-cellular structures only under very particular conditions (i.e., not alive). A diffraction-unlimited "superlens" made out of this stuff could enable us to see even smaller objects under physiological conditions. It would be fantastic if we could capture the release of individual neurotransmitter vesicles at a diseased synapse, for example.
Besides negative index materials, there are a whole class of non-linear optics (http://en.wikipedia.org/wiki/Nonlinear_optics) that allow engineers to do all sorts of funky things in their instruments, like doubling the light frequency or self-focusing.
I agree 99%. The one thing they do for us, and the one thing that any alternative system would have to replicate, is provide a signal of quality through their history and prestige. The whole constellation of scientific publications, grant writing, grant reviews, tenure committees, faculty searches, and dissertation committees revolve around publications. But it seems nobody can, or desires, to read all of an individual's publications, synthesize the content of what they've produced, and compare it to the other work in increasingly specialized sub-fields.
Instead, to evaluate the quality of work and the scientists themselves, we rely on the number of articles published multiplied by the journal's impact factor (http://en.wikipedia.org/wiki/Impact_factor), roughly speaking. This we've used as a proxy for scientific quality. And besides lazy, it's insidious, too. Before even sitting down to read an article, I've already absorbed which journal this article appeared in. If a prestigious journal, how can I help being predisposed to think favorably of it?
It's a seductive shortcut. This is why the academic journals are so entrenched. And it shows what any alternative system must replicate. It is relatively easy to create a website that unites authors, peer reviewers, editors, and a publishing/editing system. What is not straight-forward is to create a system with that last 1%: the external quality signal.
I'd guess that the IBM team would have to replicate a lot of that "hardware" to get the emergent behavior of a piece of cortex. I do know that computational neuroscientists have struggled for the last 15 years or so get reduced neuron models (i.e., point neurons with a small number of rudimentary synapses) to even crudely mimic full-featured neurons. It looks like this group recognizes this issue in that they are already starting with many hundreds of synapses.
Another thing to keep in mind is that a processor inspired by brain hardware will be most likely very efficient in the tasks a mammalian brain is good at (pattern recognition, pattern completion, pattern separation, etc.), but will concomitantly be worse at things that a standard serial instruction processor excels at. I'd bet that the IBM folks are looking to merge a cognitive processor and a classical processor into a single unit.
Also, I know very little about compilers, assembly language, and other close-to-the-metal issues, but it appears that this processor would be very different to program.
div.example_class
Content here...
p#id_example
More content...
From the previous discussions here on HN there were many skeptics, but I have found that it has greatly reduced mental friction when I have to wrestle with HTML: the markup is fast to write, a joy to read, and I get warm, fuzzy feelings knowing that my HTML source is not drowning in markup soup.Caveats:
1. It doesn't appear to be actively maintained. (But it is distributed as a single, readable python script that one can easily modify to taste. It spits out plain HTML, so if you ever want to ditch SHPAML, you will never be left high and dry.)
2. Doesn't include a round-trip translator. (Not terribly important to me.)
3. Like CoffeeScript and their ilk, the output line numbers don't line up with the source.
4. There are a few bits of gymnastics one has to do to conform to the grammar. For example, a link anchor must appear isolated on a new line to use the most absolutely tersest syntax. Since SHPAML passes HTML straight through, you can avoid this by added the standard markup.
5. SHPAML may not be automatically included in your workflow if you are using something like Rails or Django, but I believe there are a set of github projects that address this.
Mr. Jobs has done a remarkable job producing sleek, fun-to-use, affordable, somewhat disposable devices and building Apple into a company that has a chance of sustaining his principles. Even taking into account the fickleness of the mass consumer market, herding sub-component suppliers, etc., there is a certain predictability to the advances in electronics technology (Moore's "Law", increases in battery performance, etc.) that gives a solid foundation for a regularly growing business. Fashioning aluminum, cadmium, silicon, and lithium is way more predictable than discovering a pharmaceutically-treatable, financially-lucrative metabolic disease.
There are certainly smart, extremely hard-working individuals in the biotechnology fields --- perhaps surpassing Mr. Jobs in those qualities, and some we'd certainly label as "geniuses" --- but the fantastically low probability of making even one genuine discovery, and the low odds of successfully developing that into a product, make the kind of serial successes of Apple and Mr. Jobs very unlikely in the biotech field. Biology results just don't flow that easily. Also, there is no clearcut equivalent in biological research to outbidding your competitors for the latest components, streamlining your manufacturing and inventory processes, and relying on economies of scale.
I can say this now only with the benefit of hindsight: my negative answer to bh42222's question stems from the fact that we just haven't observed such figure emerge in the last few decades. Craig Venter is a good example to look at. He certainly earned a great victory in the human genome sequencing (and the underlying technologies), but he lacks even a second notable success of that magnitude. It will probably be the purview of someone else to find the utility in the sequenced genome. Dean Kamen may be a prototype figure of this sort, but as nice as his individual inventions are, no single one of them seems rise to the level of a Steve-Jobs-in-the-biotech-field. I may be stunningly wrong on this (and it would be great fun if I was), but I think the probabilities are on my side, and on the side of it not happening.
But let's put aside the negativity and allow me to ask this question: Do we need a Steve Jobs of biotechnology to fulfill our expectations of medical advancement? I would guess, "No, we don't. The good people toiling away at it right now are doing the best that can be done."
It's still incredible to me that Apollo 13 was the worst outcome and that the Apollo program, overall, ran "on schedule".
Oct 1968 Apollo 7: earth orbit
Dec 1968 Apollo 8: lunar orbit
Mar 1969 Apollo 9: earth orbit lunar module docking
May 1969 Apollo 10: lunar descent test (no landing)
Jul 1969 Apollo 11: lunar landing
It seems so composed and laid-out, and at the same time so daring. There is a clearly logical progression there, each mission building on the previous one. But each new mission followed so soon after the previous, and each new mission involved so many new goals. It's wondrous that it just worked out that way with so few missteps. Could they really internalize the lessons? With so many moving parts and subsystems, so many hundreds of thousands of workers, spread across the country in goverment offices and subcontractors, I imagine the safety mindset varied wildly. I know more than a few astronauts certainly conducted themselves recklessly on the ground and in training, and there were large swathes of folk who were fanatical about safety.
Perhaps if we had an increased sample size, we'd know which way the Apollo project leaned on safety and methodology. Records show that the NASA administrators were anxious about losing a crew and were all too happy to cancel the missions after Apollo 17 (in addition to saving money for future projects).
I heartily recommend Andrew Chaikin's A Man on the Moon for an excellent account of the Apollo program.
Clouding the issue even further is the fact that different areas of central vision processing handle different features. Some areas are tuned to react to points, some to bars, some to grids, some to movement, some to rotation, etc. Perhaps it would be simpler to make the comparison in the other direction: "How intricate do I have to make this visual scene to fully exercise the perceptual abilities of grating-sensitive neurons in the Lateral Geniculate Nucleus of the thalamus." That way, we could put some sort of upper bound on the useful specifications of a pixelated display. We would then have to iterate over all the known feature sensitivities (bars, grids, rotations, etc.).
Display engineers surely must know that below such-and-such pixels per inch, a screen can present any reasonable perceptible pattern. So outside of casual interest, ristretto and I think that eye "megapixels" is relatively meaningless.
Thus, the mapping of "pixels" to the number of retinal ganglion cells is probably less tenuous than a photoreceptor-to-pixel mapping. Now that I think about it, perhaps an even better definition of a physiological pixel would be a functional measure: the number of distinct, electro-physiologically measured center-surround fields in the thalamus. In that case, the effective megapixel rating of a human visual system is only indirectly related to the sheer numbers of photoreceptors, but more closely related to the wiring pattern. This wiring pattern is much more difficult to experimentally measure than simply counting neurons because it would involve flashing tiny, contrast-y dots of light in front of a fixated mammal while poking an electrode around in the thalamus. [This is outside of my main field, so it may have been done, but I don't know the results.]
Two side notes of interest:
1. Photoreceptors are oriented towards the rear of the retina and are embedded in a dark sheet of cells called the pigment epithelium. The upshot of this is that every photon that is involved in our visual perception has traversed a tangle of bipolar cells, ganglion cells, and their associated axons (it is easy to overlook this fact in diagrams, such as at http://en.wikipedia.org/wiki/Retina , because it is usually only mentioned textually in the figure caption). The fovea is relatively free of these light-scattering objects and, in addition to a higher ratio and density of cone cells, is why primate visual acuity is highest in the fovea.
2. In extremely low light-adapted rod cells, the absorption of single photon can trigger photo-transduction. Thus, our visual system has the capacity to operate at the very limit of physics. If I recall correctly (i.e., no citation on hand), this has even been experimentally demonstrated, although the experiment must have been pretty demanding, what with photon shot noise and all. [This last side note #2 is what I had in mind when I referred normal visual processing in the first paragraph. Even though you might think that this validates the photoreceptor:pixel metaphor, at best a human would probably just report a tiny inkling of a flash in some general vicinity with very poor spatial and temporal resolution.] Ah, I now see ristretto mention that.
All that said, a relevant question to ask is, "What is the information content of a single action potential?" There is not an agreed-upon answer among neuroscientists. The metaphor linking brain and a personal computer is extremely strained: to start, it is not quite true that neurons can carry out logical functions. Neurons are firmly rooted in the analog domain and it is much more straightforward to infer polynomial-type computations from their physiology, the order and coefficients being dictated by the particulars of the scenario. Also, the current thinking is that information is encoded in bursts or repetitive APs, called "spike trains". But if we must, I'd throw out a range of 0.5 to 5 bits per spike [Rieke, et al.]. (If you want to go down a rabbit hole and explore one of the pivotal topics among people studying neural computation, look up "rate coding" versus "temporal coding". Perhaps even phrasing the question that way is misleading.)
Here's a side note that may be of interest: during an AP, the trans-membrane voltage swings from approximately -100 mV to +100 mV (very rough figures). If you consider that this potential difference is applied across the 3 nano-meter thick cell membrane, the resulting electric field is very close to the dielectric breakdown voltage for phospholipid membranes. Our neurons are constantly operating within a factor or two of literally destroying themselves! (The preceding paragraph contains very back-of-the-envelope reasoning. To be more rigorous, I'd have to dig out my notes and reference materials, but the general point nonetheless holds.)