We get about 1 Gb/min of neurophysiological data (x4-6 hours/day) and I'm hoping to scale that up quite a bit soon. People doing microscopy also generate giant datasets, as do the sequencing folks.
Me, specifically? A labmate showed me and said "It's cool...and a timesink."
What do you think about this? Who is the I? Where can I find it? Is it just mysticism?
I'm collaborating with a woman who has also been working with him to model some of our data with NENGO, so I should really get around to that sooner rather than later.
What are the best modern methods to analyze them?
CRCNS (https://crcns.org/data-sets) has some neurophysiology data (i.e., from implanted or inserted electrodes). This sort of data is shared a little less often, in part because it's often acquired and stored in weird, homebrew formats, though that's slowly changing.
ModelDB (https://senselab.med.yale.edu/ModelDB/) has a large collection of computational models. These are mostly biophysical models, though there's some other stuff in there too.
Depending on what you're looking for, there are other more specialized repositories. NDCT (https://data-archive.nimh.nih.gov/ndct) has mental-health related clinical trial data, though you'll have to do some paperwork if you want subject-level data, which is fairly common for clinical data. MIT has a collection of eye movement data sets: http://saliency.mit.edu/datasets.html
As for the best methods, this comment box is far too small to contain all my thoughts on that :-) It depends on your question and experiment. Sometimes, all you really need is a t-test (or the randomized version), but that requires getting the experiment just right. Other times, you might need a morass of signal processing and dimensionality reduction, fed into some giant multi-level Bayesian model in a vain attempt controls for all the stuff you neglected when you designed the damn experiment. Happy to send you some pointers if you have something specific in mind though!
Physics has historically had a huge leg up on the other sciences because they had real models that made testable, quantitative predictions. We're finally starting to learn enough about the brain that we can do this for neural data too, and I'm really excited about that!
I will definitely look through, and after digging around a bit I'd love to get some pointers. I have been curious about this for a long time, but uncertain where to look, so this is very exciting. Thank you again :)
Mostly Matlab, Python, and R, with a few things that have tight time/memory requirements in C++. Matlab was really popular in neuroscience for a very long time, so we still have a lot of code in m-files, but most labs are moving towards Python (and a few towards R).
The code quality varies a lot. Some of our "core" stuff is great, but there's also a lot of stuff that was written quickly and meant to be run once ("let's just try it"), which is cold comfort when you find it years later.
People also come in with varying levels of programming skill. I'm going to try to do actual code reviews with the undergrads this summer to see if we can't make our stuff a little less embarrassing.
I'll add, I work in a lab and have seen the stuff of nightmares myself.
As I said before, we know a lot of facts. We know a lot about the spectral sensitivity of rods and cones, and the molecular mechanism that lets them turn photons into electrical impulses. We know a little bit about where the areas that process faces are and what visual features the neurons in them respond to. We’ve got pieces, but they’re not put together.
I would say that we understand vision when we can answer a question like “How do you find a friend in a crowd?”
You can start with “When you first met, light bounced off her face and isomerized some retinal from its 11-cis to all-trans form, which caused the bound opsin to change conformation into metarhodopsin II, which activated transducin, which....” Eventually, this cascade caused electrical activity that reaches cortex. A huge set of cortical areas process visual input, and these electrochemical signals flow through all of them. We can predict V1 neurons’ activity reasonably well, less so for the downstream neurons in V2, V4, or the temporal lobe areas. We have only the fuzziest ideas how those patterns are read out, tagged as important to remember, and moved into memory. You've only just met--and yet it gets worse.
To find her, you’ve got to retrieve those patterns from memory (no one knows how, but oscillations might be involved?), and use them to search in a way that’s robust against variations in the friend’s pose, position, rotation, illumination, and even dress style or age, many of which you have never seen before and will never see again. We know, for example, that some cells in IT are fairly robust against some moderate kinds of image changes. Some but not all, of this is done by circuits that look like a convNet. Whether this is a coincidence or not is debatable and how this convNet is trained is a total mystery—-it’s definitely determined by experience, but the feedback signals needed for vanilla backprop are missing.
As you scan the crowd, you’re only getting high-resolution data from a very small part of the visual field. This is (somehow) stitched together into a unified percept. You apply various heuristics—maybe your friend favors bright colors—to speed the search along. How you learn these, and how they’re mixed in with the input from your eyes is unknown, but it’s certainly reflected in your behavioural output: you'll find her faster if you successfully predict what she looks like, and you'll be much slower if you guess wrong. Perhaps you hear a familiar voice or smell the perfume you bought her. This, too, can help you find her, but how information is integrated across senses is unknown too.
Eventually, you find her. You plan a path across the plaza towards a cafe. We have a pretty good understanding of how this works in rats (3-7 Hz oscillations coordinate place cells and grid cells in the hippocampus). Those oscillations are really strong in rodents, but much weaker in monkeys and totally missing in bats, so it's not clear how this works in humans.
Now all you’ve got to do is open your mouth and order coffee....