I've wanted to do a Ph.D. for some time, but neither pure CS nor pure Maths (I have a masters in the latter) excites me anymore. Neuroscience interests me, but I want to make sure that I'm not deluding myself; therefore I'd like to work through a proper introductory text before making the next move.
EDIT: Listened to my inner grammar Nazi and fixed capitalization & sentences.
For neuroscience first-year grad students, the gold standard is:
http://www.amazon.com/Principles-Neural-Science-Eric-Kandel/...
The fourth edition does a good job at higher-level stuff. But be forewarned: It is biology intensive.
For less biology and more cognitive neuroscience, you can try:
http://www.amazon.com/Cognitive-Neuroscience-Second-Michael-...
or
http://www.amazon.com/Cognitive-Neuroscience-Neuropsychology...
I used the Gazzaniga when I taught Intro to Cognitive Neuroscience even as I wasn't super impressed (which is a common complaint - tolerance, not true acceptance). I know some folks like the Banich book instead.
If, with your Math background, you're interested in neural computation and modeling there are also some good books along those lines. Just let me know and I'll hunt down the links.
I've already started teaching myself some first year biology (from a borrowed textbook), so I'm going for Principles.
Kandel is the gold standard, but it is quite hefty and technical -- a great slimmer one is "From Neuron to Brain" (advanced undergrad/early grad): http://www.amazon.com/Neuron-Brain-Cellular-Molecular-Approa...
If you're more interested in the computational end, try "Theoretical Neuroscience": http://www.amazon.com/Theoretical-Neuroscience-Computational...
And I'll also add in a great computational textbook on spike recording: http://www.amazon.com/Spikes-Exploring-Neural-Computational-...
I'll also note that if you apply to a "Neuroscience" grad program, you're probably going to be doing biology experiments all day, or psychology experiments if you pursue cognitive neuroscience.
If you want to use CS and Math, the best you can do in a neuroscience program is data analysis. Hence, I would recommend AI or Mathematical Psychology (though the latter is about modeling high-level events).
I know neuroscientists that do few biology experiments, if at all. There are other ways to get data if you don't collect it yourself. I also know folks in Psychology programs that don't do psychology experiments, based on the same principle. For instance, different types of modeling allow for contributions that coincide with data but as theoretical and analytic tools.
I wouldn't recommend Mathematical Psychology (too much theory, not enough application). I don't know enough about AI programs to comment.
The fact is that there are very few professors working on "Computational Neuroscience," which is basically modeling experimental results. (I consider this data analysis because most of your time is spent analyzing experimental data. But, its intention is to not only explain given data but to predict the results of future experiments.)
Hence, if you apply to a neuroscience graduate program, there will usually only be one or two faculty members who do computational work. (There are some exceptions -- e.g. UCSD, Caltech.) Many programs won't even take you if you're only interested in computational work because if you somehow don't get along with the professor or it doesn't work out, you won't have a backup lab.
AI programs are divorced from experiments, but they get at the "bigger issues" and are satisfying for CS people -- whereas computational neuroscience models very complex small systems. Unfortunately, computational neuroscience also seems to be struggling -- many biologists discredit it, saying that its only real breakthrough has been Hodgkin and Huxley's model of the action potential!
Here at NYU, we've got at least 5 computational neuroscience people within the courant institute (math+CS depts), and there are several more in the actual neuroscience department. Plus we have a computational biology Ph.D. program within cims.
I know Rutgers has the BioMAPS computational biology program, and I think princeton has something similar.
In neurobiology (the original "neuroscience"), the opportunities may be more limited. I was moreso discussing this latter definition in my posts, mainly because this is what many neurobiology-centric academic departments are called.
Your purposes might also help. There's been good work done on Bayes and datamining and I think some has made it into a book format.