I want to second MacKay's book. I had a terrible statistics class in college. We spent the entire time looking up tables of p-values and t tests, without a very convincing explanation as to why. The entire topic was damaged for me from then on until I read MacKay online, and then bought the paper version.
His book starts from first principles -- simple ideas about probabilities -- and it builds a foundation for understanding Bayesian methods. And he explains, basically, why the p-values and t-test stuff is a bunch of crap, which was an immense relief.
So I don't need to flinch over the mention of "p hat" again?
Thanks for the resource, this looks fantastic.
http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
Videolectures has some very good videos as well. Zoubin Gharamani has a pretty solid lecture on Bayesian learning at http://videolectures.net/mlss05us_ghahramani_bl/ (he's a great researcher but not the most engaging speaker). Try Christopher Bishop's lecture at http://videolectures.net/mlss09uk_bishop_ibi/ as well, it might be slightly more palatable.