I think PGM's are covered by a lot of "standard" ML texts -- someone else mentioned Murphy's book which is great and is humongous but is a good reference for pretty much every method under the sun.

Bishop's Pattern Recognition and Machine Learning has a chapter thats free online: https://www.microsoft.com/en-us/research/wp-content/uploads/...

I'd just like to add that the entire PRML book is now free online, not just the sample: https://www.microsoft.com/en-us/research/uploads/prod/2006/0...

Introduction to Statistical Learning

https://faculty.marshall.usc.edu/gareth-james/ISL/

Elements of Statistical Learning

https://web.stanford.edu/~hastie/ElemStatLearn/

Machine Learning: A Probabilistic Perspective

https://mitpress.mit.edu/books/machine-learning-1

"Machine Learning: a Probabilistic Perspective" is more an encyclopedia of algorithms I would say, and it has lots of typos. I personally would not recommend it (except for the amount of algorithms that it covers, many of which are usually not found in other books).

Thanks for early warning. Will have to keep that in mind.

Are those really the best starts for "Bayesian statistics"?

Especially the first 2 are rather the standard "intro to ML textbooks", with a frequentist focus (ISL may even have zero Bayesian stuff - Naive Bayes is not "Bayesian" – while ESL still has maybe 10% bayesian content if that).

Instead, I would suggest the following for learning Bayesian methods, especially given the HN crowd: https://github.com/CamDavidsonPilon/Probabilistic-Programmin...

You make a good point. It's been a while since I flipped through them, they just come up in lots of discussions on this topic. I agree that the series you link to is really great for PPL and Bayesian methods. You may find that the library upon which it's based (PyMC3) is built on top of Theano, which has been abandoned and deprecated. PyMC4 is around the corner and uses TensorFlow Probability. Early, informal reports say it's 10x faster.

Thanks a ton for these. Added this to things I know that I don't know list. ;)

I took a course on Applied Bayesian Statistics taught by David Draper in grad school and we covered Bayesian Data Analysis (Gelman et Al.) http://www.stat.columbia.edu/~gelman/book/ and Probability Theory and tbe Logic of Science by Ed Jaynes: https://www.amazon.com/dp/0521592712/ref=cm_sw_r_em_apa_i_v3...

The former is a much recommended book since it's very comprehensive and builds everything from the ground up and was the basis for the entire course. The latter is a beast of it's own and we simply covered what was effectively the first chapter as part of the course.

For Bayesian stats, "Statistical Rethinking" by McElreath is a masterpiece.

This should be the top comment, I'm reading his newly released 2nd edition and it is outstanding. Other mentions are -

  - Doing Bayesian Data Analysis (dog book)
  - Student's Guide to Bayesian Statistics

Slightly more advanced

  - Bayesian Data Analysis 3 (currently free! http://www.stat.columbia.edu/~gelman/book/)

Thanks, everyone, for your kind suggestions. Much appreciated.

+1