News for hackers. Stuff that matters.
Re:Machine Learning Course by Tom Mitchell by shostack on Friday December 19, 2014 @01:43AM (#8771339)
Can anyone recommend a good intro in machine learning that does teaches the building blocks of the math side of things? It is fascinating as a topic, but there is such a large prerequisite learning curve that it seems out of reach for those not as strong in math.

That might just be the reality of it, but hoping there might be a better introduction (even something super simple like a Codecademy equivalent).

Re:Machine Learning Course by Tom Mitchell by ilzmastr on Friday December 19, 2014 @07:45AM (#8771978)
I was in a similar situation a year ago. Two things fixed my problems:

I took a course on Linear Algebra (Bretscher's book up to chap 9) and a Probability course (Ross' book up to chap 6) and did very many problems by hand on paper. I just finished a ML course (Bishop's book, and Jordan's book), mixed grad+undergrad, which was 80% problems w/ pen and paper and 20% code up something algorithmically trivial but mathematically challenging, and don't think I would've been able to pick up the additional math along the way without these two great books and their many exercise problems behind me.

I read layman's explanations of ML concepts a year ago, and got nowhere in terms of my own implementation+debugging/improvement upon established techniques. Now I can solve problems I saw a year ago and thought "no one can do this."

My advice is to take the gateway drugs first, Probability (Ross <- I love this book!) and Lin Alg (I like Bretscher much better than Strang, but not everyone agrees with me :) Take a course in real life (for a grade and a transcript) at a competitive university if possible, nothing makes you study thoroughly like a gun to your head.

Re:Machine Learning Course by Tom Mitchell by avinassh on Friday December 19, 2014 @11:06AM (#8772787)
I would appreciate if you can give links to those books on Amazon or some other site. Thanks!
Re:Machine Learning Course by Tom Mitchell by craigching on Friday December 19, 2014 @11:12AM (#8772810)
I was interested so I looked them up:

A First Course in Probability: http://www.amazon.com/First-Course-Probability-9th/dp/032179...

Linear Algebra with Applications:

http://www.amazon.com/Linear-Algebra-Applications-Otto-Brets...

Note that the Bretscher book has really terrible reviews, but I can't evaluate if the reviews are correct or not.

Re:Machine Learning Course by Tom Mitchell by Plough_Jogger on Friday December 19, 2014 @11:37AM (#8772959)
A First Course in Probability: http://www.ebay.com/itm/A-First-Course-in-Probability-EDN-9-...

Linear Algebra with Applications: http://www.ebay.com/itm/Linear-Algebra-with-Applications-5ed...

International editions of textbooks are immensely cheaper.

Re:Machine Learning Course by Tom Mitchell by feefie on Friday December 19, 2014 @03:42PM (#8774392)
Besides the price is there anything different between the US and International editions? There must be a trick here... ?
Re:Machine Learning Course by Tom Mitchell by arghnoname on Friday December 19, 2014 @05:06PM (#8774807)
Sometimes the paper can be thinner. Sometimes it's in black and white and the US edition is in color. The international edition is almost always softcover and the cover may be in Chinese (for instance). The problem sets may be in differing order.

Many of these things are described in the comments. I almost exclusively buy international textbooks for home reference if available. The price difference and the relatively small quality difference makes it a no-brainer. If you are doing it for a class though, find a friend with the overpriced version for homework.

Re:Machine Learning Course by Tom Mitchell by jliechti1 on Friday December 19, 2014 @01:51AM (#8771355)
My guess is that you won't find any course that explains all the prerequisite math. It's probably more useful to build a solid foundation in probability theory (and therefore calculus) before going on.

For machine learning, a good place to start is Andrew Ng's course on Coursera:

https://www.coursera.org/course/ml

It's pretty light on math, while at the same time giving you experience in implementing and understanding these techniques.

From there, I might recommend Learning from Data and the associated video lectures:

https://work.caltech.edu/telecourse.html

It is a bit of a jump, but it is a great course in presenting the field of machine learning and explaining the mathematical and statistical underpinnings in a systematic way.

Re:Machine Learning Course by Tom Mitchell by jamra on Friday December 19, 2014 @02:54AM (#8771431)
I just finished the Coursera course by Andrew Ng. It was great. The only hand waving done with math was when calculus was necessary. You can take some extra time to do that work yourself if you like, but you will not be missing the underpinnings of why things work statistically. The introduction to neural networks what finally gave me that aha moment.

It is a very self contained course that is quite easy to follow. You can skip the programming exercises if you don't have the time.

Re:Machine Learning Course by Tom Mitchell by crazypyro on Friday December 19, 2014 @09:31AM (#8772321)
For anyone interested in more about the specific math of neural networks, http://www.iro.umontreal.ca/~bengioy/dlbook has a couple good introductory chapters that give overviews of most of the necessary topics for NNs, but also provides additional resource suggestions if you need more in-depth info on a certain subject.
Re:Machine Learning Course by Tom Mitchell by drcomputer on Friday December 19, 2014 @05:27AM (#8771704)
There is a large mathematical foundation behind machine learning, that is not even taught to most computer science students. The concepts that built machine learning are often found in engineering and mathematics:

It's not easy to learn, especially if you are not strong in math, but if you want an intuitive understanding as to how machine learning works, I would recommend learning a combination of probability, linear algebra, and formal theories of computation (abstract machines):

This book was really what opened the gate of tying the content together:

http://www.amazon.com/First-Course-Probability-9th-Edition/d...

You should also be strong in discrete mathematics.

Re:Machine Learning Course by Tom Mitchell by coldnebo on Friday December 19, 2014 @05:10AM (#8771665)
It doesn't cover machine learning in a lot of detail (there is one dogs and cats example) but J. Nathan Kutz book is an excellent introduction to the math behind this and other data modeling techniques with lots of hands on examples. The best thing about this book is that it is a broad survey of the maths you need in Data Science -- you get a feel for how various subjects fit together from a high level with practical examples, then you can branch out to other sources to learn more about specific methods.

* http://www.amazon.com/Data-Driven-Modeling-Scientific-Comput...

Re:Machine Learning Course by Tom Mitchell by denzil_correa on Friday December 19, 2014 @05:16AM (#8771676)
I have found Andrew Ng's math handouts for CS229 Stanford (not Coursera) the best. There's necessary introduction and abstraction of irrelevant details to the topic at hand.

http://cs229.stanford.edu

Re:Machine Learning Course by Tom Mitchell by sonabinu on Friday December 19, 2014 @03:16PM (#8774249)
This is a book from MIT press not quiet yet finished but getting there - might be a good source for material for getting the linear algebra and the probability pieces! Core material is Deep Learning http://www.iro.umontreal.ca/~bengioy/dlbook/
Re:Machine Learning Course by Tom Mitchell by twelfthnight on Friday December 19, 2014 @10:42AM (#8772666)
I think this has to do with learning styles, but I've found that working on real problems (like those on Kaggle) is a better way to learn machine learning than reading text books. When I'm working on problems, it becomes evident what I don't know. Then I'm able to intelligently go through the books and learn the relevant bits. When I start with the math, I tend not to remember anything because I have no foundation to attach the math knowledge to.
Re:Machine Learning Course by Tom Mitchell by atrilla on Friday December 19, 2014 @05:46AM (#8771736)
A very interesting requirement. Thanks for noting it.

I'm setting up a blog on Artificial Intelligence (which inherently includes Machine Learning) focused on the contents of the AIMA book (by Stuart Russell and Peter Norvig):

http://ai-maker.com/

How would you like it to be? Code will be developed in Matlab, so the math depth is rather convenient using this tool.

Re:Machine Learning Course by Tom Mitchell by Houshalter on Friday December 19, 2014 @06:05AM (#8771778)
http://www.metacademy.org/ is a great source. It tells you what all the prerequisites are for everything as well as where you can learn them and what their prerequisites are and so on.
Re:Machine Learning Course by Tom Mitchell by robert_tweed on Friday December 19, 2014 @01:15AM (#8771303)
Does anyone have the book? Having looked through the ToC on Amazon, there are a few topics that interest me and it seems to be more in-depth than these lectures. But as it is from 1997 (and doesn't appear to have been updated), I'm concerned it will be a bit out of date.
Re:Machine Learning Course by Tom Mitchell by akramhussein on Friday December 19, 2014 @09:32AM (#8772329)
I read this cover to cover for my ML course at Imperial College London in UK. While not an easy read, reviewing the same topics a few times did make you understand the fundamentals better. AbeBooks sometimes has it going for £20(~$30). The exercises were a bit tricky as often the answers weren't attainable by simply following the book and resulted in you needing to consult other material.
Re:Machine Learning Course by Tom Mitchell by avinassh on Friday December 19, 2014 @11:05AM (#8772779)
what are pre-requisites to start with the book? Requires heavy math understanding of Linear Algebra etc?
Re:Machine Learning Course by Tom Mitchell by jskonhovd on Friday December 19, 2014 @05:00AM (#8771645)
I used this book in my Machine Learning course last spring at Georgia Tech. I wouldn't consider it out of date. It is missing a few topics like SVMs that we covered, but otherwise it's a good introduction.
Re:Machine Learning Course by Tom Mitchell by mjn on Friday December 19, 2014 @01:03PM (#8773450)
It's still a good introduction to the principles: how problems like regression, classification, and reinforcement learning are defined; concepts like overfitting, bias-variance tradeoff, etc.; some general classes of algorithms and how to analyze them.

The age mainly affects its usefulness as an off-the-shelf guide to applied ML, because some of the currently best performing general-purpose algorithms aren't mentioned [1]. It also spends quite a bit of time on algorithms now considered mainly of historical interest, like version spaces.

So imo its main current usefulness is as a foundational text, which it's quite good for. It helps that it's also well written and understandable.

[1] A recent empirical analysis found that random forests and support vector machines seem to perform most consistently well at classification tasks, neither of which are in this book. http://jmlr.org/papers/v15/delgado14a.html

Re:Machine Learning Course by Tom Mitchell by midko on Friday December 19, 2014 @04:27PM (#8774633)
Having gone thoroughly through the book, this is quite an accurate description.
Re:Machine Learning Course by Tom Mitchell by nicklaf on Friday December 19, 2014 @12:48AM (#8771249)
Does anybody have URLs for the videos? Clicking on the video link for a lecture asks you to install Silverlight.
Re:Machine Learning Course by Tom Mitchell by robert_tweed on Friday December 19, 2014 @01:37AM (#8771331)
As far as I can tell there are lots of tiny fragments each about 1-2 seconds long rather than one video file.

It's actually not just a video, but a specially designed player that shows the lecture video and the slides in sync, along with a bunch of bookmarks so you can jump to specific slides.

Try it in Chrome. At least on the Mac, there's a dedicated but cut-down version, which does not require SilverLight. It's based on FlowPlayer and it also seems to work with Flash disabled.

Re:Machine Learning Course by Tom Mitchell by warrick on Friday December 19, 2014 @02:41AM (#8771410)
Re:Machine Learning Course by Tom Mitchell by phlyingpenguin on Friday December 19, 2014 @10:16AM (#8772519)
Anybody got a mirror of lecture #6? It seems to be dead on the server.
Re:Machine Learning Course by Tom Mitchell by shamskazi on Friday December 19, 2014 @03:07PM (#8774205)
Thanks!

Lecture 6 seems to be offline, unfortunately.

Re:Machine Learning Course by Tom Mitchell by nicklaf on Friday December 19, 2014 @04:06AM (#8771545)
Thank you!
Re:Machine Learning Course by Tom Mitchell by avinassh on Friday December 19, 2014 @01:00AM (#8771276)
I am checking on my iPad, though I don't get prompts to install silver light, I get redirected to some page, it says some error with session and to contact support :-/
Re:Machine Learning Course by Tom Mitchell by rasz_pl on Friday December 19, 2014 @02:03AM (#8771368)
same on opera 15, but after a second video shows up just fine, silverlight (ahahaha lol) is used for something else (slides?) under the video

Video is displayed using Flash Flowplayer, you can extract mp4 from the source

for example first one is http://scs.hosted.panopto.com/Panopto/Podcast/Embed/257476ca...

Re:Machine Learning Course by Tom Mitchell by dalanmiller on Friday December 19, 2014 @01:24AM (#8771324)
I am registered for this course and I am quite excited!
Re:Machine Learning Course by Tom Mitchell by nitishmd on Friday December 19, 2014 @11:13AM (#8772815)
Good to see cmu opening up to open learning!
Re:Machine Learning Course by Tom Mitchell by aaronaaa on Friday December 19, 2014 @02:51PM (#8774126)
As a current student, unfortunately, this is a rare gem. CMU still fails to make any attempts at free courseware.
Re:Machine Learning Course by Tom Mitchell by aroman on Friday December 19, 2014 @03:19PM (#8774263)
As another current student... you are very mistaken.

See: http://oli.cmu.edu

Re:Machine Learning Course by Tom Mitchell by aaronaaa on Wednesday December 31, 2014 @07:30PM (#8821728)
However, with the exception of one or two of those courses, those are nothing but introductory courses. And while this is great material to have access to, grants almost no access to the wealth of knowledge at CMU.
Re:Machine Learning Course by Tom Mitchell by jokoon on Friday December 19, 2014 @02:03AM (#8771369)
I have a "pre-existing working knowledge of probability, linear algebra, statistics and algorithms", but can't he be more precise ? I mean to what difficulty are those math used ?
Re:Machine Learning Course by Tom Mitchell by anhng on Friday December 19, 2014 @03:37AM (#8771500)
This is a Phd-level course at CMU. Pretty heavy on math. You can just take a look at the material to know the difficulty. If you are not sure, you probably should try easier ones such as Andrew Ng's on Coursera.
Re:Machine Learning Course by Tom Mitchell by jskonhovd on Friday December 19, 2014 @05:06AM (#8771659)
I am assuming he means at a undergraduate level. If you need a probability and statistics book to refresh, I can recommend "All of Statistics" by Wasserman.
Re:Machine Learning Course by Tom Mitchell by jacobolus on Friday December 19, 2014 @02:24AM (#8771386)
Presumably this means at the level of an introductory undergraduate-level course in each of those subjects (the kind of courses targeted at first or second year science/engineering undergraduates). You could try working through the material, and if anything is beyond you, it shouldn’t be not too hard to find the appropriate textbook and catch up.