Thank you for creating these courses and putting them online! I started part 1 a few weeks ago. I couldn't agree more with your teaching style. I'm someone who started college as a computer science major and couldn't finish, because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff.

I finished my degree with economics, went to work in finance, and worked my way to being a developer by first doing advanced Excel, then Access, then database servers, then web development, then native development, and now more math heavy research and development. As the need arises, I keep going further down the stack and gaining more understanding. It's much more natural (for me) to learn that way.

> because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff.

But you do need to know abstract math to build stuff, particularly if what you want to build is deep learning stuff, both models and implementations.

For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it.

> For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it.

The course teaches all those things - as the comment you're replying to states, you go deeper and deeper during the course to understand all the details.

There's been a lot of research into teaching strategies that shows that this is often a more effective approach for many people than the bottom up approach widely used in math and CS. It doesn't mean that you learn any less of the foundations - just that it's in a different order.

> There's been a lot of research into teaching strategies that shows that this is often a more effective approach for many people than the bottom up approach widely used in math and CS.

I seriously doubt that anyone can effectively learn linear algebra, multivariate calculus, optimization and regression models from an onlone tutorial on deep learning. These are subjects whose basics alone take multiple semester-long courses. If a bottom-down approach was remotely effective, no one would bother teaching the basics.

Do you think it's necessary to have a rigorous understanding of all of those topics before creating a machine learning model? And that you don't learn from interacting with it, even if you don't fully understand how it works? For machine learning in particular I think that's pretty ironic.

> If a bottom-down approach was remotely effective, no one would bother teaching the basics.

This is a purist approach.

Sometimes, you sacrifice the details to broaden the audience. This has the result of getting more people interested.

Hey! Thanks for these great courses and materials! How much additional math (beyond high school and introductory college courses) do these courses teach? For example, if I were to take both courses, would I be able to understand the papers published by Surya Ganguli (e.g., The Emergence of Spectral Universality in Deep Networks, Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net)?

Ganguli's papers are at the more "math-y" end of the spectrum when it comes to DL papers. So don't worry if you're finding them a bit unapproachable.

In part 2 of the course I provide quite a bit of advice about how to approach papers in general, and you'll get plenty of practice in reading and implementing papers - but we don't cover the specific math in this particular papers.

My view is the best approach to the math in papers is to generally learn what you need as you get there. It's nearly impossible to know all the math that covers every paper you'll come across, but if you learn the meta-skill of how to learn it on demand, then you'll be just fine! :)

One year of Python, with high school and introductory college courses. Some basic linear algebra, familiarity with logarithms etc.

I think you misread my question. I'm asking what math students will learn, not what math students should already know.

Gram matrix, some non-linear optimization. Deep Learning is simple, complexity comes from loss function, trying to adjust weights of under-represented categories, different LEGO blocks in building your network and seeing if the particular non-linear optimization works in your case or not. You can go super deep with state-of-art math research in reading about "why do we think Deep Learning works, when it shouldn't", which is mentioned by Jeremy.

Hey Jeremy, thanks for making these courses! I was curious if you could offer any points of comparison between the fast.ai courses and the Udacity ML nanodegree course (or any other MOOCs you have opinions about).

I am not Jeremy but I have taken part in the current course and also took some lessons by Siraj (who also does lessons at udacity), Andrew NG, and Karpathy. I think all of them are awesome teachers. Like Siraj's lessons, the fastai course is full of practical examples, whereas Ng/Karpathy's courses are more theoretical. What makes the fast.ai course unique compared to Siraj (who covers a lot of topics very briefly) is that Jeremy delves into depth, taking a long time explaining things, so you have a lot of opportunities to understand what's going on. In the 2018 course there is actually not that much different content compared to the 2017 course, rather Jeremy took the time and tried to make everything easier for everyone to understand and built a better python library. Also another unique point about Jeremy is that he and Rachel want not only to teach people but rather inspire a community of people working together in teams. The fast.ai library will not be just a project for the course, it will be something that a lot of people will use and contribute to to make pytorch better.

Is Rachel the lady that always keeps interrupting Jeremy so that he goes out of flow all the time, making lectures less enjoyable?

Dr Rachel Thomas, the co-founder of fast.ai, is the person who takes the highest voted questions from our 600+ in-person students and passes them on to me, as I asked her to do. At least 4 different people must have wanted the question asked before she asks it.

This approach helps ensure that if I haven't explained something clearly enough that I get another chance to do so, and, more importantly, keeps me fresh and energized throughout each lesson (I get stale and boring without some interaction).

I am a firm believer in candid feedback; I talked to multiple people taking fast.ai and all of them took it as disturbing the flow of lectures. I really like fast.ai, it's an excellent hands-on course, so please don't get offended.

Since you appreciate candid feedback, I would like to explain why I down-voted you. Your post comes off as super rude and dismissive. You may have some valid criticism to give but, take more care in how you deliver it. It's a bad look for HN.

I appreciate your feedback! :) It's difficult to convey mental state over a single line of text, giving rise to various interpretations that weren't in the intent. Imagine I am your best friend and I say the same in a playful fashion - would that be more acceptable?

Absolutely! That's actually in the hacker news commenting guidelines: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith." https://news.ycombinator.com/newsguidelines.html

So to answer you question, yes, it would be more acceptable if you were saying that to me in person. In fact, I agree with you about the flow of the lectures, and I was looking for someone to bring that up and I'm glad that it's getting discussed here.

However, Jeremy and Rachel are both reading these comments and we should strive to provide thoughtful, fleshed out feedback. The work they've done on Fast.ai is a tremendous lift, and deserves more than a drive-by comment.

Personally, I find the questions helpful, and for everyone who's watching on video, you can skip whatever portion you like, can't you?

The thing is you can visibly see how it interrupts Jeremy; he is in the middle of explaining something, then has the face of a surprised person, loses context for a few seconds etc. Better IMO would be if he just finished a small section in its entirety then did Q&A, instead of allowing himself being interrupted all the time. And often those questions are missing the point, which is to be expected with newbies, so the lecture loses efficiency. If I didn't know most of the basics already, I'd have problem following them. I found just doing the exercises better for learning.

I also find the questions very helpful. I understand for many people who already have a good understanding, these questions might be interrupting the flow. Many a times the questions his students raised were things that I had not thought of, sometimes they even stumped Jeremy. Fortunately Jeremy in most cases gives answers which vary in length, according to whether it's a concept which he will clear later, or whether it is an important concept that needs to clarified then itself.

So I think this more effective way of learning/teaching even though there is a loss in efficiency.

Rachel's mic is always off until I turn it on. So if she's asking a question, it's only after she's visually indicated that she wishes to do so, and I've found a time in my presentation that I'm ready to take it. So I'm literally never being interrupted, and can't be surprised by the fact that she's asking a question (although I may well be surprised by the content).

I do try to limit the time I wait to take a question, since I don't want to move on with a topic where I've failed to properly explain some foundational piece.

I like the questions. Some of the doubts I also have. They improve a lot the course.

Same, there have been several times I have had a question and someone thankfully asks the exact one I had. And Jeremy does a great job explaining it. It really helps to get at some of the why's behind the magic.

You are missing the point, that is full part of their experience opening to unrepresented classes and styles. It is human together with efficient, the heart added to the mind.

I haven't taken the course - and I'm hopelessly biased anyway so you shouldn't trust my comments. However a lot of our students have done it. Here's a couple of their comments (search the linked thread if you want more):

- http://forums.fast.ai/t/how-has-your-journey-been-so-far-lea...

- http://forums.fast.ai/t/how-has-your-journey-been-so-far-lea...

Some folks have taken the Udacity flying car and self-driving car courses as well as fast.ai, and had success building on that combination, e.g. http://forums.fast.ai/t/meet-greet-thread-introduce-yourself...

fast.ai 1 is much better than Udacity's ML if you want to do purely Deep Learning. Udacity's DLF ND is IMO better than fast.ai 1. However, fast.ai 2 is way better than Udacity's DLF ND, and only their self-driving car has some Deep Learning models that are even more awesome (imagine driving using Deep Learning only). Take both fast.ai 1 and DLF ND if you are a beginner; after that jump on Stanford's CV & NLP courses; Oxford NLP might be a good idea as well. Then do UCL's Reinforcement Learning and you are set.

Thanks Jeremy and Rachel for the excellent materials. I am a beginner with ML and initially tried to start with fast.ai part-1. I struggled initially as I preferred to know some basics. So, I looked for a beginner friendly course and ended up with Udacity's AI programming with Python. It gives a pretty decent overview on the basics. Now, I am very excited to take up the fast.ai part-1 & part-2 courses.

I didn't take the Udacity ML Nanodegree, though I did do the Deep Learning Nanodegree.

I found it to be disappointing especially since they had hyped the collaboration with Siraj, which was nothing more than linking to certain YouTube videos.

The project feedback was sometimes helpful. I felt like most of the time though, the feedback was "you did this wrong, read this article" instead of something more personal like an elaborate explanation on why you should do things a certain way. I even once explained why I initialized a model a certain way and the reviewer ignored it when critiquing my model, which almost felt like "all students have to do it this way."

It wasn't all bad. My favorite parts were learning about GAN's with videos and a notebook from Goodfellow. And when I was trying to build more intuition about CNN's, the videos with Vincent Vanhoucke were helpful.

But altogether I felt a little disappointed in the actual projects. Maybe it was because I felt like the math was glossed over and it was too many topics with shallow exploration for a single course. I actually wished that Udacity offered a single course for say, CNN's and GAN's, going very deep into the math and processes behind them.

I'm taking another Udacity course taught by Thrun (this time, it's free) and again, he kind of glosses over why certain mathematical operations are done, at which point I spent a lot of time watching lectures by other professors who spent more time explaining it.

I think that's my biggest criticism about MOOC's in general, they can be very hand-wavy about very important concepts that underly a process. I've spent a great deal of time reading papers and course material from other colleges, writing throw away code, and watching videos from other profs in order to shore up an intuition that was simply not strongly built by the MOOC.

> Udacity offered a single course for say, CNN's and GAN's, going very deep into the math and processes behind them.

I think that's what are they going to do next with their AI school. They announced separate nanodegrees for CV, NLP and RL. However, I hope they aren't going to be such massive disappointments as AI ND term 2, where they basically didn't deliver what they promised, castrated projects and one could finish each of term 2 specializations in one weekend (i.e. removing image captioning project, removing real-world NLP project, etc.). A similar story happened with Robotics term 2, where instead of a real robot they promised you received a standard discount for NVidia TX2, and reinforcement learning was butchered from robot walking to robotic arm movement. So I am doubtful they can really live up to their promises with their current staff that keeps underdelivering. The only ND they made absolutely breathtaking, worth every penny, was Self-driving Car ND (IMO the best [not only online] course I've ever taken, and I took many from top 10 universities).

Hi Jeremy.

Just wanted to ask a quick question if you recommend any prerequisites before taking this course? I am a second year Computer Science student looking to get into AI, and I was just wondering if this would be a good starting point for me, or should I take some other courses and learn something else before?

Thank you.

This linked course is part 2, so this isn't a good starting point. But part 1 is! :) Here's the link: http://course.fast.ai .

We assume you're familiar with python and somewhat familiar with numpy. If you're not sure if you're ready, you can always just start the first lesson and see how it goes. There are plenty of resources on the forum to help fill in specific skill gaps as they come up.

Thanks Jeremy (and Rachel) for everything y'all have done; I learned a lot just from watching part 1!

Somewhat tangential question, but how do you feel about sticking with PyTorch for future lessons/research work? What are your thoughts on the new Swift integration with TensorFlow?

We're so early in the software and tools cycle for DL that it's unlikely anything that exists today will remain relevant in its current form for very long. You can see that every couple of years the entire DL ecosystem is getting replaced - I don't think that will change.

I'm glad you asked about Swift. It's one of the two directions I'm most excited about for the next stage of DL software (the other being javascript!) Swift is a much better language than Python, and having autograd and tensor types built in will be quite transformative. But it has nothing like the same data science library and tooling, so it'll take time to be a real option for most people.

Javascript has the potential to make DL accessible even for those without a discrete GPU, and allows people to get started even without installing a tool-chain. It also allows more work to be pushed to the client - possibly even on mobile devices.

Here's a vote for Julia over Swift.

Hi Jeremy - For someone who doesn't like to watch lectures, will you guys ever be releasing a textbook?

I might be helping author a textbook based on our upcoming introduction to machine learning MOOC. Which maybe one day would lead to a deep learning book - but that would be years away!

(BTW, I used to think I didn't like videos for learning, but actually I now think some material works best in this format. E.g. in this case we're walking through interactive notebooks, and you can follow along too. Some of the material is animated, and often I'm building up drawings as I talk. Maybe it won't be for you, but it might be worth giving it a go to see...)

Just another thought on this, as someone slowly trying to work his way through part 1 (and generally very happy with it, thanks for this amazing resource!): videos are great when they're really short. It's tough for working people to find enough unbroken time to watch an hour or two long video, like the ones in fast.ai (as amazing as they are), especially with frequent pauses for note-taking and such.

But it's much easier to handle smaller chunks. This is probably my only critique of/gripe about fast.ai---the videos pack a lot of topics in, and they could be broken at topic boundaries to make them easier to use.

Thanks for the feedback! One thing you may be interested in, if you haven't seen it, is that you can right-click the video and copy a link to the current time code, so you can always go back to where you were (or just pause the video in a tab and return later).

Personally, I really hate the Coursera approach of lots of separate short videos - I totally get that some people like it, and that's fine, but it's not something I want to do myself.

Oh, I did not know you could link to specific time slices! That's a big help, thanks!

another thing I used a ton when going through part 1 was being signed into my google account, google kept track of my progress through the video and if I exited out and came back on a different machine it would resume where I last left off, then I would back up a few minutes to build that context back up (or attempt to anyways).

I'm traditionally a textbook person too, but I really like fast.ai. You get to see how to tinker with things in real time (which is often clunky in a textbook), and there's a lot of informal folklore that people don't normally write in academic textbooks. If you're dead set against video you could just work through the notebooks yourself and read the referenced papers on the website.

I have the same problem. Too traumatized (when I was young) by a person teaching in a lecture format. I like self discovery and tinkering.

Jeremy and Rachel are very active on Twitter. I learn a lot from their tweets too! Thanks!!