imakecomments
Take an actual graduate course in pure mathematics, very little hand holding.
>there’s basically no time or space for one to explore one’s own thoughts and program unless it specifically matches up with what someone else (namely an advisor) wants to do.
Not necessarily true. Find an adviser that will support YOUR interest (they exist). Sounds like you have some biases against PhD programs without actual experience of being in a good one.
OP, in a university setting it is easier to find mentors. Look for professors, grad students and in-class TAs if possible.
I don't think that rings true to those with PhD degrees doing academic research in CS.
You missed the point. Poor kids in general don't go to Math & Science high schools, or high schools that offer college level courses in Calculus I-III, Number Theory, Abstract Algebra, Differential Equations, Linear Algebra, 1-2 semesters worth of programming, Data Structures, etc. These kids don't have have access to private tutors to ensure they ace their SATs, SAT Subjects, ACTs etc. They aren't enrolling in multiple ECs and they don't have teachers with PhDs writing them letters of recommendation. These kids don't have parents that went to colleges and don't have the resources handed to them as the kids that are getting into Brown, Harvard, Princeton, Stanford etc. If you look at the lower level high schools in America, they barely give any of their kids any advantage. They actually put them at a major disadvantage compared to the elite high schools.
It's great elite colleges have these needed based programs. But by virtue of their admissions criteria they are excluding lower income students on a vast scale.
I quote the author Geoff Sutcliffe here:
"Automated Theorem Proving (ATP) deals with the development of computer programs that show that some statement (the conjecture) is a logical consequence of a set of statements (the axioms and hypotheses). ATP systems are used in a wide variety of domains. For examples, a mathematician might prove the conjecture that groups of order two are commutative, from the axioms of group theory; a management consultant might formulate axioms that describe how organizations grow and interact, and from those axioms prove that organizational death rates decrease with age; a hardware developer might validate the design of a circuit by proving a conjecture that describes a circuit's performance, given axioms that describe the circuit itself; or a frustrated teenager might formulate the jumbled faces of a Rubik's cube as a conjecture and prove, from axioms that describe legal changes to the cube's configuration, that the cube can be rearranged to the solution state. All of these are tasks that can be performed by an ATP system, given an appropriate formulation of the problem as axioms, hypotheses, and a conjecture.
The language in which the conjecture, hypotheses, and axioms (generically known as formulae) are written is a logic, often classical 1st order logic, but possibly a non-classical logic and possibly a higher order logic. These languages allow a precise formal statement of the necessary information, which can then be manipulated by an ATP system. This formality is the underlying strength of ATP: there is no ambiguity in the statement of the problem, as is often the case when using a natural language such as English. Users have to describe the problem at hand precisely and accurately, and this process in itself can lead to a clearer understanding of the problem domain. This in turn allows the user to formulate their problem appropriately for submission to an ATP system.
The proofs produced by ATP systems describe how and why the conjecture follows from the axioms and hypotheses, in a manner that can be understood and agreed upon by everyone, even other computer programs. The proof output may not only be a convincing argument that the conjecture is a logical consequence of the axioms and hypotheses, it often also describes a process that may be implemented to solve some problem. For example, in the Rubik's cube example mentioned above, the proof would describe the sequence of moves that need to be made in order to solve the puzzle.
ATP systems are enormously powerful computer programs, capable of solving immensely difficult problems. Because of this extreme capability, their application and operation sometimes needs to be guided by an expert in the domain of application, in order to solve problems in a reasonable amount of time. Thus ATP systems, despite the name, are often used by domain experts in an interactive way. The interaction may be at a very detailed level, where the user guides the inferences made by the system, or at a much higher level where the user determines intermediate lemmas to be proved on the way to the proof of a conjecture. There is often a synergetic relationship between ATP system users and the systems themselves:
The system needs a precise description of the problem written in some logical form, the user is forced to think carefully about the problem in order to produce an appropriate formulation and hence acquires a deeper understanding of the problem, the system attempts to solve the problem, if successful the proof is a useful output, if unsuccessful the user can provide guidance, or try to prove some intermediate result, or examine the formulae to ensure that the problem is correctly described, and so the process iterates.
ATP is thus a technology very suited to situations where a clear thinking domain expert can interact with a powerful tool, to solve interesting and deep problems. Potential ATP users need not be concerned that they need to write an ATP system themselves; there are many ATP systems readily available for use. At the 1st order level some well known and successful systems are Otter, E, SPASS, Vampire, and Waldmeister. Higher order systems include ACL2, Coq, HOL, and Nqthm. For examples of problems written in classical 1st order logic, the TPTP problem library is a useful resource, which also has a simple interface for users to try out many 1st order ATP systems.
Fields where ATP has been successfully used include logic, mathematics, computer science, engineering, and social science; some outstanding successes are described below. There are potentially many more fields where ATP could be used, including biological sciences, medicine, commerce, etc. The technology is waiting for users from such fields."
Roughgarden's class is advance and expects mathematical maturity. You may find his course quite fast and rough if you are a beginner.
Sedgwick's class is much easier. He is a bit boring and tries to use "real life" examples (in some instances) from the physical sciences to make the material relatable. This in my opinion detracts from the material. Also, he doesn't always fully explain where he got some of the big ohs here and there.
My advice? Follow MIT's OCW course (it uses CLRS). Supplement it with Algorithms Unlocked, the Khan Academy link in OP and CLRS. If you use those 4 resources and put in the work you'll understand the material.
All 4 sources have Thomas C's DNA touch to it (he is the C in CLRS). So you'll find it consistent when you read from one source to the other. After reading/hearing the same thing about 4 different times in 4 different ways it'll begin to click.
Order of easiness is probably Khan Academy > Algorithms Unlocked > MIT Algorithms Course > CLRS.
Algorithms Unlocked is like "pre-CLRS" and Khan Academy's version is the TL;DR version of Algorithms Unlocked.
Hope this helps.
Below are the links,
https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...
https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press...
https://www.khanacademy.org/computing/computer-science/algor...
https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Algorithms unlocked: https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...
CLRS: https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press...
Both include the same author as the one in this article (Thomas Cormen).
-Multivariate Calculus
-Statistics & Probability (with Calculus)
-Linear Algebra
-Optimizations
as a baseline. You'll need more as you progress.
You want to be comfortable with the math review in: http://www.deeplearningbook.org/
I'd wager to say most people on HN would drop out of an actual graduate level course on ML. These courses (from my experience) are more research oriented and less about "hacking".
He almost got kicked out of graduate school. Later went on to win the Fields Medal.
I cannot stress enough how important mathematical foundations is. It'll make everything else much easier to learn. I haven't read the book but heard good things about: https://www.amazon.com/Discrete-Mathematics-Applications-Sus... as a beginner text.
Coursera has multiple offerings on Data Structures / Algorithms -- find one that works best for you.
For instance:
https://www.coursera.org/learn/introduction-to-algorithms
https://www.coursera.org/specializations/algorithms
https://www.coursera.org/learn/algorithmic-thinking-1
https://www.coursera.org/learn/algorithmic-toolbox
By doing all of those you'll get a good introductory exposure to the topics.
You should also look at a rigorous course offering of Algorithms. MIT has a few online to view.
Some readings for a beginner are:
https://www.khanacademy.org/computing/computer-science/algor...
https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...
https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press... (not beginner level but classic)
After all of this you should be fine with diving into interview books. You'll want to whiteboard solutions and be able to do all the difficult problems. Look into sites like leetcode, glassdoor and be able to do the difficult problems posted there.
The Linear Algebra presented in the book is easy, but if you aren't familiar with Linear algebra then it would be a benefit to learn some basics before reading.
Familiarity with mathematical notation and mathematical reasoning is also a must. If you don't have a basic understanding of being able to follow mathematical arguments, then some side comments (that aren't explicitly stated) may seem mysterious to you.
This book is very suitable for someone with a background in Python and basic Linear Algebra/mathematical reasoning knowledge.
The point of this book is to understand the math/algorithms and not treat the algorithms as blackbox solutions. You'll learn about processing your data, dimension reductions, etc., etc.
You may have to do some outside studying depending on your background, but the author provides those resources for you in the text. Overall this is a very good book and the author did a good job at writing it.
Seems like a few lectures are missing but if you google around you might be able to find them somewhere.
The interviews the authors give paint the picture this book is for the 'practitioner'. If Chapter 1 is meant for a brief review then don't advertise the book for a complete practitioner/beginner. Either make the book for the practitioner or not. If you do, then don't pretend to serve introductory math in it that the unfamiliarized reader will read and understand. They fail at their purpose there. So either make that chapter useful for the practitioner or leave it out and assume the mathematicians already know it. Maybe put it in an appendix and let us get to the meat quicker. It honestly does not take much time to define what a matrix is, give an example, define matrix multiplication, give examples etc. Same applies with basic definition and examples of derivatives. These are mindless mechanical procedures anyone can learn. It wouldn't take too much extra space to include some thoughtful examples. Maybe I should write an 'introductory group theory' textbook and start discussing geometric group theory 2 pages in if we want to get into not serving an intended audience's purpose.
I like what the author's are doing. I'm on their side, but I'm making suggestions that could serve a wider audience.
I am hesitant to recommend your book to a true practitioner due to the assumed knowledge presented within the math section. I think a better treatment of mathematics would assume the reader has little to no background but is intelligent enough to learn ground up the specific use cases of the mathematics for the deep learning techniques presented in the book. See: http://www.deeplearningbook.org/ for better treatment of the math review. It seems more thorough and makes less assumptions about the math background of the reader.
I would love to recommend your book to a practitioner but I'm afraid the math section (the version I reviewed) would scare them off/they would get little out of it.
In my opinion it wouldn't be too difficult or much effort to define what these mathematical objects are and show basic examples with basic computations to solidify the concepts. The notion of gradient descendent & derivates (or partial derivatives) isn't that difficult to understand and could be easily explained in a page or less.
For example when you discuss the Outer Product:
"This is known as the “tensor product” of two input vectors. We take each element of a column vector and multiply it by all of the elements in a row vector creating a new row in the resultant matrix."
It would be nice for the beginner to see an example of this and as stated it wouldn't take much space in the book to provide one. I think these sort of things would differentiate your book from others. If you made it more friendly more 'practitioners' would be willing to read/use it end-to-end.