I don't necessarily agree with this. Yes, a sound understanding of the domain and knowledge of the mathematics and statistics are vital to gaining insights. But. I would make a very clear distinction between exploratory data viz and explanatory data viz. Data visualization when presenting those insights is an important part of driving decision making.
I don't fully agree with this neither. Especially for mathematical concepts, visualization can give insight into how theorems are constructed and combined. This can prove to be vital when applying concepts and theorems to new problems.
I would especially like to bring forth 3blue1brown[1]. He is a creator of videos which beautifully visualizes and explains complex mathematical problems. His efforts has given me an insight into math which theorems explained in text and variables could never do.
However I do see your point that visualizations without understanding can be misleading. Hence the pure, written math is important to read and reason about, but I do believe that some concepts need to be visualized to be fully understood.
With regards to gaining math skills, this upcoming MOOC from Microsoft on EdX looks promising[1].
[1] https://www.edx.org/course/essential-mathematics-for-artific...
[1] https://www.edx.org/course/essential-mathematics-for-artific...
If you don't have an industry in mind, you can use a site like glassdoor.com and search for data scientist positions by city and industry to get a feel for demand.
Basically, any problem where you can establish relations between elements can be treated as a graph. I've used graphs for image analysis before too: pixels are vertices, edges represent neighborhood relations - especially useful when you make nonlocal connections (e.g., nonlocal means; graph-cut methods for segmentation; etc...)
I've worked with them in three of the above contexts: cybersecurity (my current projects), retail analytics, and image analysis. I've avoided social network stuff - never cared for that area much.
I definitely think a solid mathematical understanding helps to build quantitative and critical thinking skills which are very key in data science.
For example, if you have a bachelors degree from a top engineering school (MIT, Cal Tech, Stanford, Berkeley, etc.) you have proven that you are intelligent and can work hard.
People without a masters degree, but more business experience, bring a different perspective, and are often more business results focused, and potentially work more collaboratively than an individual who just graduated from a masters program.
Source: I am a Data Science hiring manager, and have interviewed 100+ candidates at several companies
I think the commenter’s point is (implicitly) about specific, directly relevant Master’s degrees. Obviously a general Master’s wouldn’t provide much of an advantage. The difficulty isn’t demonstrating intelligence and work ethic, it’s demonstrating targeted expertise.
> People without a masters degree, but more business experience, bring a different perspective, and are often more business results focused, and potentially work more collaboratively than an individual who just graduated from a masters program.
To be honest with you, this sounds to me like complete speculation. I’m not saying it’s wrong; rather it seems like it’s at best unempirical, and at worst unfalsifiable. The qualifiers you’re using (like “potentially”, or “often”) don’t seem like strong heuristics.
I think it would be helpful to discuss straightforward job descriptions. For most real data science roles, I would not weight any of what you’ve listed (except collaboration) as being remotely as useful as demonstrable expertise in computer science and statistics. For candidates without a Master’s degree, I wouldn’t take business experience or lack thereof as a signal whatsoever - I’d look for a relevant heuristic to replace it.
I've had questions ranging from reversing strings on a whiteboard to checking for valid email addresses. I had another question about flipping biased coins and calculating probabilities. It's all nonsense and totally unrelated to the skills I developed during my PhD which primarily consisted of performing massive amounts of machine learning on high performance computing systems over large sets of data to extract important insights.
But — if solving these algorithm puzzles quickly and without errors is the key to a $300k+ job, so be it. I'll just practice this nonsense until I've optimized for the skill of "interviewing", and then maybe I can contribute in some kind of meaningful way to the company with actual data science.
Is there a consensus about what kind of Master's would be most useful data-sciency stuff? Computer science? Stats?
I think the actual term is Data Engineer.
It’s why technical interviews can be so brutal, unfortunately. There are a lot of frauds out there. Money attracts frauds.
What’s the fizzbuzz test for data scientists anyway?
My phone screen "fizzbuzz" is having them calculate a standard deviation from an array of data w/out with only basic operators (no numpy.std). Then explain why they choose population/sample and explain the difference.
I studied math in undergrad so one of my requirements is "knows more math than me".
What kind of questions are you asking to ensure that they’re correct when they’re speaking about math you don’t know?
1) People who are great at the mathematics behind the statistical tooling
2) People who are great at conceptualizing a relevant question, operationalizing it, and then using a computer to apply appropriate models.
I think in most cases, for businesses needing to solve business problems, the latter kind is probably more useful. There are applications where the former is required, but you probably know if you need this kind of data scientist.
I should also add that these traits aren't mutually exclusive, but that individual data scientists typically are stronger or weaker along approximately those axes.
In general, I still dislike the term "data science" because it obfuscates meaningful distinctions between math nerds, computer science nerds, and research nerds who happen to do some applied stats.
I do, however, think anyone with some lick of statistics background should know the formula for a standard deviation. Considering how fundamental the idea of variance is in statistics.
Also, for our role, we're specifically hiring someone with extensive stats background since a large part of the role is learning domain-specific statistics of the industry we're targeting and figuring out how we can adopt those models with our data.
It’s a filter that theoretically allows false positives (which is why you continue with other questions), but it really shouldn’t have any false negatives.
This is just my n=1 opinion, but this is a terrible test for data science skills. I've had to calculate standard deviation by hand many times in my life, but my short term memory is such that despite doing that dozens of times over the past two decades, I still can't recall the formula off the top of my head. And then there's the whole n vs (n-1) thing in the denominator which has something to do with degrees of freedom, but I would just Google that as soon as I needed to know (depending on exactly what I was trying to do with the data).
So I don't understand how your question in any way tests someone's skills at analyzing data to extract valuable business insights. At best, it tests someone's ability to memorize formulas and minutiae (although I'll grant you that understanding the difference between a sample and the population is important).
Personally, I think take-home interviews with real data sets are the best way to gauge a candidate's skills. You're actually testing them with a work sample, and they are not under artificial time or memorization constraints.
Read through, and do all the exercises in, one textbook each for:
1. Calculus
2. Linear Algebra
3. Abstract Algebra
4. Analysis
5. Topology
6. Probability Theory
7. Number Theory
...more or less in that order. Make sure your calculus book covers single variable and multivariable calculus. Supplement with applied mathematical statistics. Do that, and you have the equivalent of a mathematics undergrad (as far as relevant courses are concerned).
You could even do this with something like UIllinois’ NetMath program, or some courses on Coursera. You can swap out Number Theory for Complex Analysis or deeper Probability Theory and it’d be more relevant.
Skip abstract algebra, topology and analysis. If you find yourself in the same room as a number theory book, walk away slowly without making eye contact lest it cast a spell on you.
Sure, skip number theory. Like I said, you could swap that out.
One can learn the necessary topology, analysis (etc.) in the relevant places (and the relevant depths) that they come up.
1. The context is knowing more math than someone who has an undergraduate degree in it,
2. Abstract algebra is part of such a degree, and contributes significantly to overall mathematical maturity, and
3. You can avoid some subjects in the short term, but in the long term you can’t progress further without a reasonable mastery of algebra and analysis.
Probability theory and linear algebra are heavily used in data science. You won’t be as competitive a candidate for a job if you don’t have a firm grasp of both subjects. At a certain point, linear algebra ceases to be distinct from abstract algebra, and those exercises you were doing become applicable to real world results.
But, what you're describing I would consider "data engineering" (at least how I have been hired to do it). Working through the business problems and pragmatically facilitating data, pipelines, databases, and models to solve those problems. It's less established and less "hot" but, IMO, it's a much more valuable job to most businesses.
Much like how the early hires at Twitter were not deeply experienced in high availability work -- segregating the architecture of a predominantly RoR code base to be resilient at scale, which lead to countless "fail whale" outages, before they eventually landed someone who helped them re-think their architecture to use RoR for what it's good at while introducing the JVM and other languages to handle other aspects of their workload.
I'm having a little trouble trying to parse that sentence. Could you explain it better?
Based on what I think is being asked, the question is essentially: What is a STD? I think this is a very straightforward and fair question.
For less Stat-y HNers: For normally distributed data, the STD is the root of the Variance. The Variance is just the average of the square of the difference between the data points to the mean. Essentially: Take a point, find the distance to the mean, square that, average over all points you've done that to. That's the variance. Root the variance, that's the STD.
Languages and libraries are just tools: knowing APIs doesn’t tell you at all how to solve a problem. They just give you things to throw at a problem. You need to know a few tools, but to be honest, they’re easy and you can go surprisingly far with few and relatively simple ones. Knowing how, when, and where to apply them is the hard part: and that often boils down to understanding the mathematics and domain you are working in.
And don’t over use viz. Pictures do effectively communicate, but often people visualize without understanding. The result is pretty pictures that eventually people realize communicate little effective domain insight. You’d be surprised that sometimes simple and ugly pictures communicate more insight than beautiful ones do.
My arsenal of tools: python, scipy/matplotlib, Mathematica, Matlab, various specialized solvers (eg, CPLEX, Z3). Mathematical arsenal: stats, probability, calculus, Fourier analysis, graph theory, PDEs, combinatorics.
(Context: Been doing data work for decades, before it got its recent “data science” name.)