I can do everything you can do.

When these jobs are hot and candidates are plentiful, using signals to narrow the field to a group you can more rigorously interview is typically a more effective use of time than buying into everyone's self-belief. Candidly, I find most individuals from a programming background vastly underestimate the skillset required in this space. I know that is not an uncommon perception and you are likely being penalized for it, fairly or unfairly. I'll say this: anyone who refers to data science as "just linear algebra and calculus" would be immediately removed from any candidate pool I was managing.

Others have evidence of capability, you do not. Programming experience is not evidence enough to elevate you above candidates with more reliable and relevant credentials. A shelf full of books is not evidence either. You either need to find a version of this job created by people that don't really know what it is they want (hint: if the Data Science JD says "Excel" that's an indicator, it's not too uncommon) to create a work history, find a way create a portfolio that you can use as evidence (e.g., Kaggle competitions, hobbyist projects with available datasets), or network with others in the industry and academia such that they will vouch for you.

Sorry but how would you even know?

I understand your frustrations well since I don't have a BS.

Still, as much as you seem to want to talk about how capable you are, you can't seem to understand the perspective of employers. Given what you've said about your history I suspect this is not due to a lack of intelligence but empathy.

Employers have to go through many candidates, each of which has some true capability but of which the employer can only see some signals. Signals have varying degrees of quality, and interviewing candidates costs time and money. That being the case, it is only natural that they try to use the strongest signals they have.

Nobody believes that there are not capable people who do not have an MS or PhD as you seem to be suggesting. The reality is that the proportion of people who have a BS and can do the job is much less than the number with MS or PhDs, and so it's one of the more effective filters they have at their limited disposal.

I'm sure companies are not happy about skipping great candidates like yourself, but they have not figured out a way to do so that is scalable and cost efficient. It's a difficult problem but maybe you can figure it out.

> You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for.

How much do you even know about PhD programs? It seems like not much because PhD candidates, at least in the US, get paid. It's not much but they certainly are not paying for their education.

I apologize for this unsolicited advice but your lack of humility is frankly very off putting. You sound like a very hard working person. PhD programs are extremely difficult to both be admitted into and to finish. I'd expect that you would respect others like yourself who are very hard working.

Companies are using a PhD as a proxy for having research experience because it's the the only qualification like it out there. It's a poor proxy because not all PhDs are created equal.

This is missing my pet step: doing the literature review.

I’m pretty ambidextrous when it comes to Python and R, so I’m not typically a combatant in the data science language flamewars.

But... for as much as the Python community likes to assert their superior coding chops, I’ve observed that the R community does a much better job of reading about prior art.

One of the earliest, most important, and most useful lessons I learned from a senior grad student: "a day in the library can be worth a week at the bench."

This is a grossly defensive overreaction to the parent reply.

The problem is that there are hundreds of applicants in your situation WITHOUT experience. There are usually a couple of PHD or MS applications WITH experience for every job. Who do you think the company would give preference to?

So this sucks if you don't fit the model well in a way that has you often end up as a false negative - but that doesn't' mean the model is broken.

You are claiming there is a generalization problem that causes extra error in practice. Another perfectly viable hypothesis is that the classifier is working fine, it's just tuned for true positive rate and accepts a higher false negative rate to get it. Specificity vs. sensitivity is a fundamental trade off, not a training issue (though that can make both worse)

If you're serious about getting a PhD-level job without a PhD, getting someone to recommend and vouch for you is even more important than usual. Since you're up to date with papers, why not email researchers you admire with questions that demonstrate you deeply understand their work? Many will be too busy, but some will probably be impressed by your determination. Once you have a relationship, see if you can assist with their research, even if initially it's just grunt work. It will take time, but integrating yourself into the academic "web of trust" and maybe getting your name on some papers is the only plausible way you can expect a company that doesn't know you to take you seriously.

There's always a domain specificity that sometimes comes from grad school, but data is data and industries are filled with SMEs who understand the domain.

Source: I hire data scientists for Fortune 500 companies.

>I can do everything you can do.

To demonstrate that to an employer wanting PhD workers, go get a PhD like the other PhDs did. Claiming you can do what they do when you haven't done what they have done is not going to cut it.

>you won’t even call me to talk to me that means you are missing out, and you are gatekeeping

Gatekeeping = not spending unnecessary money and wasting unnecessary time.

An employer saves significant money and time by not having to interview everyone claiming they can do what PhD can do but didn't bother to get one. Your skilled workers don't have to stop producing and do interviews, your HR people don't need to spend time and money booking flights, hotels, and such for candidates. You don't have to work through 500 resumes with 30 PhDs in the pool - you sift through 30 resumes.

All this hype makes it so everyone wants to be a data scientist. You get people who change careers to go into this new hot career. You also have a pool of people who have been working with data well before the hype with experience. The people trying to break into data science will have a very hard time competing with the people with experience over the pool of jobs out there.

I don’t have an advanced degree, just a bachelors of math from 1995, and have been breadboarding (and more), and coding since the 80s

I can follow along with ML and have implemented toys with the ML algorithms in a couple days.

It’s bourgeois intellectualism. Like a law firm only hiring from Harvard

ML is automated schema design. And the current methodology has known limits of applicability

This is “Mongo DB”, “devops” like hype all over again.

The days of a researcher producing a model to be re-implemented for production by a programmer are over, or very nearly so. A working data scientist now is expected to produce something that can run in production. That’s something a PhD doesn’t teach and that many PhDs find an uphill struggle.

Right now doing this is somewhere between “state of the art” and “new normal” depending on where you sit.

We have developed tools to deliberately abstract away the complexity of the underlying math, and they work well. They work exceedingly well. I once took a semester-long class in data science where untrained, mathphobe business students were running various kinds of regression models on cleaned up data in WEKA (poor choice of software, yes) by the end of it.

Most data scientists can and should treat the algorithms themselves as black boxes the same way software engineers treat R-B BSTs as black boxes, without necessarily having to know how they work off the top of their heads.

I once worked with a Stanford "data scientist" at a top tech company who couldn't immediately recall Bayes' Rule. He wasn't stupid. His knowledge was just structured in a way that reflected the reality of his day-to-day; this is how it works for all of us.

The primary skills needed are: munging, featurization, analysis (basic stats and then a few other things like ROC, etc), and perhaps most importantly (and the thing I see PhDs in particular chronically fail at) operationalization. You do not need to know heavy math to run a model over data, which is the maximum level of sophistication required for most applications of data science that generate real business value.

I see similar foolishness in data science as I do blockchain, to be quite honest: people hype up and gravitate towards the cutting edge, while forgetting, ignoring, or blatantly obscuring the power of simple math on big data. I guess it's important that people have an inflated view of the complexity of what most data scientists are doing because having the role at least somewhat cloaked in mystique boosts salaries in the long run.

Any argument that a reasonably intelligent person can't be trained to be a legitimately effective data scientist is a counterproductive lie.

If you don't reward (By issuing grants for) negative results, or null results, or replicating prior studies, why do you expect that scientists will aim for any of those outcomes?

You have to eliminate a lot of possibilities in a universe with this much detail

The real issue, again IMO, is more of an “expectation crisis”. We expected to repeat this and failed to. Because we’re still a ridiculously ignorant species lacking conscious awareness of many aspects of reality

Is Data Science really fundamentally different? Or is this PHD who barely programs going to either do tasks like cleanup terrabytes of data or risk that a coder with no idea will introduce a bias in the data during that process?

I find the whole emergence of the field fascinating, but I kind of feel like it is just techs recreating actuaries with what is actually a less specific education.

+ (And a worse academic style career path of going through an education you may never get to use instead of going up from apprentice to master)