And the flaw in the logic is revealed here, I think: > nearly any programmer is capable of doing "data science" or "machine learning"
Nearly any programmer might very well be capable of learning these disciplines, but they're different disciplines from business logic or database theory or networking; they're non-discrete and statistics-interpretation heavy, and it's entirely plausible that you can have an entire company full of programmers who are neither excited about nor prepared to take on these categories of challenge. Some programmers will (lots of people are polymaths), but it's not nonsense to consider them different categories of discipline.
I think smart companies provide an internal channel to get into these sub-disciplines, but it's not necessarily a problem space you can throw at any developer and say "Here you go; good luck tuning those training and test sets!"
I recently worked at a company who employed "data scientists", they rarely worked in isolation as far as I could tell, and usually required heavy assistance form system/database/storage administrators to be successful and aided by (but not always) with software engineers.
In the case of ML, I still see it as just software engineering / computer science. Sure there are pioneers in the field of deep learning, but aren't most of us just leveraging APIs or existing algorithms?
Honestly, I'm pretty confused myself lately and I work with some of this tech. Open to enlightenment on this topic by the way.
What you describe with respect to data scientists needing help is what I'd call "data engineering." A data science person who is supposed to be doing production-facing engineering should be a competent enough data engineer that he or she can extract information from the data stores without having to have outside help most of the time, but strictly speaking, in my view, these are two different kinds of development.
There are a few Bigcorps with real, novel Big Data datasets that need original statistical work slung at them. These companies need data scientists, for real: people with Ph.Ds in statistics. These people are indeed not interchangeable with programmers. They're partially interchangeable with academic computer scientists, and mathematicians, and physicists; but not with any trades-workers, including your average programmer. Their job is Science, capital-S: to do sound, replicable, high-powered experiments to learn things about the data. That is a skill in-and-of-itself, and one you need a lot of practice with—under the scrutiny of people like Journal editors—to get right (because it's hard to tell from your own observation if you've got it wrong.)
Everyone else, though—that is, all the companies but the SV Big Five and maybe some mobile-game and advertising firms—when they say they're looking for "data scientists", all they're really looking for is data engineers. The actual "data science" they're trying to do is a solved problem: you can find out the same thing a data scientist would tell you from five minutes of Googling the problem, because dozens of companies have solved the same problem for their own data-sets already and written case-studies on how they did it. You only need to implement said solution. Which, like I said, any competent programmer could do, after a few hours spent reading the API reference of a statistics package.
The majority of "data science" (read: data engineering) jobs aren't looking for people to Do Science; they're looking for regular programmers clever enough to have self-selected into the higher caste by doing nothing more than having learned some statistics-package APIs and then marketing that skill. (Much like the "officer class" of the American military in the 1800s allowed people to self-select into it just by being voracious readers.) It's basically an implicit open question at the beginning of the hiring process, "what do you want to be doing all day?", that is answered by choosing a different (pretend) job title.
The fact that data science is basically statistics supports the argument of the guy who is saying data science and software engineering are different jobs.
If you are a software engineer and want to be a data scientist, you are free to switch careers. Or maybe work as an engineer for a startup that could use someone to do statistics on the side. But that might be a bad move with no experts around. With data science, your work product is often conclusions and advice. Those are highly susceptible to bullshit. You don't get a compiler or unit tests that tell you that you are coming to the wrong conclusion and giving bad advice. It takes experience and judgement to get that right, not just hacking on an IPython notebook and accepting what the model spits out.
Perhaps it's because people with formal training and degrees often have better skills than some random kid who taught himself javascript and thinks that that makes him a software engineer?
And then, having found all this great talent, they let it rot, under-employed in maintenance work. Because they were hired to "do programming", not to "do data science" or "do machine-learning" or any of the other 'specialized disciplines' that they could totally pick up given about two hours, given how freakishly intelligent they already have been proven to be.
(If you're wondering, this isn't bitterness; I've never worked for these companies myself. I'm just summarizing a number of rants I've heard from others.)
The point of treating data science as a field is to distinguish the people doing it from "regular programmers." This creates a distinction of caste: the "regular programmers" are relegated to maintenance work, never trusted to touch any of the mathematical and CS-degree-requiring parts of their own discipline. The "data scientists" are then hired from outside, to do precisely the parts of programming that every programmer at the company would have already been proved competent at, through degree-qualification and technical interviews that seek out exactly these talents. And the "data scientists" are paid better for it, and given better working conditions, and less micromanagement.
Same with machine learning: it's just programming—certainly, a kind of programming that is heavy on CS, but certainly one you can pick up quickly enough to get some good practical results. But you hire ML people separately from "programmers", and you never hand your "programmers" ML tasks.
From yesterday's subthread on the wind-down of Starfighter, I gather that a large part of its goal was to reveal that the emperor has no clothes here: that nearly any programmer is capable of doing "data science" or "machine learning" (and, as well, "reverse-engineering", "low-level programming", and a host of other things) if actually given a chance to solve the problems.