I have seen some ninja FP artists that have written some really terse solutions in a FP style which are completely unreadable to the uninitiated.
Also, I do not think that there is anything intrinsically difficult about FP. Knowing it proves that you are ambitious and that you have mental plasticity but not necessarily that you are a good developer.
You're just as likely to become that weird dude that writes code no one understands as you are to become Mr Wizard.
It is rare the case when a company will hire an academic person to directly lead a company effort, and when this happens it is because the person is already famous and has a lot of consulting/real world experience.
The path you mentioned is great if you can find a company where you can make a major contribution using your skills, but for the reasons I mentioned it has still the same problem (bozo bit) that you cited before.
I actually hear of this happening a lot for economists in Silicon Valley, even with unfamous people.
I guess at least on the math side you can prove your algorithms against historical data and so merit assessments could be fact based.
i got interested in financial semantics lately and was amazed to find how many people are doing algorithmic trading with accounts anywhere from $10k to a few mil. in fact i know of many startups developing products and services aimed at such people.
if you read the MSM you'd think it's all about HFT and big players, but I think algorithmic trading today is a lot like day trading was in the late 90's.
Being able to derive a linear regression is less important than being able to see the end from the beginning and being able to read math. I know a coder-cum-CEO (freshly minted) who has a hard time understanding how to program a numerical derivative, yet has produced amazing works based on what he does know. I said it before--we all need each other, especially as the focus has been on specialization (so that no one person has the entire skillset requred).
[1]http://www.volunteermatch.org/search/index.jsp?v=true&af...
You can make Java a lot more fun if you use generics, functional patterns and frameworks like Google's Guava, even if you still need to create a FactoryFactoryFactory once in a while.
When I program Scala or C# I tend to curse type erasure in Java generics, but when I actually program Java the generics system makes me smile. I can very efficiently turn what's in my head into a design and code.
In a language like Haskell whatever you gain from the language you tend to pay back in missing or buggy standard libraries. If I was writing a webapp in any off-brand language, for instance, I wouldn't trust that the urlencode function works correctly until I'd written some test cases.
I'll admit functional Java is a little verbose (can't write a closure in just one line) but the Hotspot compiler is very good at inlining functional code and collecting the garbage from it. I can definitely say that I get much better performance w/ functional Java than I usually see with the Interpreter pattern with two orders of magnitude less code size.
The software engineering career is in somewhat of a mess right now. It comes down to the "bozo bit" problem. Being a software engineer (even with 10+ years of experience, because there are a lot of engineers who only do low-end work and don't learn much) is not enough to clear the bozo bit, and you won't be able to prove that you're good unless you have a major success, and it's hard to have that kind of success without people already trusting you with the autonomy to do something genuinely excellent.
It's not enough to write code, because LoC is a cost and source code is rarely actually read at the large scale. At least for backend developers, the only work-related (as opposed to political) way to establish that you're worth anything as a engineer is to have an architectural success, but it's very hard to have architectural successes unless you've established yourself as an "architect" to begin with. So there's a permission paradox: you can't do it until you've proven you can, and you can't prove you can do it until you've done it. Hence, the vicious politics that characterize software "architecture" in most companies.
Functional programming is one way to put yourself head-and-shoulders above the FactoryFactory hoipolloi. The problem is that most business people don't understand it. They just think Haskell's a weird language "that no one uses". Elite programmers get that you're elite if you know these languages, but most companies are run by people of mediocre engineering ability (and that's often just fine, from a business standpoint).
It is true that functional programming is superior to FactoryFactory business bullshit, but not well-enough known. Good luck making that case to someone who's been managing Java projects for 10 years. What is better known is that mathematics is hard. It's a barrier to entry. I doubt more than 5% of professional programmers could derive linear regression.
So I see the data science path (and yes, it takes a long time to learn all the components, including statistics, machine learning, and distributed systems) as a mechanism through which a genuinely competent software engineer can say, "I'm good at math, and I can code; therefore, I deserve the most interesting work your company can afford to fund." It's a way to keep getting the best work and avoid falling into that FactoryFactory hoipolloi who stop advancing at 25 and are unemployed by 40.