Ask HN: Breaking into Scientific Programming?
Hello HN,
I work as a software engineer in the business and web application world. I love programming and am quite successful at it, but I'd love to transition into scientific programming and data science. I have an undergraduate degree in the humanities, so my quantitative background is honestly a bit lacking. I've considered a second undergraduate degree in, say, math and physics or electrical engineering to fill the gaps. In an ideal world I pursue my studies independently, but my day job doesn't leave me with much free time, and frankly I'm not sure how seriously I would be taken without a proper degree.
Advice from science and data hackers is much appreciated.
(
10 comments
| 13 points
| on HN
)
1. _Stats: Data and Models_ by De Veaux, Velleman & Bock
2. _Fifty Challenging Problems in Probability with Solutions_ by Mosteller
3. http://yudkowsky.net/rational/bayes
Basic data analysis:
1. _Python for Data Analysis_ by Wes McKinney
2. http://camdavidsonpilon.github.io/Probabilistic-Programming-...
3. _Exploratory Data Analysis_ by Tukey
4. _The Visual Display of Quantitative Information_ by Tufte
Tools:
- R & ggplot2 & (Sweave | knitR)
- Python & numpy & pandas
- UNIX tools (https://news.ycombinator.com/item?id=6046682, https://news.ycombinator.com/item?id=6412190)
- basic SQL (https://schemaverse.com/tutorial/tutorial.php)
- data visualization: (R & ggplot2) | (Python & matplotlib) | d3.js
- OPTIONAL: C/C++/Java for hardcore Bayesian stuff, Julia for being cool, Fortran for specific academic domains
On getting people to take you seriously: If you knew the stuff up there, I would take you very seriously, even without the STEM degree. You can pick this stuff up outside the classroom (in fact it might be hard to find uni classes that cover this stuff). So if you did self-study, and blogged about it or something, people would take you seriously (esp. if you got good at something "hot" like d3.js or Bayesian). In fact, given your background in web / software / business, you could be considered even more valuable (by web / software / business people).
What are you interested in specifically? Where do you want to end up?
Why do I think this? Because (1) the day-to-day of "data science" is more of a craft than a science. Before you run complicated analyses, you are just cleaning the data, visualizing it, trying to see what questions it could answer. It feels a lot like playing in the woods to me... you're just knocking about, seeing what cool stuff is in there, building a fort out of scatterplots. And a truly wise data analyst will know when to not run tests at all. Also (2) it would surprise you how many academics are themselves kinda self-taught. They get into research because of their interest in a specific topic, and only come around to learning stats later. Think about it! The MCAT does not test prob/stats, even though doctors end up reading reams of statistical studies for work. I'm not saying these doctors are ignorant, not at all... just that after undergrad, they (and a bunch of PhD candidates) find themselves in the exact same position you're in. They teach themselves, or they squeeze into stat dept courses, and they all turn out OK.
I would just look for the data that's already around you. Cool data being produced at work? Ask if you can make some charts in R/Python/Julia to show your team. The internship suggestion was a great idea. Or you can take on some cool longish-term project and blog your way through it online. You'll do the next thing, then the next thing, and so long as you keep the awe, you'll end up somewhere neat!
Regarding the degree, credentials are important (and imperative if you want to direct your own research), but one option is to start out by contributing to an open-source project. If you have a specific area of scientific interest, then be strategic and find a project in that area. To take biology as an example, I would look at something like CellProfiler (they are on github!). Also read the papers published by that lab to get a sense for how the software is used. There are many other open-source scientific software projects, and contributions to a project could give you a foot in the door to employment as a developer in the field.
Another thing you can do is attack real world problems,there is a shocking amount of bad data science out there. See: http://www.slate.com/blogs/moneybox/2013/04/16/reinhart_rogo...
It's slightly advanced, but it might be great for you for just taking materials from to explore the scope of scientific computing at large.
It goes in to things like digital signal processing, some computer vision and other things.