Another way of looking at it: People doing STEM PhDs are extremely busy. They have to master the content in their field, keep up with new research, conduct their own research, write papers, go to conferences, give talks, teach, keep up with whatever department service responsibilities they have, etc.
They're smart people so they're going to be able to slap together some code that does what they need it to do. But to expect them to learn git and how to use it well, to learn about managing dependencies, about unit tests, about build systems, about best practices for documenting code, about programming abstractions or OOP whatever else... that's a lot to ask.
(And I say this as a STEM PhD who did learn all that stuff, more or less.)
If I had to do this kind of things again, I might do them properly, but I'm not 100% sure. Back then, I had more pressing things to do than writing excellent code, since I was writing code to drive the research (as in, numerical investigation), not as proper research (the end results were theoretical results)
I think the real problem in academia is the same as the problem in some organizations that also don't follow best practices: the people empowered to make the decisions about whether to invest people's time into improving testing, building, technical debt, or documentation do not actually believe that an increase in productivity, increase in access or decrease in bugginess will be achieved.
The time to learn all of this is almost like a field in itself say Software engineering.
I just need to get shit done.
I had a project to do bayesian hierarchical modeling. I wrote it from scratch in R without using a MCMC framework.
Yeah it's ugly, yeah it's slow. But I got initial results to satisfy my mentor and within the time limit of the summer internship. This is on top of learning two semesters worth of Bayesian statistic in 3-4 weeks.
Time isn't expendable and I can't devote my life to every single thing while neglecting other aspect (love, health, etc..).
Yeah your theory is nice only if you don't have a deadline, a life outside of work, etc...
Not all academic disciplines are as hectic as yours. When I was in a top grad school, most of my colleagues were not under the stress you describe. And yet their code was still crap.
Incentives are a bigger reason than workload.
Serious questions: Where was this amazing place? When were you there? Was it a STEMish PhD? Really.
I've never heard of a grad school in STEM these days that is anywhere near like that. I'd love to know because the way STEM PhDs are going in the US these days, I can't recommend them in general, only specific PIs and labs.
Yes. Engineering. I don't like to reveal too much about my past, so I won't say the name. Suffice it to say that it's ranked in the top 10 for pretty much all engineering + many of the sciences. EE, CE and CS are all in the top 5.
It was a decade ago, though.
Workload all depended on your advisor and funding landscape. Some were very demanding, and they had miserable lives (mostly pointless, too - most of academia seems to be converging towards a race to the bottom). But not all professors were that way. Of course, standards still have to be met, and laid back professors' students usually took longer to get a PhD because they were more relaxed.
To give you an idea, two of my fellow group mates are now faculty members. And the one whose work was most like what you describe, and who did the best work IMO, never was able to secure a tenure track position.
The lifestyle you describe is really not worth the PhD you get in the end. The ROI is just not there - financially or otherwise.
I also feel like it's a race to the bottom now. It used to be 'publish or perish', now it's 'funding or famine'. The paper numbers, impact, and truthfulness have nothing to do with anything. I see so many PIs that essentially end up as one income households, their spouse's, not their own. The 'good ones' just get pushed out or leave, an evaporative effect, leaving only the liars. The academy is in real trouble these days. It isn't on life support yet, but it's not able to climb the stairs anymore.
I once had a quite respectable lecturer working for me, and his code was awful. But that was ok - we knew that. He was hired for his conceptual skills and domain knowledge, and we paired him with a junior developer that could take his raw output and turn them into cleaner code.
It's some engineer's job to do it properly in an actual production setting.
No. One's point is to communicate one's ideas. Code is the only specification of a system detailed enough to reproduce the desired results. You wouldn't present an unfinished paper written in the equivalent of crayon so that particular attitude is the worst kind of lazy bullshit for people actually trying to make use of your hard work.
> It's some engineer's job to do it properly in an actual production setting.
Fuck this attitude so damn hard. As an academic, one _is_ the _first_ engineer who has a duty to lead one's colleagues - to mark as clearly as possible the path for those who come after. This "fuck you, got mine" attitude is unacceptable and unsustainable.
The idea is everyone writing everything the "right way" from the get-go would be premature optimization.
The right way in this case doesn't mean AdapterFactoryBeanFactoryService-land. It means taking care that your code is clear, concise and correct for others skilled in the art. The fact that academic code exists at all, as an example, means that it _is_ in production. There's no excuse for shit code that isn't written for other human beings first and computers second.
We're all busy. How long do you think it takes for time spent learning good software development practices to pay for itself? Surely not longer than the length of a PhD.
Sure, but if the core of your job, the key skill you bring to the table, is writing software, you are (hopefully) going to find the time to learn best practices for writing software.
If that's not the core of your job, if your key skill is something else and writing software is just an occasional means to an end... maybe the incentives aren't there.
> How long do you think it takes for time spent learning good software development practices to pay for itself?
If you almost never write a program longer than 1000 lines and rarely use a program six months after you've written it, it may never pay for itself.
Don't get me wrong, I personally have an interest in good software development practices. I just understand why not everyone can or will take the time to learn them.
When is your first and last meeting of the work week? 6am Monday to 7pm Friday? Have you taken more than 3 weekends off in the last 12 months? Do you measure your daily coffee consumption in gallons? And yes, I'm serious.
PhD-land is crazy, perhaps this is why sooooo many drop out.
I wish the state of affairs was prettier, but I doubt it will change any time soon. And frankly, I don't think it has to. Software engineers will always find something to complain about others' code (coding style discussions come to mind). If there's value in commercializing scientific prototypes, experts should rush in and do it. There's no need for us to invest in perfect code just in case someone may want to reuse it. Having said that, NSF is starting to fund initiatives to improve the state of affairs. For my field, it's EarthCube: https://www.earthcube.org/ but this seems to be fumbling along; all of their goals would need to be adapted by scientists outside of that community (standards, etc). Without the right incentives (funding, publications), it's just not going to happen.
Can you identify (say) five things that academics can do with an existing code base that will make your job easier (and therefore cheaper to the downstream users)? Perhaps pitch the top five pick at the abilities of a computer science student looking for a project?
Can you identify (say) three key practices to adopt when starting a new project given the likely skill levels of postgrad students in your domain?
You could get an ebook out of those I imagine.
1) test suite - this shouldn't require explanation
2) documentation - both top level docs with examples and well-commented code. keep track of units
3) one-step build - I should be able to download the project, type 'make' (or something equivalent), and start using it
You have a different view of how academia works than most academics.
Most PIs these days do NOT collaborate and are actively hostile towards it. It's a shame, but it's true.
Also, the niche-ing of academia is real, in many fields, there may be only 3 other people on the planet that understand what you are actually doing, and many of them may not speak English and you may not know they are out there until 2 years from now (publishing takes a long time).
Most code is written by grad-students that barely know what a for loop is, let alone how to use git, and they only write it to do it once for a specific paper. Look into MatLab, that is the most used language in bio, by far. It's basically psuedo-code that compiles, and it's still a mystery to most grad-students.
Well, yes and no. Speaking as someone who has done this as well: if it's UI stuff, or gluing processes together, it's fine. But sometimes they want you to fix up a custom algorithm they implemented. Which can be one of those cutting-edge scientific things that actually require a PhD-level of knowledge and insight into the topic. So you can only get half-way with untangling the spaghetti mess without sincerely not knowing for sure if you broke something or fixed a bug, because you don't know what the correct output should be.
2. Have significant test coverage
3. Have at least a rudimentary form of Continuous Integration, even if it's just running a script on every check in.
Good developers get paid enough, that it indicates that maybe our job isn't quite that easy to learn.
Our basic programming abstractions and software engineering practices are things that we as a field developed over time, based on experience; it's not like they're something you just automatically know without having to sink time into studying if you're above some IQ level. They also depend to varying degrees on how big your program is, which parts are expected to change more or less rapidly, what kind of program it is, etc.
Unit tests assume you know what the code is supposed to do before you write it, that the code will change significantly more often than the required functionality, and that the tests are easier to read and check than the code being tested.
In practice, at least for simulation code, functional, integration and regression tests are useful when employed judiciously. Most importantly, you verify your results using published benchmarks in the scientific literature or analytic results where possible. Obsessively covering every trivial bit of code with a unit test of its own has always struct me as rather a weird fad.
The advantage unit tests have over regression tests is that the time between the moment developer implemented a change and finding out they broke something is as small as possible. When tens of people work on the same codebase this saves enormous amount of effort.
Fethisizing over some rule for it's sake is silly of course. My org would waste a lot of money and time without unit tests.
Another feature unit tests provide: Think of the unit tests as a living documentation. Breaking something leaves a breadcrumb trail to the invariant which was sullied.
I am maintaining several end user critical projects related to data transform and transport, mostly by myself. I have no idea how I could develop them as efficiently without unit tests to verify my changes don't brake some corner case as new features are added.
The cadence for changes can be fairly low - I might return to some component after doing something else for six months, and I probably need to add some feature without breaking something in the process.
Sure, we have integration testing and smoke testing, but the further a bug travels the release pipeline from the developer the more expensive it is to fix. This cost cascade is quite easy to visualize - the work of whomever caught the bug is stopped, and depending where they are located in the product ecosystem their stall can cause lots of work for other people. If they are a tester they need to file a report. If they are a customer, their work is interrupted, they contact local sales, who then contact global helpdesk, who then identify and log the bug.
Much simpler and easier if there is a unit test to catch the bug in the first place.
Now, there can be domain specific flavors to this. My domain is computational geometry and transport of 3D modeling data between domains. But in my domain anyone not securing their code with unit tests is wasting their employers money and setting their end users at risk by increasing the likelihood of bugs.
There is lot of cargo cult nonsense in software engineering. Unit testing is not one of them. It saves time and effort by catching a range of bugs not caught by e.g. compiler for statically typed language and it secures the program logic for future changes.
Assuming this is serious, FWIW I'm a STEM PhD and I find that even in the smallest scripts, even just web-scraping stuff, if I don't write unit tests then I get it wrong and later discover I need to do it all again. Or worse, I figure out later, after I've built lots of code over this, that there is an error and the subsequent code all needs to be redone.
In short, I find unit tests to be a big time and effort saver. And I get the right answer, for a change. :-)
I've found that unit tests are less important when you have good CI and your codebase isn't shit. On high quality code it takes so little time to fix bugs and redeploy that unit testing is a hard sell.
Do not write unit tests just because the methodology requires it. Write unit tests when the unit needs to be tested. Typical examples:
* Smoke tests (does this code work at all?)
* Tests for known edge conditions
* Tests for known bugs (to catch regressions)
(For predictive models) evaluate the output on a test set? Depending on what you're trying to do, the contents of the black box may not matter so very much if the results on an agreed test set are good (...and you're confident it's independent from any training data...)
Manual eyeballing and sanity-checking of output? (in my experience important however many layers of automated testing you're using, and undervalued by people who focus on software as an engineering process more than an art-form).
...and, circumstantially, probably a bunch of others. I don't think this is a field where making lots of rules is especially helpful (except, perhaps, the "always eyeball" one...)
Even then, logic mistakes can and do still happen.
Funny, that sounds exactly like my IT Dept.'s approach to coding.
I work in Operations and was exposed to their approach when helping them translate some business rules into SQL for the warehouse management system. During the course of interaction the IT Director lost the code we had worked on together twice, keeps the requirements docs in his email etc. etc.
Me: your engineering practices are terrible
He: We don't have time to do it properly
Me: but somehow we have time to do it three times
I told my manager but no-one really understands or cares enough. The Warehousing Director calling out the IT Director about his approach to IT - where would that all end ?
Then there's the group of folks who tend to read those papers and turn it into products. Yes, those are the ones I expect to have repos, unit tests, build systems and all that.
I used to work as a scientific/medical translator and one of the things I loved about my work then was that with each new project, I had to learn enough about a new topic/subfield to become a bit of an pseudo-expert in it. I greatly enjoyed that research and would love to do some work in software development that would require the same type of constant learning with each new project. Diving into some convoluted, 20-year-old Fortran driving highly-specialized scientific software actually sounds kind of exciting to me (yeah, I'm not normal either :) ).
Is there much demand for this kind of work, I wonder? I'm guessing the best way to get your foot in the door is to be in academia, and those days are long passed for me.
As an undergrad, I had a good friend who was doing his PhD in Atmospheric Physics. It turns out, most of that field works in Fortran '88. This is not a very useful language, seeing as it uses GOTO statements to function as a loop. Fortunately, it does have comments. My friend managed to sweet-talk an older PI into giving him the old code for use in nuclear blast atmospherics (Exp: say you nuked all of France, what happens to Greenland's ice). At about 3 am before a project was due in the morning, he was pulling through the spaghetti that was the code, tired, jittery, and over-caffeinated. In this mess of logic diagrams he had to draw out by hand, he finally got to somewhere he thought was going to really cement all the code together for him. He follows a GOTO statement, and there was only a set of another GOTO statements. This went on for about 30 (my recollection of his words) GOTO statements, all 'nested'. Eventually, he gets to one that only has a comment line: 'HAHA MADE YOU LOOK'.
His laptop was defenestrated and he had to buy a new one with me about a week later.
You can staff projects off of grants, so it removes the need to sell the product. And the companies are probably going to be more academic, with a high percentage having graduate degrees. But companies will be under pressure to win grants. And because they have less of a need to sell a product, your work might not be widely used after the project ends. Also, the DoD is a major source of grants, so it would be helpful if you were comfortable with working on military research projects.
But it might be something to look into if you want to get into software development in an academic environment.
Academia is being pushed more and more toward industry collaboration, so if you can get the exposure and connections it shouldn't be too hard to find the opportunities from the inside.
You really have to enjoy this kind of work and some of the frustrations that come with it. I did, but ultimately had to leave when funding for my group ran out. If life circumstances allowed me to work for less I might have stayed with them.
I could keep complaining, but I don't want to give people the impression that I don't like what I do. It beats the hell out of writing CRUD apps.