I've written around 15000 lines of MATLAB for my research and only a handful of people will ever need to see it. Some is well-structured and nicely commented, but other parts are incomprehensible and were written under severe time constraints. My advisor is not much of a programmer and will not be able to figure it out, and I feel bad for leaving a pile of crappy code to the person who inevitably follows in my footsteps, but I ultimately have a choice between writing fully commented, well-tested, and well-structured code and graduating a semester late (at the cost of several thousand dollars to myself), or writing code that's "just good enough" to get results on time. This is a solo project (there is no money for a CS student to intern) and I'm not getting paid to write code unlike a professional programmer, so every second I spend improving my code beyond the bare minimum costs me time and money.
Even if I were able to tidy up and publish all of my code, most mechanical engineers would not be able to understand it because most can't write code. Those who can mostly use FORTRAN, although C is becoming more common. Nonetheless, even those who could understand my code would have little incentive to read through 15000+ lines of code.
Unfortunately, as far as research code is concerned, a lot of trust is still required on the part of the reader of the publication. I agree that the transfer of knowledge should be handled differently, but until there is a strong incentive for researchers to write good code it will continue to be bad. Especially when many research projects only require the code to demonstrate something, after which it can be put in the closet.
This concerns me. Is this kind of thinking pervasive in public academic institutions? Avoiding the copyright ownership issues that tend to accompany such discussions, would it not be better to be more open about the code in an attempt to gain peer review?
I understand your personal motivations about not publishing, but the statement about your advisor is what I'm worried about.
Of course, the problem is that it can sometimes take years to get large datasets published and this means that the code gathers dust and gets forgotten in the meantime. By contrast, the papers and results aren't, because those are the things by which academic careers are measured.
I would personally support a wholesale change in culture in this area. Code and data/results/conclusions are not as inseparable as most scientists would like to believe, and often should be published as a unit. There has been push in this direction for a while in the engineering sciences, but other informatics disciplines like biology lag badly in this respect.
As to the last point, perhaps it's time the scientific community took software into consideration along with the data and it's resulting papers. At the least, acknowledge the problem. At best, decide where (alongside the data? with the paper in progress?) the software should be stored.
(I'm not faulting you, you just react to the incentives.)
* Publish, so you can get grants
* Use that grant so you can publish more
* Get more grants;
* Get tenure somewhere in the middle.
I have to confess I was very disgusted by him saying that in front of such a large audience of scientists and graduate students.
EDIT: Formatting
When a more appropriate way of quantifying research output and its benefits is found, hopefully a beneficial change in culture will trickle down into the academic trenches.
Until there is that job security, knowing that as long as you keep working you're not going to be randomly turfed out, this phenomenon will be a fundamental part of the academic career.
Really? Your funding isn't guaranteed?
When I did my Master's I was funded as an R.A. without my advisor/lab having to tap her particular grants. Grants and fellowships were usually seen as something "extra" for master's and Ph.D pre-quals students, not their main source of funding. I find it surprising that your school or department seems to (or is forced to) think differently.
It might have something to do with recent state budget cuts (it's a state-funded university). My department has also grown dramatically over the past few years, both in terms of faculty and students, so the graduate student funding will probably lag behind for a few years more.
This is starting to shape up on the Python side of things, but it has stagnated a little bit. People who can and do write the foundational code are oftentimes too focused on making the code work, and not at all focused on improving the quality of the ecosystem that their code is part of. Open Source is a great mechanism for many things, but polishing up the last 20% is not one of them.
Well, to some extent products like MATLAB solve this problem. For better or worse, I trust Matlab's ability to generate a (pseudo) random number, parallel process my functions, invert matrices, etc., etc.
On a broader level, thanks to the specialization of academia, chances are that the code I want to write isn't duplicated by others. Even if it is, I still have to trust them to have written it well - which is the whole problem here.
I guess I don't have as much hope as you do.
http://www.willowgarage.com/blog/2010/04/27/reinventing-whee...
I've read about it in the context of speeding up global-illumination path-tracing for computer graphics.
I think it's based on work that was originally done for neutron scattering.
The problem with doing science via models/simulation is that there just isn't a good way of knowing when it's "right" (well, at least in a lot of cases), so testing and verification are imperative. I can't tell you how many times I've laid awake at night wondering if my code has a bug in it that I can't find and will taint my research results.
I suspect another big problem is that one student writes the code, graduates, then leaves it to future students, or worse, their professor, to figure out what they wrote. Passing on the knowledge takes a heck of a lot of time, especially when you're pressed to graduate and get a paycheck).
There's got to be a market in this somewhere. Even if it was just a volunteer service of "real" programmers who would help scientists out. I spent weeks trying to get my code running on AWS, which probably would have taken a few hours from someone who knew what they were doing. I also suspect that someone with practice could make my simulations run at twice the speed, which really adds up when you're doing hundreds of them and they take hours each.