There is a ton of programming in research, and almost all academic code is written by the researchers, whether they are in biology, physics, humanities, whatever. This has some of the drawbacks you mention, but the solution is not so obvious.

First, almost all academic code is really simple from a software engineering perspective, but really complex from a subject matter perspective. Having a deep understanding of both the data and the relevant hypotheses is critical, and is often actually helped by writing the code yourself. Trying to communicate every feature requirement perfectly, and making sure every assumption is met, to a third party CS person might be possible but is definitely non-trivial.

Second, most of these projects are one-time use (code up some project over a few months, write a paper, never touch again), and so spending a ton of time + money making it robust and efficient is not really worth it. For things like open source tools that are expected to be used by a lot of people it's much more feasible to get engineers involved. The Chan Zuckerberg initiative is actually funding a program that essentially does this [1].

[1]https://chanzuckerberg.com/rfa/essential-open-source-softwar...

It's kinda funny that this isn't widely appreciated. It used to be that research programs were most programs. Use of computers for other things was a kind of spin-off technology. (Now it's the other way around, of course.)

That would require something that is mostly missing in universities: strategy. Cooperation between departments and colleagues without direct sight on short term benefits is very slim. I'm quite close to my local university (father worked there, alumnus, wife works there, friends who are professor; it's a quite good Euro-business and law school). The thing I've learned: a university is more akin to a village than to a business even though the administrators try to run it as a business (but have terms a lot shorter than most professors). There is no clear line of control. Professors are tenured and some have super-star tendencies. Budgets are fragmented between departments, groups, and in our country are built upon subsidies for specific research tasks. There is a lot of cooperation between individual researchers in fields. But that is a response to it satisficing the demands of subsidies. Ýou could call that strategy, but I'd call that 'external motivation'. Strategy as in internal motivation: "How can we get greater research results from all our research staff by leveraging technology" is a question hardly asked.

> but incentivising someone from a field completely unrelated to CS to 'just code' sounds like pretty bad advice.

I’m sorry but learning to code very custom or short lived software is just not that hard. I think you’re making programming too precious.

Don’t get me wrong — writing good, maintainable, long lived software, and basically anything with a UI for non programmers, and certainly anything for sale, is much harder and usually something professionals or their equivalents will need to do especially for an institution.

But writing scripts in interpreted languages like python or ruby for the research purposes of an individual or small group is not something people need to (or more importantly can) hire a dedicated team of engineers to do.

IMO it’s good that programming has become more democratic. It’s an ongoing process. And one that has happened in basically all other creative industries to some extent.

I guess it depends on the complexity of the task. If it's something I could knock up in a few hours time, and experiment with the results myself, why would I engage in endless inter-departmental politics, of who's project is more important, etc. On the other hand, there's a threshold where project is significantly complicated and requires professional assistance.

Within the UK there is a growth in Research Software Engineering[1] groups at universities, primarily due to the work of the software sustainability institute[2].

[1] https://society-rse.org/ [2] https://www.software.ac.uk/