PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular university had really great code and they thought they could sell it. But they did not even use git and when I gave a presentation there about it, I was met with absolute rejection. They don't have an idea on how much effort it takes to maintain a commercial code, unit tests, customer support etc.
I'm now working at a company that produces simulation software spun out from university. Fantastic job and we do all of those things mentioned above. But obviously my paper output has been near zero even though we sometimes do cutting edge research.
I wish I had a solution to the problem as so much grant money is wasted that produces a paper or two and the corresponding software for that just goes to digital nirvana, it's a real shame.
Also from experience, it helps if universities have closer ties to industry. As an industrial player you can't work with software that does not deliver reproducible results.
Also, I'm currently a postdoc in medical imaging at UCL, super interested in learning a little more about the group in Oxford you mentioned if you're OK with sharing a link/group name? I may be able to guess but just want to check!
> containerisation
Pick one.
A bigger challenge is that most PIs are not project managers and have no experience as such, so they don't know how to express what they need in a structured way, or how to steer their group to collaborate properly. Outside computer science, many would struggle to budget software development or compute properly on a grant application (and the assessors have zero idea either).
The scientist just wants to focus on their research and once they have a barely working proof of concept, hand it over to the engineer to figure the rest out. The engineer wants a well specified design and prototype that they can lightly refactor to clean up, scale up and turn into a product/tool.
The reality is that approach makes it way harder for both, though most often harder for the engineer as they are generally at the end of the chain in Academia and have little power. For example, the code or spec from the scientist is often terrible, so the engineer needs to start from scratch and keep going back to the scientist to spec out the design as they were not involved at any stage prior. They may even find edge cases or flaws the scientist had not considered that are fundamentally problematic to turning it into a viable product/tool.
This is why the big corporate/industry research labs often have high level RSE that are involved in the research process and get their names in papers (they sometimes have PhD's themselves). They are not optimising for the scientists time, but for the companies resources
I know some of the big research councils do this in the UK. For example STFC has a program where they'll work with universities and companies to production-ise research code.
> The scientist just wants to focus on their research and once they have a barely working proof of concept, hand it over to the engineer to figure the rest out. The engineer wants a well specified design and prototype that they can lightly refactor to clean up, scale up and turn into a product/tool.
As you say, this is a great idea in principle. In reality I think that it's really difficult to make it work.
[0] https://www.software.ac.uk/programmes-events/carpentries/sof...
> RSEs should not be writing code for students doing PhD level projects in my opinion
So should a mechanical engineer PhD be designing and making all their own robot parts? Or should the shop engineer help them? The few mechanical engineer PhD's in robotics I know made a few early prototype test parts themselves with help from the shop engineer, but the shop engineer made and even helped design most of it, especially the final prototype.
> As you say, this is a great idea in principle. In reality I think that it's really difficult to make it work.
The point I'm making is that it does work and its proven to work very well (which is why the major industry labs do it). In my experience its Academia that doesn't like it. Anything which appears to take power/freedom away from scientists and gets in the road of their research is rejected. Though I think the core reason is (as other comments have mentioned), there is no incentive for Academia to make it work. The funny thing is that having a RSE working with them would actually help the scientists in the long run and allow them to focus more on the research because they wouldn't have to do everything themselves.
I would argue these should be included in best practices for software engineering.
> So should a mechanical engineer PhD be designing and making all their own robot parts? Or should the shop engineer help them? The few mechanical engineer PhD's in robotics I know made a few early prototype test parts themselves with help from the shop engineer, but the shop engineer made and even helped design most of it, especially the final prototype.
This is an interesting example. Every mechanical engineer I know has huge respect for their in-house machine shops. Everyone has a story about some design they submitted for fabrication, only to be told by the machinist that their design was terrible and they should do it another way. Generally machining jobs are very well-defined though, you have to submit CAD documents, tolerances etc.
The shops in universities I've worked in have a strong incentive to help people optimise designs because they're the ones doing the manufacturing, and they know what sort of things will work and what won't. But by and large this is informal. Usually this comes in the form of "have you thought about designing this another way, because this is really difficult/expensive/time-consuming to machine". Maybe this is just a cultural thing for machinists?
The PhD question - if your project is to design a new type of part then you should probably do the design. Should you make it though? It depends if the project is specifically looking at fabrication. Otherwise it's normal to dispatch this to a workshop.
In my opinion, it comes down to what your PhD is training you for or what you're hired to do as a postdoc. If your job is data analysis, then I think you should be writing code, but you should be able to get guidance and support. If you're a field biologist with no coding experience and you want to develop an app to take measurements, then that's a case when contracting it out to an in-house development team makes sense. I'm not saying it can't work, but the make in making it work is important.
If you incentivize RSE's properly then their time will become expensive and we need ways of figuring out how to maximise their impact.
That would ensure that the contributions of the PhDs does not get lost and they learn how to properly contribute to a project.
There are of course scenarios where someone comes up with some very high impact work, and there's an obvious need to make it robust or user-friendly, spin-out, etc.
It works less well for groups where everyone works on different or loosely related projects. That's not an efficient use of an RSE's time, in my opinion. Though of course you can have a situation where lots of people do random projects using the lab's core code. In both cases, there is a use-case where RSEs embedded in a university can train students on good coding practices.
And what if the PhD’s output relies on software to even exist?
I'm saying this as someone who did a PhD and who wrote a lot of code, including refactoring legacy codebases in my group.
Where the utility in having an RSE lies is where the group is all using some shared codebase that gets touched by everyone. This is the sort of cruft that I had to work with: legacy frankencode that generations of students and postdocs had added to. It would have made a ton of sense to pay someone to spend a year optimising it (which is ultimately what I did). But you want to make sure that RSE's maximise utility in the group. Having them work on individual student projects is not an effective use of their time IMO.
What needs to be said though is that reimplementing something from a paper is near impossible. I had to do that a couple of times and I was only fully successful if the paper was accompanied by some open source code as there always are tiny little edge cases or initialization details that won't be covered in the paper.
They are almost always time and resource strapped, so it becomes a near necessity to deprioritize factors like readability and maintainability.
Still, many RSEs could immensely benefit from applying basic best practices.
I am a mechanical engineer by degree, but a software engineer by 30 years of practice. That has allowed me to thrice straddle the divide where I worked on software that was not just software for softwares sake, but rather as a value add addendum/enablement to an institution whose roots were in more real-worldly devices. I also spent time on the other side of the divide working twice for companies whose primary product was software.
This “less than” is a very real thing. Software is still a very new thing relatively and it’s taken over the world in just 30 years time. The power dynamics, entrenched in generations haven’t had time to rebalance.
I’ve seen this run the other way as well. Where companies that are primarily software, look down on the other disciplines that participate, a necessary evil, but resented.
I have yet to find a company/institution where the trifecta of mechanical, electrical, and software is balanced in mutual respect. If one such exists and you are looking for someone that would love to work in that environment, drop me a line. I’m skeptical that there are any. It seems one discipline or the other always triumphs and tramples the others down. On off axis variant of Pournelle’s iron law of bureaucracy .
I switched from engineering into comp sci at my university and I can safely say that the elitism for software was stoked by the elitism of engineering.
While I was in engineering, everyone bragged about how they were going to get “The iron ring” (a thing we give to only engineers in Canada). When I switched over to comp sci, all my old engineering buddies would say to me, “yeah, but you don’t get the ring” like it meant everything.
After I graduated, I noticed the comp sci “software” folks didn’t like engineering folks much (especially new grads). When I asked why, they shared similar stories to mine. The engineers and their pumped up pride at my local university had hurt their relation to other fields by being arrogant.
I’m not trying to justify this kind of prejudice, but the reasons it happens are fairly obvious to me. It’s sad cause all of us are so similar in our trades and will likely end up doing similar work as well.
In reality the line is much blurrier. There can be no innovation and iteration without implementation of ideas and the RSE work is just as important.
But unless this view changes, nobody wants to be an RSE.
Is there some way to make human respect feel like it's not a zero sum game? The world may never know.
Respect is non zero sum but status is definitionally a positional good. If you’re number 1 someone else isn’t. There can be uncertainty about status but ambiguity always collapses eventually. Everyone can be treated with respect but there will be a prestige or dominance hierarchy in any group of humans, subtle as it may be.
I did tech support at a university for a bit. I certainly wasn't as high status as the professors, but they mostly respected me and the value I provided (especially when you do things like help them recover that next book they were working on, or whatever).
I am a licensed professional engineer (mechanical) and I work in academia (though my work is varied and does involve some code), my official title (in french) is "Professionnel de recherche", which translates to Research Professional. It is my understanding that in most of the english-speaking world, this position (someone that works for a research lab, who is qualified above the technician level, but not a PI nor a postdoc or student) is called "Research Assistant".
I write "Research Engineer" on my résumé/linkedin, because TBH for most people who don't have much experience in academia, "Research Assistant" sounds like an admin assistant (secretary).
Inside academia and outside of it.
Inside academia you are not on a tenure track or similar, and will have to put up with a lot.
Outside academia, your peers will be making 3x or more than you, working less hours, with less stress, etc.
The reason RSE's jobs are hard to fill and often aren't even opened is that they don't make sense. If you are good enough for an RSE job, you will be good enough for research postions at FAANG. Those pay 10x more, so you also need someone willing to not accept that 10x pay, and also willing to work double the hours.
RSEs making a reasonable pay for the skills they require make no sense either, because that would put your pay at 2x that of professors, etc.
On that note I wish levels.fyi had RS and RSE roles...
You don't have to be at a "tier 1 paying company" to exceed 320k TC, that's easily achievable at any tech company that has gone public in the NYC metro or Bay Area. TC at a FAANG (MANGA or whatever they're calling it) for a more senior IC role can easily cross the $500k mark. If you've been their awhile and have been accruing shares that have increased dramatically in value, crossing the 7 figure mark is not unheard of.
So you don't have to stretch the facts too much to realistically achieve the 10x. If you take two very qualified engineer graduating with a PhD, one chooses to go to Google and stay there the other goes to a university and works as an RSE. Fast forward a decade, you'll definitely be seeing a 10x difference in total comp.
Why would anyone want to do that, when a six-figure tech company salary is just next door?
First, if you choose to work for a non-profit, you should expect a huge pay cut. That's the way the society works. Or actually it's the opposite. If you work in a for-profit role, the society allows your employer to keep most of the value you create. If you create a lot of value, that gives you the leverage to negotiate much higher compensation than your peers in less profitable roles.
Second, like in any business, you should focus on what brings the money in if you want to advance your career. You should be the PI instead of the lab technician. Instead of building the software other people ask, you should become an expert in the field and build the software other people will need in the future. For example, bioinformatics as a whole used to be more of a support role, but today there are many high-profile bioinformatics PIs.
Third, many people in the industry fail to realize that you don't really work for your employer in the academia. You work to build a reputation for yourself, which makes you valuable to the employer. If you want to continue working for other people, the upward career path from support roles goes to the administration.
It may be the culture in some ecosystems but it's entirely false. A nonprofit can pay competitively with organizations of its size. I've worked at nonprofits and I've seen this line towed far too many times as an excuse to lower labor costs for people producing value for the organization. If your nonprofit is service oriented in anyway and requires people, appropriately investing in those people through their compensation and WLB is an appropriate investment.
In theory, a nonprofit should be able to outperform and outpay a for-profit entity of the same size, largely because there shouldn't be an expectation of the organization to cut a big slice out to investors but instead to invest back into itself. This could mean increased hiring, grabbing top talent, investing back in a cause and so on.
Very often, the level of funding is what it is and the non-profit has very little control over it. If you choose to pay higher salaries, you get less work done. This is common in the academia, which is usually funded by taxes and tuition fees.
Sometimes there is even an inverse correlation between funding and salaries. If a charity chooses to pay higher salaries, its "administrative costs" increase. Donations may then dry up, because people consider the charity inefficient.
Many non-profits rely extensively on volunteer labor. Salaried professionals often contribute their time and expertise to a worthy cause for free. That puts paid employees in an awkward position. It's hard to argue that you deserve a higher salary when the market rate for your services is 0.
That feels like how it works outside of academia as well.
In the industry, your employer tends to own the projects you work on.
The RSE role doesn't seem to fit into their current model. It doesn't work from a career path, nor from a competitive compensation standpoint.
I suspect that the housing crisis is also going to push more people out of these positions.
If you have a PhD in material science, physics, chemistry, biology, etc... and are reasonably knowledgable in Python (or similar), perhaps spend a year as a postdoc. After that year, seriously consider moving to a tech company.
On the other hand, I've never worked in the national lab system. If you're still monitoring this thread, I'd appreciate it if you contacted me at the email in my profile!
Would love to see more startup medicine development companies ...
Alternatively, would it make sense to provide RSE services as a company, for very high hourly rates, separately from the rest of the university, so status and rules are less of a problem?
a. You're implying your colleagues are incompetent.
b. You can't use that money for "science".
c. It puts in painful relief that the software guys are much better paid than you are, implying that their work is more valuable than yours. But a lot of scientists put up with low pay because they believe their work is really valuable.
1. Benchling 2. Enable Medicine 3. Radix.bio
From the perspective of the university and the PI, they can just get a graduate student to do the work for 1/5 the cost.
That is what they do. However, some projects grow in scale where a grad student cannot handle it well, or if they try, it will be a detriment to their research. To give you an example, only one of the PhD students (non-CS engineering) in my group and the ones around me had taken data structures or algorithms - yet everyone's thesis was "computational" (i.e. numerical computation, etc). None of the advisors would appreciate their students going off and taking serious CS courses.
When a research team wants to go to the next level, they need to hire someone with better SW skills.
From my experience "science" does not appreciate RSEs, the compensation is bad, the freedom is bad, and you get all of the big company bureaucracy from the University.
It's just a bad environment and to top it off, even if you have a PhD many will see you as "less than."