My impression was ->
Software developers main focus is developing software so they spend a HUGE amount of time developing software, maintaining software, noticing patterns and bugs and pitfalls in software and thus they get pretty decent at writing software.
Data scientists main focus is developing models, so they spend a HUGE amount of time developing models, tweaking models, finding data, and cleaning data, and write basic software to achieve some of those goals.
I wouldn't expect Albert Einstein, Mozart, or Beyoncé to be some fantastic software developer just cause they're smart individuals. I'd expect people who spend a lot of time writing software to generally be the ones who write decent software.
I'd argue it's more about startup time, or whatever the technical term is for the time between when you first start reading code and when you've built enough of the model in your working memory to be able to touch it without breaking everything.
That heroic developer doesn't need to concern themselves with other people's experiences, and they can just write whatever weird idioms suit their brain.
That said, having used a lot of research oriented software packages over the years, researchers typically do not produce high quality software. Their objective is to produce papers, not software that other people can use. If someone else can use it too, then great, but that's not the primary motivation.
Most of the time you're just testing the viability of things and just want to fail fast if they don't work. Getting too attached to some pipeline or abstractions is usually a bad idea.
I don't think this excuses bad code, but I kinda get it when it happens, since you can't just use best practices from the beginning like in software development
I suspect it will be a downvotably unpopular opinion, but a typical researcher has a significantly larger cognitive capacity than a typical software developer. Consequently, a piece of code of certain complexity will look simpler, and be easier to manipulate for a researcher than for an average software developer.