The best and most memorable advice I've seen on this, unfortunately more memorable than the name of the author:

Entomologists write "bees carry pollen". Botanists write "pollen is carried by bees".

In case it isn’t obvious, the lesson here is that passive voice can be used to place the important thing first in a sentence.

Whether that is for the best, however, is another matter. Some topics would be much clearer if the authors didn't ardently adhere to maintaining a passive voice.

And this makes contemporary scientific writing hellish to read

Or should I say contemporary scientific writing is made hellish to read by this…

If we refer to scientific writing as journal articles or conferences, it is just a matter of tradition and a certain gusto for science as an "objective" enterprise. Years ago, during my PhD, one of the professors working in my department referred to a colleague as a "research monk." As I saw it, he was just a person who didn't take care of his hair.

"Linear regression models were fit by first checking the normality of residuals" for some old-timers sounds better than "we fit linear regression by first checking the normality of residuals." But what is the difference between the two sentences? In the first case, it may sound like you, me and someone else's cousin would have gotten the same results; in the second, that we did those analyses and got those results. If others were to perform the analyses, who knows what would happen. But it is just an impression, isn't it?

I would argue it depends on what the paragraph is about. If the paragraph is about the regression models, I would argue every sentence's subject should be "regression models" using passive voice. If the paragraph discusses how the research was conducted, then every sentence's subject should be "we" with active voice.

The sentence is about fitting regression models, not what regression models are, which implies a choice was made. Therefore, the active voice should be preferred, even if the sentence is unambiguous in either case.

I prefer the active voice to the passive voice, although, especially in solo papers, repeatedly writing "I did," "I carried out," "I tested," sounds a bit vain after a while.

As author and teacher of good writing Giuseppe Pontiggia said, to be a good writer you need to develop a good ear for words and sentences.

Sure. If the paragraph is discussing how the research and models were performed/fitted, then use "we" as the subject with active voice.

Passive voice: it's a tool[0]. And in some scenarios, more preferable to active (as you correctly mentioned). Not only does it depend on genre (e.g. scientific writing), but on the writer's intent and how they want the reader to interpret the text.

It boils down to who's the subject/agent of the story, what you are trying to convey, and what you are trying to "stress" or emphasize. You, as the writer, get to control 1) who's story it is and 2) what should be emphasized.

For example:

- I wrote this software program - This software program was written by me

It would be silly to say that sentence one is better because its written in the active voice. What we need is context. If we want to tell a story through the lens of the software program itself and emphasize it being written by me (i.e. "I"), then I would argue the second sentence does a better job. Passive wins. In contrast, if we want to write a story about the perspective from me (i.e. "I") and emphasize the software program (which is in the "stress" position), then then the first sentence takes the cake. Active wins.

[0]: https://georgegopen.net/wp-content/uploads/2019/12/Why-the-P...