It's been long ago enough that I don't remember all the details, but my perspective was that it was naive and arrogant to assume we had a good handle on everything about DNA, especially then as it was all new, and there was evidence then that a lot of the noncoding DNA had function.
Dogmatic attitudes are surprisingly prevalent in academics. There's definitely a weird hubris accompanying published findings that goes something like peer reviewed -> established fact as if the point of academics isn't to understand things but to score points like on a test. Some skepticism about certain things seems warranted but other times there's really nothing but overly simplistic models that were really never evaluated to begin with. I wish there was more awareness of the topography of uncertainty in research findings.
https://en.m.wikipedia.org/wiki/Planck%27s_principle
“Science progresses one funeral at a time.”
Having a list of proteins (coding DNA) does nothing to tell an organism how to combine those proteins to form working systems.
It's like claiming that the parts list in an assembly manual is the only information of importance. And the step-by-step instructions are junk, because they don't describe a useful part in the box.
Did we think that the whole organism just self-assembles once it makes the tens of thousands of individual parts described in coding DNA?
"Junk DNA" was a terrible terrible phrase to begin with.
Disclaimer: I believe in a creator. (https://www.jw.org/en/bible-teachings/science/)
Imagine a big machine run by paper tape. On the tape are two possible holes. Punch one and feed it in, the machine makes a man. Punch the other, it makes a mouse.
Does the tape 'encode' the essence of 'human being'? (or even 'mouse'?) Of course not.
The machine is a vital part of the equation. Likewise, dna in a jar is just a molecule. It has to have a cell, evolved over a billion years, ready to make something of it.
So no self-assembly, no magic in the DNA.
What we don't know is how an organism with no intelligence (cell), could develop code (DNA), archive it, rehydrate it and then parse and copy it like someone with intelligence.
Without getting too deep into it, it's just not convincing anymore to appeal to vast amounts of time and random chance to do something which is fundamentally within the realms of engineering.
The fact is, we don't know how these systems could have created themselves or been created abiogenesis. There are wildly different theories, but it's far from a closed case.
I used to think the same thing as you because that's all I was taught to think, but perhaps if you looked at what synthetic chemists think about abiogenesis you may find it interesting / informative to see what they have to say about the challenges involved in it. An interview with a synthetic organic chemist on the topic: youtube.com/watch?v=r4sP1E1Jd_Y
An evolutionary process, on the other hand, doesn't fear removing random sections so their continued presence indicates the "junk" must (somehow) be important to the organism's fitness.
Irrespective of this, enhancer RNA transcription is very useful to track active regulatory regions and to perform QTL studies.
https://jacquesmattheij.com/junk-dna-no-way/
with the newly found insights. Still: No Way. Just not-yet-understood-DNA.
Many of these non-coding areas are actually enhancer elements. These elements produce enhancer RNA, previously thought to be noise, and regulate distant genes thanks to chromosome conformation. Chromosome conformation means DNA folds in complex 3D patterns.
The interplay between chromosome conformation, enhancer RNA, gene expression and disease variants is very complex and interesting [1,2].
I know some startups trying to regulate genes by targeting enhancer elements. The field is very promising, but a big roadblock seems to be to deliver compounds to the right cell type.
Junk DNA was always contentious because there was evidence of conservation in these areas. In other words, evolution was selecting against mutations in some junk DNA regions. My impression is that junk DNA fell quickly out of favor once the ENCODE Project started to publish large scale epigenetics data in the early 2010s.
[1] https://genomebiology.biomedcentral.com/articles/10.1186/s13...
[2] https://www.nature.com/articles/nature13138