Anyway, even B5 had a plenty of "monster of the week" filler episodes as well. They were generally of much lower quality than the plot episodes.
I’d be really interested if they did an abridged version of DS9 in HD. Sadly it’s the fillers that are putting me off from rewatching it.
Battlestar Galactica was a great show but Unfortunately I ruined it a little for myself when I watched a “how it’s made” style show midway though the second to last series and they commented about some of the writing decisions they made. That’s when I realised they were just making it up as they went along as well (previously I thought the same as you in that they had a pre-defined arc). Battlestar Galactica did still have some amazing writing though, even if I was a little saddened by my realisation.
It wasn't Babylon 5 that sparked the trend, but in 2004 Lost, Battlestar Galactica and then later on a few following shows like Prison Break came along and were very successful with the long story arc.
The concept of the seasons-long story arc is now basically endemic in modern TV. I think it's a situation where the highs are higher, but the average is much worse across all TV.
The issue I think is that the writers can't experiment like they could in the past with the more monster-of-the-week type formula.
A bad idea in a modern show is baggage you have to deal with for the rest of the show. In old school TV, the end of the episode came around and erased all sins. It also forces the writers tell a complete story each episode.
Consider Star Trek: TNG. There are a whole pile of really terrible episodes of that show, but there are also amazing ones. I don't think it's possible for a show like Star Trek: Discovery to ever reach the heights of TNG given the baggage it has signed itself up to deal with.