These engines being tailored-made for a specific type of game, people wanting to do something else had to create their own engine, often in C++, sometimes in Python or Lua, reinventing the wheel most of the time. The barrier for entry being so high, game designers had to team with coders, but their end-goal not being always aligned (creating the game vs learning how game engines are coded) this lead to the demise of many projects.
With Unity, a general purpose game engine became available for a very low price (in particular compared to Flash). Many game designers saw it as a way to work on prototype without needing to recruit a team of coders. And for many coders, it was a way to try their hand at game design, while being able to push the engine further using C#.
And even then, CryEngine, Unreal and Source are all engines centered on a very specific kind of game (fps style). There was no serious general-purpose game engine before that, that devs actually used.
The very first Half-Life was already using one of the Quake engines. As was Hexen.
The first Deus Ex was using the Unreal engine.
That was all probably ten years before Unity.