I think a combination of technical limitation and then the sudden hit of certain sites. When porn hit the web it was still something of a niche. It went from being fairly socially acceptable in the early 20th Century to a bit of a stigma around the time of the web (before my time so only what I have been told). Of course the web changed all that turning an already successful industry into a huge one.
So the makers had to find a new niche (edit: I mean before the web it was a niche - or rather what I really mean is that it was nowhere near as accessible). Suicide girls is a huge success because of the personal touch (I would say that 90% of PC's I see have porn on them and a vast majority of them have at least a few SG's images). Another popular one was the "wild girls" type spring break porn (soft and hardcore) - pretty much for the same reason, they werre girls you could see on a beach somewhere all the time, not super-made up pornstars. Again the personal touch.
And finally the customer base is more discerning? or perhaps more demanding :) Im guessing a lot of people prefer a personal touch to their porn to help get over that social barrier.
All just educated guesswork :)
EDIT: anyone happy to admit to being a regular (pay-for) porn consumer (I dont think HN will be judgemental in the way society seems to be!) and throw some light on these ideas from that side of things?
Here's my question back to you... why did the industry wait until recent years to have a sudden surge in personal approach porn? Was it due to a technical barrier? Or the increase familiarity with personalized content on the internet? Any thoughts?