If someone is directly targeting you, then yes it's an issue (but even so, it's less of an issue than using exactly the same email address for all of your accounts).

In a mass compromise like the Adobe one, it's highly unlikely that the hackers are going to go out of their way to attack people who use this method when there's millions of much easier targets already in their list.

Using this approach also makes it a lot easier to spot spam - if I get an email to "[email protected]" claiming to be from my bank, it's highly unlikely to be genuine. If it's coming to "[email protected]", there's at least a fair chance that it's real - I still treat it with a fair amount of caution, but as I've filtered out the obvious junk I can spend more time checking out these reasonably genuine-lookuing one. Using a random email like [email protected] would make this quick filtering a lot harder.