You've increased the places where things can go wrong (in third party client apps) while not having any control over their quality. We'd hear "oh, IRS opened things up!", then deal with a dozen half-baked clients, and conclude "IRS sucks at something so simple!".

Agreed - public API first is the wrong place to start.