A critical part to fixing "the problem" is both creating vehicles for new kinds of companies to access government and reduce the overall risk of government officials purchasing from those companies.
"Nobody ever got fired for buying IBM / Accenture / Whatever" applies in government as well, but the appetite for risk can be even lower than in the private sector, with higher costs of entry.
Top-level butt-coverage from the White House / OSTP and another agency demonstrating that it is possible and creating the contract vehicles to get more agile / lean / innovative businesses in the door goes a long way to solving this problem.
Take a long existing systematic problem which produces 100 bad effects a day (people getting double billed, database errors, what have you). Someone thinks of an improvement to reduce this to 10 bad effects a day instead. A major improvement. But they don't do it.
Why? Because anyone who approves of the new system is now the person responsible for the 10 errors a day. It goes from 100 errors per day because that is just how the system works to 10 errors a day because of Bob two cubes over. Bob ends up being blamed for the 10 errors far more than being congratulated for the 90 that no longer occur.
In my eyes, it's entirely possible the USDS is and ca be successful because it's new. I think it's (unfortunately) entirely possible that in 10-20 years they will suffer from the same bloat and inertia problems large companies and existing government agencies do. I'm not sure how to combat that, but having a way to revitalize large companies or government agencies without losing too much of the existing internal knowledge and expertise would go a long way towards helping combat this, but I'm not sure what that is. There are probably numerous successful companies that have done this that I'm entirely unaware of, so maybe there is a good amount of information on how to combat this...