Even if one accepts that existing trade will be accommodated through direct agreements, that only protects existing industries. What does it mean for future industries and how existing industries grow and change? When a new industry develops and wants to export, they are going to find they face trade barriers until such time as the EU and UK get around to negotiating a new (or modified) trade agreement. That's a huge disadvantage and will see a lot of investment, especially in tech companies go to the EU where previously it might have gone to the UK.