For example, we don't have a unified banking system in the EU. Each country has its own preferred method of payment. In the Netherlands they have iDeal. In Belgium Bancontact. Dankort in Denmark. The list goes on [1]. Credit cards are not that common in Europe and not everyone has one. To appeal to a wider audience, one must support local payment methods. This is just one example of where things get more complicated compared to doing business in the US.
[1] select the "Europe" tab: https://www.adyen.com/payment-methods
Basically all you need is the IBAN.
The currency will be converted to euro but you don't need to be within the eurozone, only within EU.
The example I gave was regarding accepting online payments as a business from inviduals and other businesses. For US busineses, accepting credit cards is usually enough. Large bills can be paid by wire transfer. In the EU, accepting online payments is not that straightforward. Just figuring out the right VAT rate is complicated to begin with.
See: https://ec.europa.eu/taxation_customs/business/vat/eu-vat-ru...
As for the supposed insane amount of regulation, the majority of international trade is regulated at the EU level so it's quite unified. Not to mention e-commerce which is completely regulated by the EU.
I am not saying that there is not a considerable amount of particularity that may be hindering small startups in fast expansion across Europe but I would find it quite hard to believe that it is much of a factor for the likes of Apple and Amazon.
Doing business in the EU is just a lot harder. Even though a lot of regulation is on an EU level, each country has their own rules and laws on top of that. Just invoicing your customers is a total mess. It's incredibly complicated. Then there is a ton of privacy and data protection regulation (GDPR for example). Germany for example has a dozen or so more rules on top of that. It's one of the reasons why Google Streetview is non-existent in Germany. I haven't even gotten to the the fact that each country has a different language. Although most people of younger generations do speak and understand English just fine.
I totally understand why companies avoid the EU as a result of this. It's a mess. Ironically, I do support a lot of these regulations.
On the plus side, that gap presents sort of a business opportunity but the downside is, it's not a particularly big one.