They might not appear to have a moat, but they definitely have an advantage that they can keep for 5+ years. They also have a "copy of the internet" that no one else does.
Google may already be terminal
Google of all companies is sitting on exaflops of TPU capacity, they can retrain models larger than anyone else, faster than anyone else.
Loads of internal position papers read exactly like this. I've written docs that write like this.
1) algorithmic improvements beat processing capacity. One of the examples in the paper was that $100 in training compute budget was enough to get within 2% of Google Bard performance
2) data quality (even for "tiny" data sizes) beats dataset size in improvements
Hence both aspects of the moat of large companies (dataset size and available compute) matter less than you'd expect.
In contrast the advantage of "open source"/academics/"the market", the many warm bodies trying out new strategies, yields improvements at a rapid pace that Google, even with a huge team, cannot match.
Google has a good, two part strategy available.
First is to neutralize the threat: support the open source efforts covertly and/or overtly, as those efforts undercut the revenue of OpenAI and other potential competitors for the base technology. This is what they did with Android: at a minimum all they had to do was ensure that Apple did not become a monopoly, which they accomplished (the secondary effect was to cut the underbrush of any other competitors). A name for this is to “commoditize your competition”.
The second half is to use their internal experience with LLMs and their other ecosystem to use supercharge their strengths. Side inventions can either be expanded in the hope of a (desperately needed, unrelated to LLMs) new revenue stream or given away to starve potential new entrants.
The big guys have good experience in weaponizing open source. But google has become flabby and seemingly lost any culture of execution so I don’t know if they can pull it off. But the path is there.