Yes, but it is still running an OS (rather than, for instance, an RTOS) there’s more that can go wrong, and bugs may require updates, which require network access.

That said, I’ve got many friends who do exactly as you have said and never connect it. IIRC, there is some talk about including 5G modems in near term models to further encourage connection. The data is valuable enough to merit the inclusion.