The oceans are our frontiers.
We know more about deep space than we do about deep water.
Additionally, an underwater civilization would make Humanity more resilient to some kinds of extinction events, if you care about that.
An underwater civilization would be much less resilient to a nuclear attack originating from Earth, than an extra-terrestial civilization would be.
Submarines and nuclear tipped torpedoes can strike any point on the sea floor with almost no warning.
Indeed! If we can survive the wrath of The Deep Ones, we can survive anything!
The oceans are all owned by various parties. They're use is governed by treaties and governments aggressively militarize development there (see how fishing boats are tracked and monitored).
They're very much not a frontier.
That covers the surface.
There are places on Earth that we have never been to. The deep ocean is incredibly hard to get to.
Space actually is one. And there are no (sentient) existing inhabitants.
IMO our current civilization has no way to account for, nor price, that reality - yet.