I'll admit I'm not familiar with Texas but that doesn't seem right. Here in colorado, you're never forced to sell your mineral rights and let people on your land, but if your neighbors sell their mineral rights, and an oil field that is under your property is being drawn from, you will receive royalty checks, even if you'd rather the oil stay in the ground. But, no one can make you build an oil well on your land.
You are correct; if you own the surface and mineral rights you are not obligated to allow for any surface use. The thing you describe where someone drills on neighboring property and you are tagged along is called Force Pooling and is a much more complex topic.
You are not guaranteed to be force pooled -- if your neighbor has enough land to lease a full unit, it's conceivable that resources may still flow from your minerals towards the productive well without being compensated. It's usually in the landowners best interest to negotiate in good faith with a lessee than to be force pooled.
If you own only the surface rights to your land and someone else owns the subsurface minerals or a lease to the subsurface minerals you can absolutely be obligated to allow surface access for development and production under reasonable and usual terms for such.
I can only imagine how upset an HOA would get if drilling operations were performed without an approved architectural modification request.
You don't get to make choices in a vacuum.
"Under split estate law, the surface owner must allow the mineral rights owner reasonable access. Protections for the surface owner vary; in some states an agreement is required that compensates them for the use of the land and reclaims the land after extraction is complete."
https://en.m.wikipedia.org/wiki/Fracking_in_the_United_State...