Not specifically physics engines but graphics engines use "hidden surface determination" to avoid rendering what's outside the camera view. Frustrum culling, occlusion culling, LOD optimization, space-partition based clipping, etc. This video does an amazing job explaining some of those. https://youtu.be/C8YtdC8mxTU (highly recommended channel too)