If the oceans can't sustain such a small population of whales, then the ocean ecology is itself a problem.

Bear in mind that before voracious industrial-scale harvesting, oceans were teeming with life.

This book says it all: https://www.jbmackinnon.com/the-once-and-future-world