(Leaving obligations aside, USCG takes its mission very seriously, and 14 USC §521 gives them incredibly expansive powers to do whatever they deem necessary to protect lives and property on the high seas and coastal waters. Playing stupid games on deep waters is not something that the Coasties like to see.)
I found this[0] but it appears to be ongoing. The state appears to again argue they have no duty to rescue anyone.
[0]https://www.courthousenews.com/lawsuit-over-coast-guard-call...
Whether this would apply to the instant case is arguable, but turns mostly on the definition of when the duty of care kicks in (I think there's a decently-strong case to be made that once they stopped the "vessel," they pretty much didn't have an option to let it go). It's worth noting that USCG doesn't have any statutory mandate to rescue anyone (their duties enumerated in 14 USC §102 say nothing about saving distressed mariners), so your fundamental point is correct: for example, if someone had called in "crazy guy in hamster wheel out at sea," USCG could likely decide "not my sea monkey, not my circus" and let it go; it's just that once they're on-site it's a more legally-fraught situation. (This is on top of the fact that the Coast Guard treats rescue as a core mandate, so no chance they weren't going to stop this guy from ending up Gulf Stream flotsam.)
Courts have repeatedly ruled that the government has no obligation to protect its citizens.[0,1]
If they didn't go after him and his family sued its a virtual certainty that they would lose.
[0]https://www.sun-sentinel.com/2018/12/17/cops-and-schools-had...
[1]https://prospect.org/justice/police-have-no-duty-to-protect-...