GKE and Google Load Balancer has similar issues, changes to the loadbalancer/ingress take up to 10-15 minutes to propogate and if you miss a health check you'll be serving 500's until it magically balances itself out or you just nuke it out of frustration and wait the 10 minutes for it to configure with your ip again.
As far as I recall, this is all configured on the load balancer, rather than directly inside Fargate. Somehow makes it feel less like a "fully managed" solution, but rather something you have to still tinker quite a bit with.
(compared to Lambda, which you really don't have to worry about scaling at all)
EDIT: [0] indicates that the minimum you can set is 10 seconds (minimum 2 intervals of 5 seconds to consider it "healthy"), if I understand it correctly
[0] https://docs.aws.amazon.com/elasticloadbalancing/latest/appl...