Given most things are frontended by API Gateway anyway this isn’t too big of a concern since there’s already an effective 30 second timeout there. For ETL like stuff you can use Stepfunctions too.

It's still sad, because then you need a lambda for just these one or two things that might run long (or maybe even forever) and now you have to mix lambda and e.g. ECS.

That's fair, if you do need a sort of long running connection for most Websocket tasks (there is the websocket/ApiGW hook but that's less useful), WebRTC, raw sockets etc there is no choice than to use ECS or something similar.