Just serve and not transcode? Because it sounds like the Amazon Prime microservice was doing some heavy processing of uploaded video - breaking it into frames and applying some ML algorithms to them, and also the same with the audio. Sounds like even if you can fit in under the 10 min lamda execution timeout (assuming internal teams are limited to the same), that it's not great as a generic job queue handler.

Writing every frame to S3 and reading them back in other steps was probably a major factor.

There is not much of an option with step functions, outputs/inputs can be max 256KB so especially if you go into or out of a map/parallel state you can‘t do much more than pass references to the data and not the data itself.

Sure but I guess I mean the nature of that task probably made them the wrong architectural choice.

>I wonder what they're doing differently

Largest thing that stands out to me is simply scale

Serverless isn't "cheap". It's rightsized. If you aren't serving 24/7, it makes sense to move to an on-demand model. However if you compare the unit of compute pricing, Lambda is comparable to other services, and likely more expensive than buying compute wholesale. You can just buy it in smaller portions.