I do know that the same index on a physical machine can process 200 million items in ~100ms just to give an idea. On lambda it was more like 100,000.
Keep in mind I stopped looking at that point. If someone is able to prove otherwise id be happy to switch over to this, but then again I don't have a real need currently.
When running locally on your physical machine, is your code only utilizing a single CPU core or does it make use of multiple cores? If it's the latter one, increasing the memory for the AWS Lambda function even further should improve performance as well.
Also if you hit a performance plateau with a memory size of less than 1769MB for your AWS Lambda functions, you could be bottlenecked by something else. I can imagine details like memory bandwidth playing a role there as well.
[1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-f...
My guess based on my tests is that memory access and cpu are the limiting factors in lambda for this but that because it encourages scale out less of an issue.
1. Put it in a layer instead. A bit more space and easier to maintain.
2. Put it in EFS and cache to /tmp. You get almost a half gig of temp storage which persists across executions.