Have you tried using Algorithmia for this? We get around a lot of these limitations that Lambda has (you can also call us from lambda and we can host most of your DL models https://algorithmia.com/developers/algorithm-development/mod...).

Disclaimer: work @ algorithmia