Re:Ask HN: What's your serverless stack for AI/LLM apps in p...
by fazlerocks on Friday January 10, 2025 @04:09PM
(#42660266)
Thanks for the detailed response!
I actually tried fly.io briefly with Next.js apps and the deployment experience was smooth. Really interesting to hear you're using it for AI workloads too.
For fly.io with AI workloads: Are you using their Machines or Apps? I'm particularly curious about how you're handling cold starts for LLM tasks, since that was one thing I loved about fly.io for regular Next.js deployments - the cold starts were minimal.
I actually tried fly.io briefly with Next.js apps and the deployment experience was smooth. Really interesting to hear you're using it for AI workloads too.
For fly.io with AI workloads: Are you using their Machines or Apps? I'm particularly curious about how you're handling cold starts for LLM tasks, since that was one thing I loved about fly.io for regular Next.js deployments - the cold starts were minimal.