this is what I'm curious about, serverless automatically makes me think of cold startup time.

Some poking around suggests that currently they scale to a single copy, so for sporadic workloads latency will be good (relative to normal latency for warm code).

Based on the roadmap in the repo[0] (under "In Design").

Disclosure: I also work on a FaaS project, Project Riff.

[0] https://github.com/nuclio/nuclio/blob/master/ROADMAP.md

nuclio auto-scale today (use k8s HPA), the roadmap item is to scale-down to zero (today holds one instance when idle and will soon be able to shut it down when idle).

nuclio latency is as low as 30 microsec in Py & Go vs 10s of milisec in Lambda

Warm vs cold start is not a very meaningful comparison, though. Lambda typically scales from zero. Any warm system is going to greatly outperform a cold one. Java on Lambda can vary from nanoseconds to seconds, depending on whether you hit a warm copy or a cold start.

Disclosure: I work on a different FaaS, Project Riff.

I benchmarked all the frameworks in cold & warm, Warm is also very slow, no concurrency etc'. how many events/sec can u do with a warm Riff instance e.g. in Go & Py? whats the latency?

Let me put it another way: are you saying you have no cold-start penalty?

Edit: and to answer your question -- no, I have not personally benchmarked it. Can you point me to your suite? I've been thinking it would be useful to work on a benchmark that different implementations can share to catch regressions (and earn bragging rights, of course).

Great idea, we suggested to create an open benchmark at CNCF, a bunch of members opposed, we can come with a joint proposal. You can use our test tool https://github.com/v3io/http_blaster, it can generate a workload of > 600K http req/sec with one client and is programable.

Do you have a link to the benchmarking discussion? I'd be interested to see what the objections were.

I raised the topic today in a team meeting and I'll probably get to tinker over the next few weeks. I'm interested in exploring a few different axes of stress. RPS is one, but also things like "how does it behave with ten thousand functions installed?", "how does it behave with traffic yo-yoing?" etc.