Show HN: Email API benchmarks – Real-world performance data for email providers
https://knock.app/email-api-benchmarks
Hey HN, I'm Chris, CTO of Knock (https://knock.app). We built email.info because every developer we talk to asks the same question: which email provider should I use?The existing resources are mostly vibes — "Best of" listicles, anonymous Reddit threads, or LLM answers trained on that same content. Objective performance data is always missing.
Knock is a notification infrastructure platform, so we send billions of API requests to downstream email providers every year. We aggregated that data into a public dashboard covering 10 providers (SendGrid, Postmark, Resend, Mailgun, Amazon SES, Mandrill, Mailtrap, Mailersend, Sparkpost, and Mailjet) over a trailing 90-day window. What we measure:
- Response time — Time to receive a 200 from the provider, computed across p50/p90/p95/p99 using exact quantile calculations in ClickHouse. Includes connection pooling, network latency, and any automatic retries. - Error rate — Ratio of 5xx responses to total requests. Since Knock retries on most 5xx codes, a single request can produce multiple error entries if the backoff sequence runs. - Channel growth — Which providers are gaining or losing adoption on our platform. - Public incidents — We pull each provider's status page via RSS/API and overlay incidents on the performance data, so you can see how well observed degradation correlates with public acknowledgment. - Pricing comparisons — Modeled price curves with head-to-head comparisons at different volume tiers.
Some things that surprised us: SendGrid's median response time is 22ms at 500M+ messages, Postmark is 33ms, and there's a meaningful long tail gap between providers at p99. Volume bands indicate our confidence in each provider's data — low-volume providers are flagged accordingly.
We're already exploring deliverability and time-to-inbox metrics. Would love feedback from HN on what you'd want to see in a resource like this.
One thing I'd flag from my experience: don't just chase raw speed. Look at error handling and retry logic in these APIs. SFMC integrations often choke on transient failures, like rate limits or auth hiccups, so providers that bake in robust fallbacks save you from midnight alerts. Also, factor in regional performance—I've seen US-East endpoints fly while EU ones lag, which murders A/B tests across time zones.
If you're building something notification-heavy, I'd dig into the consistency of webhooks too; polling APIs feels prehistoric when you're trying to trigger real-time automations. This data helps narrow it down, but always run your own load tests with your payload sizes—email with heavy personalization or attachments behaves differently. Props for putting objective numbers out there; it's rare in this space.