News for hackers. Stuff that matters.

A comment on Container technologies at Coinbase: Why Kubernetes is not part of our stack. Parent comment

Re:Container technologies at Coinbase: Why Kubernetes is not... by dvt on Monday June 08, 2020 @05:51PM (#23461212)
The first iteration (I actually wasn't around for that) was trying to run a cron for every "data ingestion job" -- at some points, we were doing about 50k+ API requests daily (FB/Instagram/Twitter/etc.) and that was absolutely not tenable using k8s cronjobs.
[ Parent | on HN ]
Re:Container technologies at Coinbase: Why Kubernetes is not... by snuxoll on Monday June 08, 2020 @07:16PM (#23461841)
Why use cronjobs at all for this? This is a classic work queue problem.
[ Parent | on HN ]
Re:Container technologies at Coinbase: Why Kubernetes is not... by dvt on Monday June 08, 2020 @07:43PM (#23462046)
I wasn't there for this decision, but I assume cronjobs were being treated as "cloud functions" -- and to be fair, the k8s documentation kind of makes it seem that you could technically do that, but fails miserably if you try to do so in practice.
[ Parent | on HN ]
Re:Container technologies at Coinbase: Why Kubernetes is not... by dilyevsky on Tuesday June 09, 2020 @12:06AM (#23463537)
50k/day is less than 1 qps. This is nothing. This is either not the full story or your cluster was setup completely wrong
[ Parent | on HN ]
Re:Container technologies at Coinbase: Why Kubernetes is not... by regularfry on Tuesday June 09, 2020 @02:59AM (#23464390)
Depends how spiky the distribution was. 20k a second at 2pm? Gonna have a bad time.
[ Parent | on HN ]