How We Knew It Was Time to Leave the Cloud

Re:How We Knew It Was Time to Leave the Cloud by x0x0 on Saturday November 12, 2016 @04:35PM (#12940341)

Also -- if you look at the infra updated from the linked article, they mention something about 3M updates/hour to a pg table ([1], slide 9) triggering continuous vacuums. This feels like using a db table as a queue which is not going to be fun at moderate to high loads.

[1] https://about.gitlab.com/2016/09/26/infrastructure-update/

[ Parent | on HN ]

Re:How We Knew It Was Time to Leave the Cloud by pcarranza on Sunday November 13, 2016 @04:01PM (#12945644)

It's not updates, it's just querying, updates are not that bad, the main issue there was that the CI runner was keeping a lock in the database while going for the filesystem to get the commit, this generated a lot of contention.

Still this is something we need to fix in our CI implementation because, as you say, databases are not good queueing systems.

[ Parent | on HN ]

Re:How We Knew It Was Time to Leave the Cloud by sytse on Sunday November 13, 2016 @06:01AM (#12942791)

For sure there is much we can improve to reduce DB updates, but we do use Redis for queues.

[ Parent | on HN ]