How We Built Uber’s Highest Query per Second Service Using Go

Re:How We Built Uber’s Highest Query per Second Service Usin... by bsaul on Tuesday March 01, 2016 @04:25PM (#11206200)

really interesting to see that go channels weren't used due to performance concerns. channels is the often advocated tech for concurency in this language so having to fall back to mutexes seems like a downpoint. I would have loved to see a benchmark in the article.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by pcwalton on Tuesday March 01, 2016 @04:29PM (#11206233)

This is very common. Sharing memory is extremely important whenever you're doing anything related to CPU parallelism/concurrency; often if you don't use it you end up worse than the sequential algorithm.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by haberman on Tuesday March 01, 2016 @04:35PM (#11206270)

It's surprising to hear you say this. I thought a design goal of Rust was highly discouraging shared-state concurrency. Has this design philosophy evolved somewhat over time?

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by pcwalton on Tuesday March 01, 2016 @05:16PM (#11206560)

> Has this design philosophy evolved somewhat over time?

Yes, it has. Post-1.0 Rust's design philosophy is about taming shared state concurrency: that is, disallowing data races statically, making sure you take locks or use atomics where possible, and having a robust set of generic concurrent data structures ready for use so you don't have to implement your own.

This article, for example, rightly observes that atomics are difficult to get right, making it not scalable to have every program implement concurrent data structures from scratch. The benefit of having a rich set of generic concurrent data structures readily available is that it mitigates this problem.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by haberman on Tuesday March 01, 2016 @05:52PM (#11206778)

Makes sense! It's definitely a compelling story to offer the same abstractions that exist in other languages, but with static guarantees of correctness. A mutex that you can't mess up, that's a great thing.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by doomrobo on Tuesday March 01, 2016 @05:15PM (#11206555)

Not at all. Rust gives programmers the ability to use shared state or channels at their discretion. The traits Send and Sync were made specifically for these use cases. Rust just guarantees that you can't use these primitives in an unsafe way.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by tyoverby on Tuesday March 01, 2016 @04:44PM (#11206338)

Rust has very good support for shared-state concurrency. In fact, the borrow-checker is used to enforce invariants about locks and mutices that make shared-mutable-state a compile-time error.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by paulsutter on Tuesday March 01, 2016 @04:44PM (#11206340)

I wrote an HLS video proxy as my first go project last summer using only channels and goroutines.

In retrospect it was clear that channels were very elegant for certain parts of the code, and in other areas mutexes would have been simpler.

After a bit of experience with go, you gain an intuitive sense for which is appropriate in each case.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by cdelsolar on Tuesday March 01, 2016 @11:51PM (#11208152)

The sync package's RWMutex is perfect for this problem. It will perform faster than channels and is much easier to reason about. Since it can support multiple simultaneous reads and one write the performance impact of a lock is minimal.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by kasey_junk on Tuesday March 01, 2016 @10:22PM (#11207905)

Channels are "just" mutexes as well. They are also not very fine grained because they have to deal with the general case. You can get much more precise writing your own locking code in Golang.

[ Parent | on HN ]

Re:How We Built Uber’s Highest Query per Second Service Usin... by aaronbee on Tuesday March 01, 2016 @04:35PM (#11206272)

When it comes to accessing shared data it's often much simpler and more efficient to use a lock to protect that data.

Channels are good in other scenarios.

[ Parent | on HN ]