News for hackers. Stuff that matters.
Re:Building Real Time Analytics APIs at Scale by shrumm on Tuesday April 10, 2018 @05:30PM (#16805642)
Most of the discussion (rightly so) focused on DB optimization. The decision to build the API in Go was barely mentioned. I’m curious if you evaluated any other frameworks / languages or was Go just an automatic choice?
Re:Building Real Time Analytics APIs at Scale by sfg75 on Tuesday April 10, 2018 @06:24PM (#16806126)
Pretty much automatic. With the exception of our search engine which is in C++ (as performance is paramount there), Go is becoming our language of choice for most of our backend services. We found in Go a great balance in terms of productivity and performance.

After building the aggregation and ingestion services in Go, sticking with this language for the API sounded like a good idea as well since Go makes it trivial to build an http server and the logic of the API is simple enough that we didn’t see the need for any web framework.

Re:Building Real Time Analytics APIs at Scale by shrumm on Wednesday April 11, 2018 @12:56AM (#16808157)
Thanks! I asked because i’m looking at the same problem, though at a smaller scale than you guys.

I decided to build our personalization API using Python’s Flask, worked great at the start because it helped us move quickly adding new features. 6 months later, we have more clients and hence more traffic, the response times have gone up significantly. I ran a benchmark doing a simple db query which returns the result as JSON with the apache benchmark tool and found my Golang implementation to be in the order of 20-25x faster, compared to falcon which is meant to be a lot faster than Flask.

Decided to just go ahead and implement the most performance sensitive parts of the API in Go.