Re:Introducing Atlas: Netflix's Primary Telemetry Platform
by copperlight on Saturday December 13, 2014 @12:23AM
(#8744267)
Telemetry traffic is a small fraction of the the total traffic running through a region, partially due to the use of the Smile data format (binary JSON) for delivering metrics from the client to the Atlas backend.
When you give developers tools for creating and aggregating highly dimensional metrics, they tend to create lots of metrics so that they can answer interesting business questions about the use of their applications. We have some developers who have written code that produces up to 150,000 metrics per instance and the vast majority of these metrics are application-level. We typically see 3-5% of the metrics delivered from an instance are system-level performance metrics.
Re:Introducing Atlas: Netflix's Primary Telemetry Platform
by CrankyFool on Saturday December 13, 2014 @01:39AM
(#8744438)
And this, of course, doesn't account for cases where a minor developer error results in code that, say, creates a new metric for every source IP address from which we see a request. Dynamic metric names FTW.
Re:Introducing Atlas: Netflix's Primary Telemetry Platform
by lifeisstillgood on Saturday December 13, 2014 @06:56AM
(#8744990)
Thank you both for your insights
My client is lookin at these telemetry problems now, is there possibility of commercial high-level consultancy coming out of Netflix / colleagues ? Ping me on details in my profile if you can help?