Re:AWS X-Ray – Distributed Tracing System
by mmclean on Thursday December 01, 2016 @07:00PM
(#13084789)
Stackdriver Trace [1] is Google's external implementation of Dapper with additional analysis features. It's available for free, even for workloads not running on Google Cloud Platform.
(Disclosure: I work on it)
Re:AWS X-Ray – Distributed Tracing System
by tonyhb on Thursday December 01, 2016 @08:49PM
(#13085303)
This is awesome! Can Google update the docs so that the links for external zipkin integrations work? They currently link in circles and there's no info on how to integrate into OpenTracing/Zipkin
Re:AWS X-Ray – Distributed Tracing System
by mmclean on Friday December 02, 2016 @02:28PM
(#13091506)
Uh oh - are you referring to the docs on GitHub or the actual docs site (https://cloud.google.com/trace/docs/zipkin)?
Re:AWS X-Ray – Distributed Tracing System
by mmclean on Friday December 02, 2016 @03:02PM
(#13091816)
Nevermind, I think we found it. Fixing now.
Re:AWS X-Ray – Distributed Tracing System
by tonyhb on Friday December 02, 2016 @04:22PM
(#13092398)
This part: https://cloud.google.com/trace/docs/zipkin#configure_zipkin_...
It doesn't show zipkin configuration :)
Re:AWS X-Ray – Distributed Tracing System
by mmclean on Friday December 02, 2016 @08:13PM
(#13093641)
Yep, an issue with the anchor links that'll be updated soon. It's supposed to just link to a lower part of the page.
If you scroll down you'll see a top-level header also titled "Configure Zipkin tracers" that describes how to configure the Brave tracer. You don't really need to do anything special here - just point your Zipkin tracers at the Stackdriver Zipkin Collector rather than your existing one.
Re:AWS X-Ray – Distributed Tracing System
by giovannibonetti on Sunday December 04, 2016 @10:25AM
(#13100229)
Since you work on Stackdriver, could you explain to me how did the pricing change last week? Previously I thought $8/instance/month was expensive, but it seems like now there is a cheaper basic tier. What is a good use case for the basic tier and when will I need the premium?
Re:AWS X-Ray – Distributed Tracing System
by dgacmu on Thursday December 01, 2016 @04:21PM
(#13083623)
On the research front, going back prior to Dapper is x-trace: http://www.x-trace.net/wiki/doku.php
and their follow on work, Pivot Tracing: http://pivottracing.io/
(which got a best paper award at SOSP'15.)
Tracelytics was a startup I co-founded based on X-Trace, which ended up as AppNeta's Traceview and is now part of the Solarwinds monitoring cloud suite of products (joining Librato, Pingdom, Papertrail). So look out for more tracing from us as well.
Re:AWS X-Ray – Distributed Tracing System
by Roritharr on Thursday December 01, 2016 @01:55PM
(#13082279)
Those are the comments I come here for. I was reading this and dreaded the lock-in. Thanks for pointing out sane alternatives!
Re:AWS X-Ray – Distributed Tracing System
by tonyarkles on Thursday December 01, 2016 @06:12PM
(#13084513)
My masters thesis is X-trace derived, as well as a paper we published at ICPE a few years ago. I'm on mobile so don't have a link handy. You can probably guess my name from my username.
Re:AWS X-Ray – Distributed Tracing System
by SEJeff on Thursday December 01, 2016 @02:19PM
(#13082505)
Appdash[5] :)
Re:AWS X-Ray – Distributed Tracing System
by seanp2k2 on Thursday December 01, 2016 @03:41PM
(#13083314)
Somewhat related: https://github.com/Netflix/vizceral
Re:AWS X-Ray – Distributed Tracing System
by roncohen on Thursday December 01, 2016 @07:15PM
(#13084895)
And if you're curious about how this sort of thing could look on the client/browser side of things, check out https://opbeat.com/react
(I work on it)
Re:AWS X-Ray – Distributed Tracing System
by martin_ on Thursday December 01, 2016 @08:44PM
(#13085275)
Looks awesome! When're you opening up the invites?
Re:AWS X-Ray – Distributed Tracing System
by roncohen on Friday December 02, 2016 @01:47AM
(#13086668)
Thanks!
If you shoot me an e-mail i can get you access: [email protected]
Re:AWS X-Ray – Distributed Tracing System
by AYBABTME on Friday December 02, 2016 @12:05AM
(#13086182)
Been using OpenTracing with LightStep and they have a really solid product, and a super well thought out API design in OpenTracing. Highly recommend checking them out, especially since OpenTracing saves you from vendor lock-in and works using AppDash/Zipkin/etc.
Re:AWS X-Ray – Distributed Tracing System
by dangets on Thursday December 01, 2016 @05:57PM
(#13084384)
We've been instrumenting Zipkin in our environment over the last couple of months and it has been excellent to work with. While there are a couple of snags here and there, it is _amazing_ at how quickly it can bring bottlenecks and bad network calls to light.
Re:AWS X-Ray – Distributed Tracing System
by allengeorge on Thursday December 01, 2016 @07:44PM
(#13085034)
Are you guys using finagle?
I'm sure this is not new, but still useful to get the word out about the power of universal X-Request-Id:
Re:AWS X-Ray – Distributed Tracing System
by AYBABTME on Friday December 02, 2016 @12:11AM
(#13086206)
Request IDs and their variants are only giving you correlated events, they don't encode enough information to connect causality and ordering of events. Basically: with request IDs you get a unordered bag of events, with proper tracing (trace id/span id/parent id) you get a DAG of events.
In practice, the analysis a DAG allows you to perform are way more powerful. So if you're in the process of setting that request ID stuff up, consider doing the tracing way.
As a starting point, I would recommend reading Google's paper on their project "Dapper." [1] It's essentially the core of most distributed tracing systems. At least those I've encountered.
There's a lot of tooling out there that take their cues from Dapper. I've recently been looking into integrating OpenZipkin[2] with our systems. I see at as a more viable alternative (no tie in!) to Yet Another Propriety Thing in AWS (YAPTA). There's other as well, like AppDash.
Recently there has been a push towards an open-standard for the collection side called OpenTracing[3]. I came across it when investigating LightStep[4]. Ideally that means no vendor lock-in, which of course has lots of knock-on effects.
If you know of any, I'd love to be pointed in the direction of _different_ and not just divergent techniques.
[1]: http://static.googleusercontent.com/media/research.google.co...
[2]: http://zipkin.io
[3]: http://opentracing.io
[4]: http://lightstep.com - Impressive team behind this.