News for hackers. Stuff that matters.
Re:OpenTelemetry by candiddevmike on Wednesday September 23, 2020 @06:02PM (#24572638)
Here be dragons. I really want to use this project, but the spec and libraries keep changing. I would not adopt this yet, too much is in flux.

As an aside, I think this project is suffering from analysis paralysis. Ship a stable 1.0, iterate towards a 2.0 eventually based on user feedback. You aren't going to release a perfect 1.0.

Re:OpenTelemetry by jrockway on Wednesday September 23, 2020 @06:27PM (#24572865)
I agree. They were very quick to update their website to say "we're making a new thing", but the new thing isn't ready.

I used the metric implementation in Go (as someone with extensive Prometheus experience), and was shocked at how they managed to write so much code to do so little. You will be amazed at how many levels of indirection there are, with a little assumption hard-coded at every single point of indirection. (The net result being, in my case, that I couldn't actually create a gauge in Prometheus.)

I just use Jaeger and Prometheus together and they've never treated me wrong. I'm watching otel closely, but I wouldn't suggest that you use it quite yet, at least not for Go.

Re:OpenTelemetry by tignaj on Wednesday September 23, 2020 @08:35PM (#24574029)
Disclaimer: I work on OpenTelemetry specification and Collector.

I don't contribute to Go implementation but I encourage you to submit the feedback to https://github.com/open-telemetry/opentelemetry-go or https://gitter.im/open-telemetry/opentelemetry-go

You can also attend Go SIG meetings and provide the feedback directly. SIG meetings are open for attendance: https://github.com/open-telemetry/community#special-interest...

I am sure the Maintainers of Go implementation will appreciate your feedback.

Re:OpenTelemetry by candiddevmike on Wednesday September 23, 2020 @06:50PM (#24573153)
Which Jaeger go library are you using? https://github.com/jaegertracing/jaeger-client-go?

For prometheus, I was using the official Go client, but I found the victoria metrics one to be a lot simpler and lighter. My go.mod was a lot smaller after swapping.

Re:OpenTelemetry by jrockway on Wednesday September 23, 2020 @07:14PM (#24573365)
I use the official ones for both, but haven't even considered looking for others.

Jaeger is a little annoying in that what configuration the library accepts depends on the language. For example, you have to modify your code to emit B3 (zipkin-compatible) traces, which is annoying. Leads to PRs like this: https://github.com/grafana/grafana/pull/17009

Re:OpenTelemetry by tignaj on Wednesday September 23, 2020 @08:31PM (#24573997)
Disclaimer: I work on OpenTelemetry specification and Collector.

It is true that the spec and libraries have been changing. This is to be expected since we haven't yet made a stable release. We are aiming for 1.0 GA release this year. The current plan is to freeze trace spec this week, followed by metric spec freeze in 3 weeks, followed by the release of several language implementations in 4 weeks after the spec freeze.

Like everyone else I wish we could release earlier, but the reality is that making dozens of companies agree on a standard takes time. I am very excited that we are approaching the finish line.

Re:OpenTelemetry by tonyhb on Wednesday September 23, 2020 @09:07PM (#24574266)
But it's been around for 5 years. First as opentracing (actually thanks Ben and LightStep!), then google decided to compete with opencensus (no thanks google), and now we have opentelemetry.

It's really, really, really annoying. After 5 years you might expect things to have calmed down. It was great at the start - like, actually. Even when baggage was rusty it had a direction and was usable. It probably still is, but the churn is crazy.

You say making dozens of companies agree is difficult, and yeah - it is - but thats because of other companies throwing out specs trying to compete with Lighttstep.

Re:OpenTelemetry by harpratap on Wednesday September 23, 2020 @09:58PM (#24574554)
This is not in competition to lightstep or any one vendor in particular. It is to converge all open source & proprietary solutions to follow a common spec and allow them to build on top of a common base.

OpenTracing dealt only with Tracing, it didn't care about logs or metrics. OpenCensus did all 3. This is combining the best of OpenTracing (which is Tracing) with idea that OpenCensus brought - just one library to collect and ship all the observability data that your application is supposed to emit.

Re:OpenTelemetry by oaiey on Thursday September 24, 2020 @03:30AM (#24576224)
And like always when vendors / standards are built: It takes a while. Not surprised here.
Re:OpenTelemetry by tonyhb on Thursday September 24, 2020 @01:21AM (#24575492)
On the fundamentals you agree with me: the goalposts shifted from telemtry to logging.

IMO they're different things. One deals with call graphs, timings, and associated baggage. The other is logs. Should we hold off on the spec for another 5 years and add datadog/prometheus metrics too?

I dont think that one spec should be all encompassing. Opentracing is great for tracing. Logs can be baggage. Memory use can be baggage. The temperature of the CPU and the current disk space is another platform altogether.

It's like saying everything in the world should run webassembly, and no other ISA matters. It's an idea! But, certainly a distraction from the benefits. Why not let tracing deal with tracing before messing with the 1.0?

Re:OpenTelemetry by harpratap on Thursday September 24, 2020 @02:55AM (#24576023)
> the goalposts shifted from telemtry to logging

Telemetry has always included logging as one of it's components.

> Should we hold off on the spec for another 5 years and add datadog/prometheus metrics too?

I think you should look at OTel once again. OTel already includes metrics and supports exporting to Prometheus.

And I don't want to reply to the rest of your comments because I don't feel you understand this problem space enough before making such remarks on it.

Re:OpenTelemetry by tonyhb on Thursday September 24, 2020 @03:03AM (#24576079)
Fair. I worked with Ben to integrate opentracing into docker before Kube won, and at that point it was tracing only. Tracing as in spans, annotations, and baggage spanning services. So, it has changed from call graphs/timing etc to telemetry in the broad sense.

We're talking about churn here, not my knowledge, and I think considering how you say I don't understand it we can both agree it has churned into something else. As OP said, there is a lot of churn. Too much for your common system to keep up with in the (pun intended) span of a couple years. 3 rebrands in fewer years isn't promising.

Re:OpenTelemetry by jcchavezs on Thursday September 24, 2020 @07:19AM (#24577455)
Unfortunately this is what happens when you marketing promises instead of libraries. If CNCF wasn't that eager to market OTel over Opencensus this spec would have evolved naturally but you know, this is business work.