> is a royal pain in the neck.
It's fun to see how different people put the same unpleasant experience into words in this thread. Thanks for adding your personal touch. Every time we hear something like that, we're re-assured that we're on the right track.
> Even if `opstrace create` spawns them easily, the problem is running/maintaining them
Yes. You're right. While we can be proud of our setup/installation process already, we know that there's so much more to it. We don't underestimate that. Maybe also see https://news.ycombinator.com/item?id=25998587, where I just commented on the robustness topic.
> However, we are happy for you to be a custodian of that data.
Great.
> We want storage to be someone else's problem.
I share that perspective. We, of course, are happy to let S3/GCS do the actual job.
> We're happy for you to use a cloud platform like AWS/GCP and charge us a 50% markup.
That's great to hear, and I hope you can be enthusiastic about the fact that our markup is _not_ going to be relative to storage volume. It's going to be independent of that.
> Transparency, predictability of costs, open source --> all excellent.
Thanks for sharing. That's incredibly motivating.
Keep an eye on us, and we'd love to hear from you!
By the way, we are requiring that any vendor we choose in this area support OpenTelemetry as we've already instrumented our apps with it. Lightstep, Datadog, and others already supporting.
And yeah OpenTelemetry is where lots of the industry is going and so are we. You can already do cool things with us and the collector but we plan on automating a lot and expand on that.
A little bit about that in this commment: https://news.ycombinator.com/item?id=25995424
As for scrubbing PII, they are now supporting the OpenTelemetry tracing API which does this as standard. For query endpoints you will see something like "QUERY Users where name=? email=?", i.e. masking with "?" chars as you only care about the keys since those what determine your indexing or lack thereof. (This is handled in the OpenTelemetry application library/plugin level.)
As an aside, PII scrubbing should be done that whether or not you own the cluster, because even if you own the cluster you generally don't want your support staff seeing PII esp. as organization grows larger.
On the cost topic, last week we published a blog post analyzing the cost of running Opstrace on AWS (https://opstrace.com/blog/pulling-cost-curtain-back). (In fact, feel free to do a local repro to confirm our results.) As mentioned elsewhere here on HN, we are incentivized to provide total transparency in terms of what you spend on your cloud infrastructure. We haven’t compared ourselves to everyone, but feel confident that letting our customers pay S3 directly is the best deal possible.
It might not fit your use case...but it might.
In a nutshell, running all these various components (Grafana, etc) is a royal pain in the neck. Even if `opstrace create` spawns them easily, the problem is running/maintaining them. We want someone to run these for us as a SaaS/PaaS and we're happy to pay them.
Re: your principles:
(1) The customer should always own their data --> we agree. However, we are happy for you to be a custodian of that data.
(2) We don’t want to be a storage vendor --> neither do we. We want storage to be someone else's problem. We're happy for you to use a cloud platform like AWS/GCP and charge us a 50% markup.
(3/4) Transparency, predictability of costs, open source --> all excellent.