News for hackers. Stuff that matters.
Re:Vector: A high-performance observability data pipeline by nextworddev on Sunday March 17, 2024 @04:34PM (#39737800)
Is this a Splunk alternative?
Re:Vector: A high-performance observability data pipeline by makmanalp on Sunday March 17, 2024 @04:55PM (#39737982)
It's more like a fluentd alternative, with a lot of improvements: much less sluggish throughput if you're doing anything nontrivial, nicer overall architecture and language, more 'batteries included' integrations ecosystem.

We're in the process of switching to this at work for some very high volume logs and I'm quite hopeful - other teams saw pretty decent improvements.

Re:Vector: A high-performance observability data pipeline by nextworddev on Sunday March 17, 2024 @05:14PM (#39738120)
It'd be awesome if a simple UI + hosting is provided if it's in your roadmap, I'd be interested in paying for that (as a startup segment)
Re:Vector: A high-performance observability data pipeline by loloquwowndueo on Sunday March 17, 2024 @06:18PM (#39738578)
I don’t think hosted (saas?) Vector makes much sense. It’s meant to sit next to your deployed workloads ingesting data and sending it elsewhere for storage/analysis whatever (and it does make sense for that target to be hosted - think datadog, betterstack, etc). So you’d deploy it as a sidecar in a pod, or as another container in your deployment or as a plain old executable in your server/vm or whatever.

Also a UI is relatively pointless since you only mess with the config occasionally and otherwise just leave Vector running doing its thing. Want to see how it’s doing? Have your Prometheus scrape Vector for its own metrics and set up alerts/analysis using Prometheus itself or Grafana.

Re:Vector: A high-performance observability data pipeline by atombender on Sunday March 17, 2024 @04:47PM (#39737906)
Not directly. Vector is a tool to build pipelines that receive, transform, and send data. But it doesn't index data to make it searchable. You can use it to ingest into Splunk, however.

Vector could be used to build something Splunk-like. For example, you can use it to ship logs into Kafka, then let it ingest that data into ClickHouse, and then use a frontend like Grafana to search logs using ClickHouse.

Re:Vector: A high-performance observability data pipeline by marco_z on Monday March 18, 2024 @04:27AM (#39741513)
> you could ship logs into Splork, then let it ingest that data into FlorbHut, and then use a frontend like Glorply to search logs

Seriously though, is there a single OSS product that does all of this ? Like, for a small multitenant app (i.e not "web-scale"), and that doesn't force one to get a degree in observability just to get stuff done.

Re:Vector: A high-performance observability data pipeline by ikut3hva on Sunday March 17, 2024 @04:45PM (#39737891)
No, it is an opensource of timber.io but now Timber was acquired by Datadog ;)