When there's a problem in my system, I want to bring up multiple views, manually annotate, highlight the new debug log statements I just added, mark events as good/bad from complicated rules, etc.
None of the current systems (except maybe Grafana+Loki) really do this, they focus on ahead-of-time ingestion rules and tags. I would love a desktop or web tool that can do that, working from a log file that I can easily extract from my log platform.
When there's a problem at work it's usually easy to see that something is wrong on a dashboard, but when I want to drill down into it I usually end up with ten tabs of different searches and Grafana dashboards that I'm trying to correlate between manually.
I haven't fully formed any ideas about how to tackle this since there are still fairly basic features missing in Logsuck that I need to work on first but it's definitely something I'll be thinking more about in the future!
Maybe I can add yours a JSON view and call it done :)
I added an issue about it: https://github.com/JackBister/logsuck/issues/7 - if you want to chip in with any comments or even help out with implementing it it'd be much welcomed!
Maybe you can combine https://vector.dev that is actually a good shipper and transformation of logs and focus in the visualization and integration? Anyway I hope yours get very good. Will be nice if the whole setup is just a simple go exe and that is all!
Wasn't really happy with any of the existing solutions and ability to collect and export both metrics and logs is just brilliant.
Datadog is really great for this but can be expensive
I haven't had a chance to try Datadog yet but it looks great, the one complaint I've had with Splunk (or at least the way we use it at work) is that it's still "Just logs" and sometimes you need more than that to investigate issues while Datadog seems to cover more areas of monitoring in one package.
I haven't really tested what the limits are yet so unfortunately I can't put a concrete number on it, but my vision is that Logsuck should at the very least work for a small-medium sized company with "normal" logging needs.
The current state is that it's got quite a few bugs and is lacking many features you would expect from an ELK/Splunk alternative but the basic concept is working. If what you need is a GUI for grepping across logs spread out on multiple servers it can be used for that today, but it's not doing much beyond that yet.
I do have a list of known issues on a private Trello board, I should probably move those to GitHub so that they are at least visible to people, I'll take care of that right away.
There is some info in the readme on how to get started, if you want to try it out and need any help beyond that let me know!
The idea is to have a free Splunk alternative which you can set up with just one binary. I use Splunk at work and love it, but it just doesn't seem like a product for solo developers (I can't even find a pricing page on splunk.com), and the primary free alternative, the ELK stack, seems a bit complicated to set up.
I am sure that I'll never be competitive with Splunk or Elastic in terms of features or scalability but I'm trying to build something that is at least useful for my own projects.
I built it in Go and use SQLite with the FTS (https://sqlite.org/fts3.html) extension to store the log events in a way where they can be searched quickly.