News for hackers. Stuff that matters.

Show HN: Wild Moose – Autonomous agent for production debugging

https://www.wildmoose.ai/

Hi Hacker News!

We launched an autonomous agent that helps debug production issues, and we’re curious to get your feedback.

Try our tool with your Datadog’s logs and metrics >> https://app.wildmoose.ai/slack/install

Setup demo >> https://tinyurl.com/mvaj8emf

Today's GenAI devtools, such as Copilot, are limited: they are great for writing code, but we all know that programming is only 20% coding, and 80% debugging.

So how can GenAI be used for debugging? As opposed to code completion or test automation, production debugging is not about generating text. Debugging is mostly about root-cause analysis. We realized two things:

1) Generative AI is drastically changing the way we work with data, thanks to its ability to not only generate queries, but also run code and analyze unstructured data. This enables building better data-exploration experiences with far more intuitive interfaces.

2) RCA is all about exploring different types of data. When you don’t know why a transaction was dropped or which customers are affected – you explore metrics, logs, your code, other people’s code, old slack messages, and whatnot, to figure out what’s broken.

Putting those two together, we built an autonomous agent that helps debug production issues. Our LLM-moose connects to your logs, metrics, and other observability data, and lets you extract and analyze them by conversing with it. Once it gets a task, it will start reasoning, invoking APIs, and running code, until it comes back with an answer.

For example, when requested to “show me IDs of transactions that took over 1 minute today”, the moose will fetch those transactions from Datadog for you. You might then ask it if long-running transactions correlate with a metric such as DB CPU load. The moose will fetch the metric values, visualize them on a graph alongside the long transaction frequency, and give you the answer.

Our moose both runs code and invokes API calls; the interplay between these is nontrivial to design and a fertile ground for innovation. There are “textbook” solutions to let agents write and run code (open sourced by, for example, Open Interpreter), and also to invoke tools/APIs (for example, Gorilla). But doing both together is harder, and yet required. You’re welcome to share your thoughts on this!

If you want to see other integrations, or have ideas for features, or you’ve spotted behaviors of the moose that seem off - we’d love to hear.

Hit us up in the comment or chat with us here >> https://calendly.com/wildmoose-yasmin/30-min

( 19 comments | 10 points | on HN )
Re:Show HN: Wild Moose – Autonomous agent for production deb... by jadbox on Tuesday October 03, 2023 @05:14PM (#37757663)
It sounds neat. I'd need to see how productive it is for a real world production environment to figure out the value. The splash page design though is an absolute delight for me. Scrolling feels slow though.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Tuesday October 03, 2023 @05:32PM (#37757855)
Thanks, we'll fix the scrolling issue! Do you have any examples for recent prod issues you faced? Would be interesting to see how we could help speed up the investigation there.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by pico-games on Tuesday October 03, 2023 @06:02PM (#37758210)
Being able to go through massive amounts of production logs quickly is super interesting - how can we pinpoint and trace requests quickly going through multiple microservices?
Re:Show HN: Wild Moose – Autonomous agent for production deb... by roeischuster on Tuesday October 03, 2023 @06:20PM (#37758417)
Roei the CTO here :) During onboarding we preliminarily index some data about your microservice architecture so we have some "common language" with your Datadog instance. Couple that with run-of-the-mill distributed tracing deployed at the user's end, and our moose can start querying/reasoning about multi-service transactions, and/or at the finer granularity of individual requests and log lines.

Does this answer your question?

Re:Show HN: Wild Moose – Autonomous agent for production deb... by infixed on Tuesday October 03, 2023 @06:34PM (#37758537)
One of the issues with AI products is that they often require a good amount of input to use. It seems to me that a great AI product would save me time and do things for me without being asked to.

For example, at our company we have a quite a few of alerts set up. Datadog also automatically detects anomalies. It would be neat if this (or something else) could automatically do an initial triage without being prompted and give me a free headstart on issues that come in.

Otherwise, it feels like it's "work" to learn how to use the product, which seems to miss the promise of AI (doing things for us!).

Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Tuesday October 03, 2023 @06:47PM (#37758638)
100%. We already have some of those more-automated features, e.g. giving you context about recent changes that might be related to an alert, like deployments that might be related, feature flags, etc. But def interesting to do more around triage. Are you using anything today for triage or is it all manual?
Re:Show HN: Wild Moose – Autonomous agent for production deb... by morsela on Tuesday October 03, 2023 @06:37PM (#37758552)
Pretty wild. I can see these kinds of ai-based tools become used more often as they connect to more systems.

With that said, in my experience, usually, coming up with a good AI question can sometimes be harder than the act of looking for the data itself, i.e. asking "what were the last 3 errors yesterday" is not something I have ever done.

Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Tuesday October 03, 2023 @09:49PM (#37759922)
Those are good points, but like you said, the more systems connected the deeper the value the tool can bring. It goes beyond a better interface for querying logs and metrics: since our agent can run code, it can give you the bottom line for questions you have based on information aggregated from multiple data sources. You don't need to think of "AI question", rather just ask directly the thing you want to know, and our agent will query multiple sources of data and analyze those to give direct answers. For example, if you're trying to understand who are the users impacted by a certain issue - you can just ask exactly that.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by roei on Tuesday October 03, 2023 @09:56PM (#37759968)
True, there is so much to say for and against AI-based interfaces for working with data. But in reality, following the advent of products such as code interpreter, we're actually seeing users beginning to expect their data to be accessible not just via baroque query languages and multi-hop UXs. Finally, as Yasmin noted above, "Last 3 errors yesterday" is a bit diminutive :) you might ask where are transactions dropped, correlate errors with metrics, create dashboards on the fly, ...

Please feel very free to continue this discussion here -- it's an important one for us!

Re:Show HN: Wild Moose – Autonomous agent for production deb... by ozfive on Tuesday October 03, 2023 @07:38PM (#37759020)
Clicking links in your (wildmoose.ai) stack menu on the site doesn't close the menu on Android Chrome. The link location loads underneath though. Just a friendly heads up.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Tuesday October 03, 2023 @09:21PM (#37759771)
Thanks so much!
Re:Show HN: Wild Moose – Autonomous agent for production deb... by denvrede on Wednesday October 04, 2023 @06:10AM (#37763047)
Support for the Grafana stack (especially Loki & Tempo) would be great.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Wednesday October 04, 2023 @07:14AM (#37763510)
Great, thanks for the input! Do you mind sharing how many eng. are in your company? We’re trying to get a better sense of what tools are used per r&d size so we’d be able to create solutions that fit the needs of each :)
Re:Show HN: Wild Moose – Autonomous agent for production deb... by denvrede on Thursday October 05, 2023 @10:48AM (#37779310)
It's around ~15 SWEs
Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Thursday October 05, 2023 @04:56PM (#37784186)
Got it, thanks, that's helpful to know.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by mdaniel on Wednesday October 04, 2023 @02:02PM (#37769148)
please don't use link shorteners, they're a 404 waiting to happen and we're all professionals here and not scared of URLs with UUIDs in them: https://www.loom.com/share/9a4adc39806742c48d14cdd39da6e560?...
Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Wednesday October 04, 2023 @02:30PM (#37769553)
Fixed. Thanks for the catch!
Re:Show HN: Wild Moose – Autonomous agent for production deb... by jbmsf on Wednesday October 04, 2023 @07:07PM (#37772967)
Neat idea. Looking forward to support for more integrations.
Re:Show HN: Wild Moose – Autonomous agent for production deb... by yasmind on Wednesday October 04, 2023 @10:30PM (#37774255)
Thanks! Let us know if you have any specific ones in mind that would be helpful for you.