Using GPT-3 for plain language incident root cause from logs

Re:Using GPT-3 for plain language incident root cause from logs by bbu on Tuesday January 12, 2021 @04:21PM (#25753898)

This is pretty cool! However, these two samples are very simple to solve. I'd love an "AI" to find root causes for problems that are not obvious. Just throw the whole log collection at it and let it solve all the issues. One can dream ;)

[ Parent | on HN ]

Re:Using GPT-3 for plain language incident root cause from logs by stochastimus on Tuesday January 12, 2021 @04:40PM (#25754207)

Yeah, my experience so far has been that if I just pile a bunch of logs in there, if it's not given salient lines, the language model tends to either rat-hole on some particular detail that's irrelevant, or else construct a non-factual narrative. But, when Zebrium did the picking of these lines autonomously and GPT-3 summarized them, we see a meaningful summary. Having said that, I would also like to get GPT-3 more and more savvy with larger and larger log-based prompts, and I'm hoping it's possible with some tweaks to the prompt and some finetuning. I'll keep the blog posted as we do more experiments.

[ Parent | on HN ]

Re:Using GPT-3 for plain language incident root cause from logs by a-dub on Tuesday January 12, 2021 @05:16PM (#25754700)

this is really cool!

[ Parent | on HN ]

Re:Using GPT-3 for plain language incident root cause from logs by m463 on Tuesday January 12, 2021 @04:52PM (#25754387)

GPT-3-stackoverflow

[ Parent | on HN ]

Re:Using GPT-3 for plain language incident root cause from logs by deeeeplearning on Tuesday January 12, 2021 @05:17PM (#25754719)

But seriously, was Stackoverflow part of the training data used to train GPT-3? Would definitely be an interesting fine tuning experiment

[ Parent | on HN ]

Re:Using GPT-3 for plain language incident root cause from logs by stochastimus on Tuesday January 12, 2021 @05:21PM (#25754769)

From what I've read, the answer is "yes", stackoverflow was crawled. EDIT: I looked and stackoverflow is included in the Common Crawl dataset, which is one of the datasets on which GPT-3 was trained. Having said that, it's not clear to me the degree of coverage of that domain guaranteed by that crawl... looks pretty comprehensive, though. http://index.commoncrawl.org/CC-MAIN-2020-24-index?url=*.sta...

[ Parent | on HN ]

Re:Using GPT-3 for plain language incident root cause from logs by sillysaurusx on Tuesday January 12, 2021 @07:27PM (#25756193)

The training data was so large that I don’t think a single epoch was used. I.e. it’s not a guarantee that any specific site is known by GPT-3 just because it’s in the data.

But, with the nature of random samples, it’s probably likely that it saw at least a little bit of SO.

[ Parent | on HN ]