This is very interesting but I fail to see an issue. You listen to all incoming log messages and only store != success. What am I missing (obviously)
I'm interested we well
Re:Remembering Bob Lee
by dumpster_fire on Thursday April 06, 2023 @04:29AM
(#35465777)
It's a system design question. High throughout on the scale of Google means you'd laser any log ingestion service to death.
He was probably looking for a distributed way to label logs and build an ingestion service that shards regionally with high availability, but still able to search and filter within an acceptable time for debugging purposes.
How can I learn about these topics? Junior self-taught dev, I read these things and kind of see what they're about, but I wouldn't know where to even start with this...
Re:Remembering Bob Lee
by shubhamkrm on Thursday April 06, 2023 @07:20AM
(#35466859)
Designing Data Intensive Applications (DDIA)[1] is a great introductory book for such things. It covers a broad number of system design topics, and you can then go into detail for stuff that interests you.
Other than that, several companies (such as Uber) operating on a large scale maintain engineering blogs.
You can practice the mindset by writing a toy solution for the problem, then listing the reasons why you consider your solution just a toy. This develops a fundamental skill of engineering, which isn't understanding whether a system will fail (it will always eventually fail, even if the first reason is that the sun explodes), but rather when and how it will fail. You can make a whole career out of pulling on that thread.
(I worked at Google with Bob in the mid-2000s. I might have interacted with him once or twice on the internal Java and Guice mailing lists, but never met him in person. He was supremely helpful and knowledgeable, and I admired him from a distance.)
Re:Remembering Bob Lee
by squirtlebonflow on Thursday April 06, 2023 @11:29AM
(#35469710)
But what do you log when there is a failure? A huge call stack, a huge object, some kind of information about what went wrong, what?
RIP :(