jkharrat

Notes from the author:

This story is not a push for a standard but rather an attempt to create a logically organized log format optimized for log parsing systems like newRelic or ELK etc. It will help you generate useful dashboards, metrics and event notifications (example: trigger an alert if the percentage of errors exceeds 5%).

Implementing a standardized log system in your applications will cost you time and money. Debugging will costs you too, especially in edge cases where information is key. Your decision should consider both sides of the equation.

Logging is a very opinionated and sometimes divisive conversation. However, no matter which format you will use, it is always better than no logs at all.

I agree with the simplicity approach I like it too. However, I think sometimes we do not have a choice especially in distributed sytems or "micro"services environement. I think with great "complexity" comes great responsibility.

This is an attempt to create a logically organized log format optimized for log parsing systems like newRelic or ELK etc. It will help you generate useful dashboards, metrics and event notifications (example: trigger an alert if the percentage of errors exceeds 5%). You can still turn logs into metrics and with that hit 2 birds with one stone.

Sometimes you face an edge case that cannot be reproduced easily even while waiting for this enhanced logs level. I would 1/ still log as much as possible, 2/ implement a smaller logs rotation period (upload logs to glacier or S3 after a week), 3/ tune down the verbosity if you want to avoid some unwanted noise later. Use a return of experience approach to get the perfect level for you!

I would do the same if I was you. I will still have a log whenever we catch an error because it will help debugging this cron or job or even identify an issue in the provided data. No matter how divisive the opinions about logging are, I still think that mini logs are better than no logs at all.

Der @thamer thanks for the advice. I have added it to the article :) Much appreciated