Until you want to know which query is slow, which metrics won't tell you.

Being able to do "where query_time_ms > 5000" and then seeing the query has made some of my systems incredibly debuggable¹.

Or just finding a failing request with "where response_code = 500". Yeah, I can "grep 500", but you would be shocked at how often those characters occur by happenstance in logs "grep ' 500 '" is a bit closer but … you're just dancing around a solution engineered to do the job: structured logs.

¹and then finding out someone drew a rather large bounding box that managed to be centered precisely between and encapsulate both Philly & NYC. This kills the DB.

Most metric systems include tags, which allows you to have infinite dimensions of data correlation at your finger tips. Want the full query, or just a normalized version of the query? It should take a couple minutes in most metric engines to add such a simple requirement.

> Most metric systems include tags, which allows you to have infinite dimensions of data correlation at your finger tips.

Unfortunately with most SaaS metric services, your credit card does not scale to support infinite dimensions of data correlation.

Tagging a high cardinality value like a customer ID for example would quickly blow out your bill.

A mergable data type “sample values” would be awesome for these things. Host is dc1b001, api called is /whatever, p99 latency is 117 ms, status is 4xx, sample_corr_id is 0xfafedd12346

Logs should be the input to generate metrics (in addition to any metrics published directly).

Otherwise, it's hard to convert a stack trace into a metric ;) But you can process the logs to generate a metric about the frequency of stack traces.

Ditto for logs telling you "user Foo doesn't have permissions for that operation". Again, you can make a metric about "user permissions failure" but you still need the log to know that it was user Foo.

If you need a metric on the frequency of stack traces, you have a way more serious problem than structuring your logs.

Or you are working for people with such a problem.

This is an attempt to create a logically organized log format optimized for log parsing systems like newRelic or ELK etc. It will help you generate useful dashboards, metrics and event notifications (example: trigger an alert if the percentage of errors exceeds 5%). You can still turn logs into metrics and with that hit 2 birds with one stone.