Ideas to improve the user and developer experiences of databases

Re:Ideas to improve the user and developer experiences of da... by sa46 on Wednesday March 10, 2021 @02:34PM (#26415173)

I've used Datadog but if you need per query tracing from the database logs, I think the only way is to embed a trace ID in a SQL comment (or in an no-op column) like the grandparent showed.

Datadog is tracing the request from the application akin to something like

    openTrace("Query Name", queryStr)
    db.Query(queryStr)
    closeTrace()

The gap with that style of application-level tracing is that the database logs give no indication of where a query came from, hence the need for embedding a comment in the query with the trace ID.

I would love for a better mechanism than SQL comments for distributed tracing all the way to the database.

[ Parent | on HN ]

Re:Ideas to improve the user and developer experiences of da... by theptip on Wednesday March 10, 2021 @03:38PM (#26415890)

I guess what I'm getting at is questioning why you need to look at the database logs (beyond what your APM system can pull from the DB's metrics).

What are you getting from the DB logs that you can't get elsewhere?

[ Parent | on HN ]

Re:Ideas to improve the user and developer experiences of da... by sa46 on Wednesday March 10, 2021 @10:31PM (#26419866)

Sure, off the top of my head:

- Explain plans for slow queries, via auto_explain in Postgres. You could get really fancy here and convert the pieces of an explain plan (Aggregate, ModifyTable, FullTableScan) into proper distributed traces. Hard to tell, but it looks like GCP might offer that.

- Various errors logs associated with queries that require more detail than a SQL state code. For example, lock timeouts.

- Debugging, especially root cause analysis when you're trying to figure out why something broke. The trace IDs help build up context.

[ Parent | on HN ]