Re:InfluxDB 1.0 GA Released: A Retrospective and What’s Next
by chrissnell on Thursday September 08, 2016 @03:03PM
(#12456205)
Sweet. Since it's UDP, it's connectionless so any penalty for not batching data points is likely minor. We're not a Python/uWSGI shop, but it looks like I might be able to emulate the Influx format as you have done using this nginx logging module:
uWSGI has a very flexible logging system. You can create a log with just requests, and completely customize the format of the line.
On top of that, uWSGI supports creating multiple log targets, and supports logging directly into a UDP socket.
If you make it match InfluxDB's line format (https://docs.influxdata.com/influxdb/v1.0/write_protocols/li...), and you set up InfluxDB to accept connections from a matching UDP socket, then uWSGI can essentially log every request directly into influx.
Like this for example:
You can also add 'path="%(var.PATH_INFO)"' to that but there is a potential data injection vulnerability if you don't re-parse the output.With that, you can create nice graphs with Grafana to analyze your loading times, error rates, page sizes etc:
https://i.imgur.com/fzbXgUd.png
You can also set up Kapacitor to identify when your page load times are abnormally increasing. We had the idea after a database connection leak increased our average loading times by 300%.
Do note that this doesn't batch inputs, so 1 request = 1 connection to influx. But it's pretty easy to put a middleware agent in between those to do validation and batching.
The nice thing about all this is it's application level timing metrics. There's not a lot of ways to get that, except with application middleware which can't always catch everything you want.