Re:Nanolog supports logging with 7 ns median latency
by pas on Friday August 23, 2024 @12:17PM
(#41330275)
probably in this case it's important to use some kind of synthetic timestamping to preserve the ordering (for example, for a 5.123 [ms] timestamp one can add ~1000000 ns timestamps, so let's say are a thousand entries that need to be ordered, one can then represent them as 5.123000[n] ... and the "000" part is just a silly in-band way to give a hint to someone who will later examine the logs)
Re:Nanolog supports logging with 7 ns median latency
by jnordwick on Friday August 23, 2024 @02:21PM
(#41331562)
since you aren't going to be writing a message per nanosecond, you can always just do `last nanos = max(last nanos + 1, cur nanos)` and then use last nanos for the timestamp. you can even do it in rdtsc ticks and get 1/3 of nano values. Obv the clock isn't nearly that accurate, but it lets you use those fractional nanos to ensure a strictly increasing ordering.
Re:Nanolog supports logging with 7 ns median latency
by gpderetta on Friday August 23, 2024 @12:09PM
(#41330199)
FWIW, I have relied on the accuracy of log timestamps well into the low microseconds.
If you have an event loop, it might be acceptable to sample the counter once per iteration, but less than that it becomes too lossy.
Not sure it matters a lot of to have multiple messages with the same timestamp, since they were added in order you still know which one is older, the problem might arise when you send those logs to a remote place and the order of insert is discarded and the timestamp is used instead.
I assume that when you use a single thread with a queue / ring buffer the order of insertion is kept.