Re:A request to cloud providers: Give us access to the logs,...
by mkjones on Thursday May 26, 2011 @03:21AM
(#2586934)
Even ignoring other commenters' valid concerns about sensitive data stored in logs, this can be a tough thing to scale.
Say the logs live on disk on a filer somewhere. Need to do a security investigation? OK, dig up the file the logs are in for that particular user and grep around. Lots of disk seeks, but pretty fast.
Scaling that to support production-scale queries is a significant undertaking. (Oh, you can only do 100 seeks per second, need on average 100 seeks per query, and desire to service a low demand of 100 qps? Not gunna happen!)
Say the logs live on disk on a filer somewhere. Need to do a security investigation? OK, dig up the file the logs are in for that particular user and grep around. Lots of disk seeks, but pretty fast.
Scaling that to support production-scale queries is a significant undertaking. (Oh, you can only do 100 seeks per second, need on average 100 seeks per query, and desire to service a low demand of 100 qps? Not gunna happen!)