Re:Stuff that every programmer should know: Data Visualization
by anko on Wednesday July 30, 2014 @04:45PM
(#8110834)
Basically I'm importing logs and system events. I run queries like show me the top 10 events over the last 24 hours from this source that were marked critical. Or for each farm shipping web logs, aggregate on the hosts, and then aggregate on the status code, and then give me the number of documents in each summed for each hour of the day.
At the moment I roll up daily stats and store them in a separate database for longitudinal analysis, but eventually I'd like to ship data that is more than a couple of weeks old to hadoop.
At the moment I roll up daily stats and store them in a separate database for longitudinal analysis, but eventually I'd like to ship data that is more than a couple of weeks old to hadoop.