Re:Three things you should never put in your database
by rplnt on Wednesday May 02, 2012 @04:33AM
(#3918076)
Well, he advised on using tools like Splunk which can do all that for you. But I'm pretty sure they use database internally. So conflict advices I'd say.
Re:Three things you should never put in your database
by frankwiles on Wednesday May 02, 2012 @03:48PM
(#3920788)
Splunk, last I checked, used PostgreSQL for it's backend. While maybe badly worded, my point was more toward the users who have a two column log table with a timestamp and a syslog style line of text. The problem with "advice" posts like these is you can't cover every possible scenario without confusing the less experienced readers. Do I think loggly or splunk cloud should store their logs as flat files? Hell no, their app IS logging. But in the general case of application architecture I don't think it is appropriate or terribly useful for most. I know they say "never say never", but I felt it justified when I believe it's a bad idea in 99% of apps.
It's really more about using the right tool for the job, Splunk, Loggly, NoSQL solutions, etc. are a MUCH better fit than an RDBMS for this sort of work.
Re:Three things you should never put in your database
by nl on Wednesday May 02, 2012 @07:48PM
(#3921757)
Splunk certainly does NOT use Postgres for it's backend(!)
It uses custom index files stored on a filesystem.
It can also connect to MySQL[1] and to HBase[2], but its own data is kept in its own index.
[1] http://blogs.splunk.com/2011/12/20/swimming-with-dolphins/
Re:Three things you should never put in your database
by frankwiles on Thursday May 03, 2012 @12:36PM
(#3924538)
Makes sense, it's been a long time since I installed or had shell access to a Splunk box, I just remember it needing PG installed. Likely that's changed over the years.
Don't be afraid of putting logs in databases and then figuring out how to scale it. It's not that hard and it's very worth it.