Re:Migrating Graphite from SQLite to PostgreSQL
by yeukhon on Saturday December 14, 2013 @10:07PM
(#6908195)
There is no reason why one would use SQLite in real production. Even during testing and dev, I am a firm believer of testing against real database that you are going to use in production. The problem "it's hard to setup a new database" is a result of not willing to invest in re-provisionable system.
Re:Migrating Graphite from SQLite to PostgreSQL
by obfuscurity_ on Saturday December 14, 2013 @10:30PM
(#6908254)
Except that while Graphite is a "real web application", it's generally internal-facing only. As such, I never looked at it as needing a true "production-ready" RDBMS with real concurrency. And to be honest, we got by just fine with it in production at Heroku and GitHub, at heavy volume, running in a single server on SQLite. It never bit us, and all things considered, was a reasonable choice.
That said, knowing what I know now, I would've never done that. ;-)
Re:Migrating Graphite from SQLite to PostgreSQL
by yeukhon on Saturday December 14, 2013 @10:38PM
(#6908270)
I disagree because it is internal facing lol. If the software is capable of running on a true production-ready DBMS, there is no point why it can't be done already.
If SQLite is cheap to use because it's a single file, well, a sudo apt-get install PostgreSQL and then a few commands to set up the database and user password won't take you longer than 10 minutes. That's what I am arguing. There is no reason why we would use SQLite for heavy loading application such as logging. And I think we can now agree that's indeed true and I think we should promote people to use production-ready DBMS even during dev and testing, because those are the one you are going to use in production :)
Re:Migrating Graphite from SQLite to PostgreSQL
by obfuscurity_ on Saturday December 14, 2013 @10:50PM
(#6908292)
Nobody ever said it /can't/ be done. I would never have argued against using a real RDBMS. But I've often been guilty of telling folks that SQLite was "good enough".
Installing PostgreSQL as you described is not "good enough". If you're going to bother with that, you need to understand the database enough to tune it properly, setup backups, test restores, etc. There are more considerations than simply installing it from apt-get and then forgetting about it.
Re:Migrating Graphite from SQLite to PostgreSQL
by mst on Sunday December 15, 2013 @02:54AM
(#6908732)
There are plenty of reasons.
* Single file to back up
* Unix fs permissions sufficient for security
* Simplicity valuable over write concurrency
* Ability to write functions/aggregates/collations in the language you're embedding sqlite in
* No need for maintaining and securing a system daemon
This article isn't, at all, about a problem with SQLite. It's about graphite shipping a stupid postgres schema. Had they used text for the column like anybody who knows how to use postgres would recommend, the graph names would've transferred fine.
TL;DR - graphite devs don't understand postgresql, graphite user misdiagnoses the problem and blames sqlite for it.
Re:Migrating Graphite from SQLite to PostgreSQL
by Someone on Sunday December 15, 2013 @05:42AM
(#6908979)
The way I read this, this _is_ a problem with SQLite (somewhat). Both schemas limit the length of graph names, but SQLite ignores that part of the schema.
One can discuss whether it is a good idea to have that arbitrary, low limit, but that is not what this is about.
I agree that there are plenty of reasons to use SQLite in production. IMO, the conclusion should have been that one should run the same database in development, test, and production.
That said, knowing what I know now, I would've never done that. ;-)
If SQLite is cheap to use because it's a single file, well, a sudo apt-get install PostgreSQL and then a few commands to set up the database and user password won't take you longer than 10 minutes. That's what I am arguing. There is no reason why we would use SQLite for heavy loading application such as logging. And I think we can now agree that's indeed true and I think we should promote people to use production-ready DBMS even during dev and testing, because those are the one you are going to use in production :)
Installing PostgreSQL as you described is not "good enough". If you're going to bother with that, you need to understand the database enough to tune it properly, setup backups, test restores, etc. There are more considerations than simply installing it from apt-get and then forgetting about it.
TL;DR - graphite devs don't understand postgresql, graphite user misdiagnoses the problem and blames sqlite for it.
One can discuss whether it is a good idea to have that arbitrary, low limit, but that is not what this is about.
I agree that there are plenty of reasons to use SQLite in production. IMO, the conclusion should have been that one should run the same database in development, test, and production.
Yes there is. Small storage for a local app? SQLite is ideal.
Localstorage and app in desktop and mobile are definitely arguable.
Also, at the very least, graphite should have been able to enforce them at the app level in addition to at the DB level.