News for hackers. Stuff that matters.
Re:You Are Not Google (2017) by cheez on Thursday April 04, 2019 @04:08PM (#19576567)
Here I am still using SQLite (had to add an index yesterday!) Maybe one day I will grow up :(
Re:You Are Not Google (2017) by sli on Thursday April 04, 2019 @04:34PM (#19576820)
SQLite is good quality software, and very heavily tested. There's nothing wrong with using it so long as you don't outright misuse it, and they are clear about when you shouldn't be using it.
Re:You Are Not Google (2017) by snazz on Thursday April 04, 2019 @06:50PM (#19577891)
I very rarely run into a situation where I need more than SQLite. The page on the subject is great: https://www.sqlite.org/whentouse.html
Re:You Are Not Google (2017) by collyw on Friday April 05, 2019 @04:56AM (#19580856)
I's not so good for multi user systems as I understand as it has a lock on the whole database when writing.
Re:You Are Not Google (2017) by snazz on Friday April 05, 2019 @09:33AM (#19582300)
Queuing works most of the time, does it not?
Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @12:00PM (#19583813)
Queuing and retry works most of the time. Only time I truly get "database is locked" is when I open the shell while other processes are accessing the DB
Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @01:06PM (#19584657)
Are all writes blocked during database migrations?

ALTER TABLE is quite limited in SQLite (for example there is no DROP COLUMN). Have you missed this?

Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @04:25PM (#19586663)
All writes are blocked during database migrations, so there is potential offline time. For example, when I added the index, I needed to be offline. Fortunately, my application is only needed during business hours.

Yes I miss proper ALTER TABLE a lot when it would have been useful. But my experience shows me that I can design software pretty well following YAGNI and the occasional instances of when I need missing functionality is more than made up for the ease of use otherwise.

Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @04:48PM (#19586891)
Thanks for sharing your experience with SQLite :-)

What are the advantages, in your own experience, of using SQLite instead of something like PostgreSQL? With PostgreSQL, you could run database migrations during business hours and use a proper ALTER TABLE.

Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @05:05PM (#19587040)
Main reason is that SQLite is embedded with my application. There isn't a second, third or fourth "microservice" to spin up. I know Terraform/Docker, love them, but the gymnastics I need to orchestrate for my particular situation is not worth it. So for the tradeoff of complexity for functionality, SQLite hits a very good sweet spot for me.
Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @01:02PM (#19584593)
Where do you queue: in memory? on disk? in a dedicated queue server (Redis, Kafka, ActiveMQ, etc.)? in another SQLite database?
Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @04:26PM (#19586677)
So far, haven't needed queuing outside of a single thread in the application (I know, what happens if it crashes!) Hasn't happened in ~8 years. Won't cry if it does.
Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @04:51PM (#19586914)
I have an app that uses an in-memory queue too in one thread :-)

I'm curious: why do you queue instead of directly writing to the database?

Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @05:05PM (#19587037)
At the moment, I queue to batch insert real time data at ~5 second intervals. It seems less wasteful of resources, but quite frankly is probably not needed.
Re:You Are Not Google (2017) by collyw on Friday April 05, 2019 @10:07AM (#19582608)
To be fair I have only really used SQLite for early development stages so haven't investigated workarounds in much detail.
Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @03:10AM (#19580400)
How do you access remotely, from your workstation, to the SQLite database on your server, while it is still serving requests?
Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @12:01PM (#19583827)

  ssh user@server
  sqlite file://$PWD/db.sqlite?mode=ro
Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @01:04PM (#19584618)
Yes, this will work of course :-)

I was thinking of something more "graphical", similar to pgAdmin or SequelPro, when you need to navigate in a dataset.

Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @05:14PM (#19587109)
I use https://sqlitebrowser.org/ if I need to but primarily Jupyter w/ Pandas to explore and understand the data and prototype new queries.
Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @05:33PM (#19587251)
Regarding SQLiteBrowser, I guess you have to download a copy of the database on your local workstation?
Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @07:11PM (#19587893)
Correct. I use it rarely.
Re:You Are Not Google (2017) by cheez on Friday April 05, 2019 @07:24PM (#19587992)
Just ran it using X remotely, so that's an option too.
Re:You Are Not Google (2017) by Redoubts on Friday April 05, 2019 @04:14PM (#19586567)
There's https://sqlitebrowser.org,

But I suppose you'll have to string up something if you want it to access a remote db.

Re:You Are Not Google (2017) by ngrilly on Friday April 05, 2019 @04:43PM (#19586830)
There is also https://www.phpliteadmin.org, but I'm not a fan of the user interface.
Re:You Are Not Google (2017) by marcosdumay on Saturday April 06, 2019 @02:36PM (#19592279)
Whatever exists specifically for sqlite, sshfs is your friend too.