News for hackers. Stuff that matters.
Re:Back to basics: Writing an application using Go and Postg... by pstuart on Monday November 22, 2021 @06:34PM (#29312880)
Postgres is awesome technology but for a lot of applications SQLite will do the job just fine. This gets you a single binary with zero external dependencies.
Re:Back to basics: Writing an application using Go and Postg... by Ingon on Monday November 22, 2021 @06:55PM (#29313051)
The main thing stopping me from going this way is that sqlite forces you to use cgo.
Re:Back to basics: Writing an application using Go and Postg... by benhoyt on Monday November 22, 2021 @07:06PM (#29313127)
You should check out https://pkg.go.dev/modernc.org/sqlite -- it's an (automated) conversion of SQLite's C source to Go so you can use SQLite without CGo. Apparently it passes all the SQLite tests. I've used it in a (toy) project, and it seems to work well.
Re:Back to basics: Writing an application using Go and Postg... by henvic on Monday November 22, 2021 @07:03PM (#29313100)
Yes! This is why I made sure to comment about SQLite briefly, and link to this page: https://sqlite.org/whentouse.html
Re:Back to basics: Writing an application using Go and Postg... by Thaxll on Monday November 22, 2021 @07:40PM (#29313348)
How do you do when you have more that one process accessing the DB?
Re:Back to basics: Writing an application using Go and Postg... by simonw on Monday November 22, 2021 @08:38PM (#29313696)
SQLite has a WAL mode you can switch on which handles that scenario: https://til.simonwillison.net/sqlite/enabling-wal-mode
Re:Back to basics: Writing an application using Go and Postg... by exhilaration on Monday November 22, 2021 @11:13PM (#29314529)
I was curious what WAL mode was and this Sqlite page explains it: https://www.sqlite.org/wal.html
Re:Back to basics: Writing an application using Go and Postg... by rakoo on Tuesday November 23, 2021 @11:14AM (#29319548)
Digression: what's your usecase for having multiple processes share the same database ?

SQLite is not an equivalent to a standard networked database, as a developer you're supposed to see it as an enhanced data structure, like an array++. You wouldn't share an array between multiple processes (unless you know exactly what you're doing) so you wouldn't do the same for SQLite. If you really need multiple processes writing the same kind of data, maybe it should be partitioned into multiple independent files ?

Re:Back to basics: Writing an application using Go and Postg... by candiddevmike on Monday November 22, 2021 @06:50PM (#29313009)
I would consider writable, durable disk to be an external dependency in this context.
Re:Back to basics: Writing an application using Go and Postg... by tekstar on Monday November 22, 2021 @07:33PM (#29313302)
You can run sqlite in memory if you pass `:memory:` as the connection string.
Re:Back to basics: Writing an application using Go and Postg... by caseyohara on Monday November 22, 2021 @07:55PM (#29313441)
Does it persist data to disk in memory mode? If not, then I'm not sure I see the point.
Re:Back to basics: Writing an application using Go and Postg... by pstuart on Tuesday November 23, 2021 @02:22PM (#29322060)
If by persist to disk you mean "when you're done", it can by replicating the db to a file. Another option is to back it up over the network, e.g., https://github.com/benbjohnson/litestream
Re:Back to basics: Writing an application using Go and Postg... by rakoo on Tuesday November 23, 2021 @09:59AM (#29318540)
SQL is a very nice querying language, if you can stuff your data in it and run structured queries instead of some spaghetti code it can help a lot. This is the same domain as redis initially, ie a "database" with helpful structures and data types for simple data mangling
Re:Back to basics: Writing an application using Go and Postg... by tekstar on Monday November 22, 2021 @08:58PM (#29313803)
It's great for running tests in parallel.

Or for working with data in memory.

There's a whole world of software outside of saas crud

Re:Back to basics: Writing an application using Go and Postg... by candiddevmike on Monday November 22, 2021 @09:10PM (#29313866)
An in memory database isn't the same thing as PostgreSQL in this context though.
Re:Back to basics: Writing an application using Go and Postg... by w-j-w on Monday November 22, 2021 @06:40PM (#29312932)
I keep seeing this comment on HN, but I never really get it. In the world of platform as a service, launching a database instance is far easier than managing disk based storage yourself. It also has a very obvious route to horizontally scaling the web process. Deployment is easier too with an external database, there's no data on the disk of the machine running the web process that needs to be carried over.
Re:Back to basics: Writing an application using Go and Postg... by mtlynch on Monday November 22, 2021 @09:37PM (#29314026)
I had the same objection to SQLite, and then I heard about Litestream, and it won me over.[0]

Litestream watches your SQLite database and then streams changes to a cloud storage provider (e.g., S3, Backblaze). You get the performance and simplicity of writing SQLite to the local filesystem, but it's syncing to the cloud. And the cool part is that you don't have to change any of your application code to do it - as far as your app is concerned, it's writing to a local SQLite file.

I wrote a little log uploading utility for my business that uses Litestream, and it's been fantastic.[1] It essentially carries around its data with it, so I can deploy my app to Heroku, blow away the instance and then launch it on fly.io, and it pops up with the exact same data.[2]

I'm currently in the process of rewriting an open-source AppEngine app to use SQLite + Litestream instead of Google Firestore.[2] It's such a relief to get away from all the complexity of GCP and Firestore and get back to simple SQLite.

[0] https://litestream.io/

[1] https://mtlynch.io/litestream/

[2] https://asciinema.org/a/I2HcYheYayeh7aHj23QSY9Vyf/embed?size...

[3] https://github.com/mtlynch/whatgotdone/pull/639

Re:Back to basics: Writing an application using Go and Postg... by tekstar on Monday November 22, 2021 @07:38PM (#29313336)
> launching a database instance is far easier than managing disk based storage yourself

Look, it comes down to what you, the developer are comfortable with. If you know and are comfortable with running something like postgres in the cloud, then go for it.

Some of us prefer to not. Similarly, not all of us nuke the entire filesystem on every deploy :)