Re:A pure Go embedded SQL database
by eatonphil on Saturday January 29, 2022 @05:31PM
(#30131254)
Well for one, if it's in Go, it wouldn't need Cgo.
[ Parent
| on HN ]
The same author ("cznic") has also built a C-to-Go compiler/transpiler and used it to convert SQLite to pure Go source, so that it doesn't require CGo: https://gitlab.com/cznic/sqlite ... the converted Go code is pretty much unreadable and it's somewhat slower than SQLite via CGo. But a pretty neat idea! I use it in one of my projects and it works well.
I saw that recently. I would be interested in using it but it wasn't clear what level of compatibility it has or how many tests it's passing/not passing.
Yeah, fair concern. I saw a while back (looks like Aug 2020) that he got all 928,000 SQLite tests passing -- search for "0 errors out of" on this page: https://pkg.go.dev/modernc.org/sqlite#section-readme
Wow! I'd almost trust it more if only 99% of tests were passing rather than 100% of tests passing. :)
Hehe. Well, if you look at the previous release it shows 99.7% percent of (a smaller number of) the tests passing. So you've got the best of both worlds. :-)
Nice. I'll have to give it a try.
I think there are many tests used for testing sqlite that are not public. Hipp said so in a podcast I recall.
Re:A pure Go embedded SQL database
by Beltalowda on Sunday January 30, 2022 @06:06AM
(#30135877)
There's TH3[1], and you indeed need to purchase a license for that. The price is listed as "call"[2], so I guess it's not cheap.
Nice. I tried to cross-compile a Go app using SQLite for darwin/arm64 a couple months back and couldn’t figure it out for the life of me. Ended up just shipping a darwin/amd64 binary and letting Rosetta handle it. Will give this a spin when I have time.
Re:A pure Go embedded SQL database
by preetamjinka on Sunday January 30, 2022 @12:02AM
(#30134196)
I should write a blog post this but here’s what I do in GitHub Actions for a Go binary that uses SQLite:
https://github.com/crossjoin-io/crossjoin/blob/main/.github/...
Re:A pure Go embedded SQL database
by throwaway894345 on Saturday January 29, 2022 @07:33PM
(#30132240)
Not sure why this is downvoted. SQLite is great and all, but not depending on FFI is really nice for a lot of things (e.g., cross-compiling is a lot simpler). I'll trade quite a bit to avoid even the very highest quality C dependencies.
Re:A pure Go embedded SQL database
by anderspitman on Saturday January 29, 2022 @06:00PM
(#30131485)
It's pretty easy to statically link SQLite[0]. Are there other disadvantages to CGO?
Without additional configuration cgo breaks cross-compilation.
Re:A pure Go embedded SQL database
by anderspitman on Sunday January 30, 2022 @02:24AM
(#30134834)
Oh right how could I forget that.
Re:A pure Go embedded SQL database
by christophilus on Saturday January 29, 2022 @06:08PM
(#30131544)
Yes. It doesn’t play nice with goroutines and Go’s concurrency model, from what I understand.
Yes, function calls across the boundary are slow. It's fine if the functions do lots of work, it's very expensive for function calls that should be cheap.
For sqlite it might not matter too much, since any query does quite a bit of work.
Re:A pure Go embedded SQL database
by throwaway894345 on Saturday January 29, 2022 @07:36PM
(#30132274)
Agreed. I wouldn't be surprised if a pure-Go alternative would be slower. Someone remarked upthread that the same author wrote a C-to-Go transpiler specifically to make a pure-Go sqlite and it was (somewhat unsurprisingly) considerably slower than the CGo version. It's possible (and to an extent, likely) that the transpolar was generating suboptimal Go code, but I would wager that a hand-written version would also be slower.
A query may do a lot of work but iterating over many results could be very slow.
Yes, the API is probably suboptimal for Go. It's better to make a wrapper that does the query and fetches many results in one call, into a Go buffer.
There is pure go port of SQLite
There are a few go libs now that offer SQLite without cgo.
https://github.com/crawshaw/sqlite https://github.com/zombiezen/go-sqlite
crawshaw/sqlite actually uses CGo -- it's a wrapper around the C version of SQLite. For example, see https://github.com/crawshaw/sqlite/blob/23d646f8ac00d9dd2390...
zombiezen/go-sqlite uses cznic's pure Go converted version of SQLite, so avoids CGo. It's explicitly stated to be "a fork of crawshaw.io/sqlite that uses modernc.org/sqlite, a CGo-free SQLite package. It aims to be a mostly drop-in replacement for crawshaw.io/sqlite."