News for hackers. Stuff that matters.
Re:SQLite in Go, with and Without Cgo by eloff on Friday May 13, 2022 @08:55AM (#31366964)
It's not as simple as NIH. Using cgo means you have to build and link external dependencies in another language. Compared to Go, this makes it quite a bit more difficult to build and distribute than a single binary. But I think the biggest reason is cgo is slow. Unlike rust which uses the C calling convention (correct me if I'm wrong) and has no garbage collector or coroutine stacks to worry about, Go has a hefty price to pay when calling a C funcion. For something like sqlite that causes a noticable slowdown. Typically what I like to do in this case is write a C wrapper around the slow API that let's me batch work or combine multiple calls into one. With sqlite, instead of fetching rows one at a time, I would write a wrapper that takes some arrays and or buffers to fetch multiple rows at a time. That amortizes the overhead and makes it less significant. When that's not possible, I use unsafe and or assembly to lift just the problematic C calls into Go. That can sometimes work wonders, but it's also not a magic bullet.
Re:SQLite in Go, with and Without Cgo by Georgelemental on Friday May 13, 2022 @11:03PM (#31374746)
> correct me if I'm wrong

Rust can use the C calling convention to call C functions or export functions to C code, but this requires extra annotations. By default, Rust uses its own unstable ABI.

Re:SQLite in Go, with and Without Cgo by apayan on Friday May 13, 2022 @02:13PM (#31370835)
Interesting! I've never considered batching my calls to SQLite from Go that way. Do you have any numbers you can share about performance when doing that?
Re:SQLite in Go, with and Without Cgo by eloff on Friday May 13, 2022 @02:54PM (#31371256)
I don't, but I'm general it matters most for the cheapest C calls, so the functions doing the least work. Batching those somehow can give big speedups, over 2x, depending on how much of the total time was going into cgo overhead.