One person’s small is another person’s large. It all comes down to your typical use cases; no surprise there. I have lots of tiny sqlite databases myself, so for me, a few tens of thousands rows is large. But at least I am well aware that multi-gigabyte sqlite databases can and do exist.

For SQLite seems reasonable? I remember it advertised as ini file replacement as well :)

Sqlite easily manages millions of rows. I’ve used it in the past as part of a lazier implementation of an ELK-like stack and it worked fine.

Optimizations like that will be part of the next alpha release after the one I'm working on.

I just honestly cannot imagine how it could start to be slow at a few thousand rows. Calling it an optimization feels wrong somehow.

Glad you hear you are planning to look at it though :)

Larger querying larger databases should be a problem for the RDBMS not for the UI wrapper. So I’d wager he is processing every record on the db irrespective of whether it’s visible on screen or not. It would explain the TUI panic posted elsewhere too.

I have no control over which portions of the screen get updated when without updating the library I'm using.

One way to optimise this is to pull back the entire SQL results in an [][]interface{} (row, column) but then only pass a sub-slice to your TUI (depending on how many rows are visible).

I assume your term library doesn’t create the actual table? So you’d know the term height and you’re then plotting the cells? If that’s not the case then I don’t think you’ll get much further with this project without switching to another term library (or rolling your own — which is actually easier than it sounds).

Lol I just did this. Now way faster and can handle arbitrary amounts of data (there is some flickering on scroll for large data sets I'm trying to sort out).