Re:Software engineering topics I changed my mind on
by xxs on Sunday January 24, 2021 @05:32AM
(#25890904)
>doing more complicated query-like stuff
That's some kind of fallacy - standard datastructures would totally destroy any-sql-alike thing, if it comes to performance (and memory footprint). I guess it does depend on where the background comes when it comes to convenience - or how people tend to see their data. However like I said - for close to 3 decades I have not seen a single reason to do so. On the contrary I've had cases where optimization of 3 orders of magnitude was possible.
Re:Software engineering topics I changed my mind on
by jawns on Sunday January 24, 2021 @09:33AM
(#25892290)
It's easier to find devs who know basic SQL than it is to find devs who know pandas or whatever your language specific SQL-like library is. And the more complicated the queries, the more the gulf widens.
Re:Software engineering topics I changed my mind on
by recursive on Sunday January 24, 2021 @01:04PM
(#25894128)
I don't think pandas was the proposal here. I think "standard data structures" refers to arrays, hash tables, trees, and the like.
Re:Software engineering topics I changed my mind on
by busterarm on Sunday January 24, 2021 @11:00AM
(#25893044)
Performance is not god. It is not the altar at which we sacrifice all other considerations.
Re:Software engineering topics I changed my mind on
by throwaway08320 on Sunday January 24, 2021 @08:23AM
(#25891817)
> for close to 3 decades I have not seen a single reason to do so
Evidently you don't have dataset far exceeding the amount of RAM you can afford.
For a good example look at LMDB.
But if you're doing more complicated query-like stuff, especially if you want to allow for queries you haven't thought of yet, then the DB might be useful.
Sometimes a hybrid of query-able metadata in a DB along with plain old data files is good.
That depends very much on your data, how much things key to each other, and what you're doing with it.