Re:Building a recommendation engine inside Postgres with Pyt...
by systemvoltage on Tuesday October 26, 2021 @06:14PM
(#29007033)
Database is the last thing that scales usually so if you put a bunch of computational load on it besides queries, you've set yourself up for scaling/sharding sooner than later.
[ Parent
| on HN ]
Re:Building a recommendation engine inside Postgres with Pyt...
by lowwave on Tuesday October 26, 2021 @06:54PM
(#29007433)
Not always! If computation involves math over set of records, then Postgres is great for that. Have the operation inside the db reduce connection pooling on the application level.
Re:Building a recommendation engine inside Postgres with Pyt...
by rbanffy on Tuesday October 26, 2021 @08:48PM
(#29008362)
> you've set yourself up for scaling/sharding sooner than later.
OTOH, if you can pre-process the data (or shards) in the server (or servers), you won't need to move it across the network for processing.
Hard problems are hard and one of the reasons databases outlive their systems and developers is because moving that data is a Royal Pain.
Re:Building a recommendation engine inside Postgres with Pyt...
by jamesfinlayson on Tuesday October 26, 2021 @07:41PM
(#29007867)
I remember a friend telling me in a previous job me they loaded the database with stored procedures because it ran on the most powerful server when the product launched.
And then the database server hit its capacity very quickly.
Re:Building a recommendation engine inside Postgres with Pyt...
by rbanffy on Tuesday October 26, 2021 @08:44PM
(#29008343)
Reminds me of why I learned to program in PostScript - the PostScript printer was, by far, the fastest computer I had access to - also it had a ton of memory and its very own SCSI hard disk.
Plotting a Mandelbrot on the Mac would take a lot longer, even in C, than just making the printer do it.
Usually overnight, because the program took a couple hours to run most of the time.