Re:Updating a 50 terabyte PostgreSQL database (2018)
by manigandham on Monday March 22, 2021 @09:16AM
(#26540334)
There is nothing wrong with that use case. It works just fine. It may not have the same latency as an OLTP database but there are many factors (indexing, sort/partition keys, compression, etc) that can return similar performance for those queries.
Re:Updating a 50 terabyte PostgreSQL database (2018)
by KptMarchewa on Monday March 22, 2021 @09:21AM
(#26540393)
Just to clarify what you advocate for: you want to run this type of queries on analytical databases (like Snowflake, Redshift etc) to display 10 rows on frontend device?
select * from transactions where user_id='asdf' and completed_date <'2021-02-01' and completed_date >= '2021-01-01'
Re:Updating a 50 terabyte PostgreSQL database (2018)
by manigandham on Monday March 22, 2021 @09:25AM
(#26540439)
Sure. Again the database and data layout make a difference. Snowflake has startup latency since the "warehouse" might not be running but Redshift will do fine, along with many others that are always on.
Re:Updating a 50 terabyte PostgreSQL database (2018)
by darksaints on Monday March 22, 2021 @09:56AM
(#26540809)
Redshift has some pretty significant and fairly unpredictable per-query overhead. For example, all queries are compiled, then code is distributed to every node in the cluster, then executed and gathered before returning. And most analytics databases use a scheduled execution model, which can delay execution from a couple seconds to several minutes, depending on database loads.
Unless you want to use columnar database to retrieve 15 transaction that particular user did month ago...