All modern columnnar/analytical databases have no problem in pulling back older data; in fact, that's the point of running large scale analytics.

I agree, but analytics is orthogonal topic.

Unless you want to use columnar database to retrieve 15 transaction that particular user did month ago...

There is nothing wrong with that use case. It works just fine. It may not have the same latency as an OLTP database but there are many factors (indexing, sort/partition keys, compression, etc) that can return similar performance for those queries.

Just to clarify what you advocate for: you want to run this type of queries on analytical databases (like Snowflake, Redshift etc) to display 10 rows on frontend device?

    select * from transactions where user_id='asdf' and completed_date <'2021-02-01' and completed_date >= '2021-01-01'

Sure. Again the database and data layout make a difference. Snowflake has startup latency since the "warehouse" might not be running but Redshift will do fine, along with many others that are always on.

Redshift has some pretty significant and fairly unpredictable per-query overhead. For example, all queries are compiled, then code is distributed to every node in the cluster, then executed and gathered before returning. And most analytics databases use a scheduled execution model, which can delay execution from a couple seconds to several minutes, depending on database loads.

Your comment clearly illustrates that you have no working knowledge of Clickhouse or parquet file format or data archiving capabilities available in 2021. It's OK! I was in the same boat until I needed to implement such a solution for my use case. What I'm suggesting does not limit their customers from searching any historical data. Matter of fact, it might be 100x to 1000x faster for them to do so with the suggested solution. I strongly believe that mission critical transactional databases (postgres in this case) MUST be run very lean to keep their app running at hyper speeds at all times. 50TB overhead seems very inefficient when you take into account the the low cost solutions available in this day and age.

Based on my personal experience of achieving 94% compression on 2TB of data using snappy parquet file format, they could be looking at a final dataset size of 3.5TB on Clickhouse.

> Your comment clearly illustrates that you have no working knowledge

Please omit swipes like that from your comments here. The rest of your comment would be find without that bit.

https://news.ycombinator.com/newsguidelines.html

>Matter of fact, it might be 100x to 1000x faster for them to do so with the suggested solution.

That must be trolling. 1000x faster than single digit millisecond indexed query retrieving 15 rows?

The fact that you keep talking about storage size means that you're talking about analytics not transactional needs.

>Your comment clearly illustrates that you have no working knowledge of Clickhouse or parquet file format or data archiving capabilities available in 2021. It's OK! I was in the same boat until I needed to implement such a solution for my use case.

Also, fuck off with that condescension.

> fuck off with that condescension

Please follow the site guidelines, regardless of whether or not someone else has broken them. Otherwise we just get a downward spiral.

https://news.ycombinator.com/newsguidelines.html

Sorry, I get way too heated when I see this type of condescending argumentation.

Yes, I experience that too and I bet most people do. One way I often explain this that seems to work well is this:

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...