News for hackers. Stuff that matters.
Re:The Future Database by rmbyrro on Monday May 23, 2022 @01:16PM (#31481880)
My dream was having a database like AWS DynamoDB, but with analytical capabilities built-in the API.
Re:The Future Database by azurezyq on Monday May 23, 2022 @01:36PM (#31482177)
I think you are talking about HTAP(https://en.wikipedia.org/wiki/Hybrid_transactional/analytica...). There are already decent products on the market. But don't expect 100% analytical capabilities like BigQuery / Snowflake. There are tradeoffs.

If your workload fits the model well (mostly OLTP, but occasionally some with heavier aggregations, etc.), it would be awesome.

Example: TiDB

https://docs.pingcap.com/tidb/dev/explore-htap

https://www.vldb.org/pvldb/vol13/p3072-huang.pdf

Re:The Future Database by dravita on Thursday May 26, 2022 @05:04PM (#31523348)
When it comes to HTAP workloads, SingleStoreDB tackled that years ago and has been the leader in that category of workloads. In recent years, the product has expanded to handle broader workloads and use cases. https://www.singlestore.com/resources/research-paper-cloud-n...
Re:The Future Database by Scarbutt on Monday May 23, 2022 @03:02PM (#31483550)
Doing application programming in Dynamodb is a nightmare.
Re:The Future Database by gnulinux on Monday May 23, 2022 @07:29PM (#31486142)
I personally found DynamoDB enjoyable but the trick is you need to understand DynamoDB gives you exactly no way to have analytics, so you have to have a separate and robust story for analytics e.g. we periodically dump our DynamoDB into a RedshiftDB so we get best of the both worlds. I personally find it enjoyable to program, and had a better experience than when everything was Postgres.
Re:The Future Database by rmbyrro on Monday May 23, 2022 @10:18PM (#31487314)
Same experience.

From an OLTP standpoint, the limitations to query the data are actually amazing, because it forces you to think about the right questions from the beginning.

On more flexible DBMS like Postgres, it's very very easy to shoot yourself in the foot.

I find it easier to understand and take into consideration the DynamoBD constraints (explicit) than SQL shortcomings (implicit).

Re:The Future Database by Scarbutt on Tuesday May 24, 2022 @12:37AM (#31488050)
because it forces you to think about the right questions from the beginning.

For many apps thinking that you can have the right questions from the beginning is delusional.

Re:The Future Database by gnulinux on Tuesday May 24, 2022 @04:18PM (#31497078)
> For many apps thinking that you can have the right questions from the beginning is delusional.

Strong disagree, I think you might be approaching this from a traditional angle where analytics questions are also questions of the system. In DynamoDB you only focus on access patterns you need for system to work real-time, without bothering with analytics (as stated above). Any question you want to ask on data outside of the operational code is out of the scope.

Re:The Future Database by dravita on Thursday May 26, 2022 @04:58PM (#31523280)
Here is the database of your dreams: SingleStoreDB. https://www.singlestore.com/customers/fathom/
Re:The Future Database by thdxr on Tuesday May 24, 2022 @09:39AM (#31491940)
been pairing DynamoDB with rockset to solve this
Re:The Future Database by dravita on Thursday May 26, 2022 @04:56PM (#31523253)
Try SingleStoreDB for the combined DynamoDB workloads and analytical workloads in a single database tech, see https://usefathom.com/blog/ditched-dynamodb
Re:The Future Database by rmbyrro on Wednesday May 25, 2022 @06:26AM (#31502537)
Yeah, I do pairing with analytical DBMS.

The issue is the pairing.