HermitX

Wow, there seems to be a lot of impressive new features!

Great blog, thanks for sharing!

Excellent content! It looks like the Iceberg + StarRocks open lakehouse Architecture is highly effective.

Great article, thank you for sharing! I have a question I’d like to discuss with the author. Spark SQL is a great product and works perfectly for batch processing tasks. However, for handling ad hoc query tasks or more interactive data analysis tasks, Spark SQL might have some performance issues. If you have such workloads, I suggest trying data lake query engines like Trino or StarRocks, which offer faster speeds and a better query experience.

I believe Snowflake not only needs an open-source table format but also an open-source data lake query engine. Currently, Snowflake has too little presence in this area, and its stock price is reaching new lows.

I suggest you try using StarRocks to query the data lake directly. I know many large companies, including Tencent and Pinterest, are doing this. StarRocks has a truly mature vectorized execution engine and a robust distributed architecture. It can provide you with impressive query performance.

“Specifically, online analytics workloads would be migrated to Druid/StarRocks”, I'm very interested in this part. Look forward to knowing more about it.

As long as the United States is the world’s leading superpower, there will always be a second place trying to challenge its position. Previously it was Japan, now it is China, and there will be other countries in the future.

When the times leave you behind, they won't even say goodbye.

Is ClickHouse a suitable engine for analyzing events? Absolutely, as long as you're analyzing a large table, its speed is definitely fast enough. However, you might want to consider the cost of maintaining an OSS ClickHouse cluster, especially when you need to scale up, as the operational costs can be quite high.

If your analysis in Postgres was based on multiple tables and required a lot of JOIN operations, I don't think ClickHouse is a good choice. In such cases, you often need to denormalize multiple data tables into one large table in advance, which means complex ETL and maintenance costs.

For these more common scenarios, I think StarRocks (www.StarRocks.io) is a better choice. It's a Linux Foundation open-source project, with single-table query speeds comparable to ClickHouse (you can check Clickbench), and unmatched multi-table join query speeds, plus it can directly query open data lakes.

As long as there is greed in human nature, people like SBF will continue to emerge.

To the best of my knowledge, Meta has significantly reduced its investment in the Velox project. Apart from Meta, I'm not aware of any other major company that really uses Velox in a production environment. Frankly speaking, Velox may have already missed the window of opportunity for rapid development. If you're looking for a vectorized execution engine, you could consider ClickHouse (www.clickhouse.com) or StarRocks (www.starrocks.io). If your data analysis scenarios require more multi-table join operations, StarRocks is clearly a better choice.

I This is an intriguing article, clearly showing the team's fondness for Postgres. However, Postgres is an OLTP product. I'm curious about what system Figma's data team uses for their data analysis tasks.

Oh my, the Dragon Ball series of comics are wonderful memories from my childhood! I even want my kids to read these comics now! I hope Akira can rest in peace.

Oh my, the Dragon Ball series of comics are wonderful memories from my childhood! I even want my kids to read these comics now! I hope Akira can rest in peace.

Yes, I understand

If this video call gives the interviewee a final opportunity to express their thoughts and has the potential to change the outcome, then I think it still has positive value. However, as mentioned in the article, most people feel very awkward and have nothing to say during this meeting, which is simply torturous. In that case, I think it would be better to replace the video conference with an email, after all, if you don't allow the possibility of changing the outcome, why not make it more comfortable for both parties?

In the movie The Matrix, should those people trapped in the cultivation tanks be considered as having seven days of rest a week?

I've read his book 'Amp it Up,' and I think it's pretty good. However, I would still recommend that all Snowflake users pay attention to your costs. Here is a plan that could save you 80% on costs, worth checking out. https://medium.com/p/deb87aa69bc5

Every time I see a beautiful tattoo, I always feel an itch, eager to try it. But whenever I think about how much more troublesome it would be to remove these tattoos if I changed my mind, I back down. So, I haven't even started to worry about whether these inks could potentially harm my organs :-p

What do you want? You can connect with the StarRocks eng team on the slack channel. https://join.slack.com/t/starrocks/shared_invite/zt-z5zxqr0k...

Here is a fantastic video made by Soumil Shah, using MinIO+Hudi+StarRocks+Superset. It is amazing to have an interactive query experience on a data lake directly! https://www.youtube.com/watch?v=JkKBzrQTKx0

I've already thought of the follow-up to this article, 'Querying a billion rows on a $XX budget.' Let me give you my answer directly: switch from Snowflake to StarRocks. It's an open-source project under the Linux Foundation, with speed that's more than adequate, especially for queries involving multiple tables. If you're interested, you might want to check it out, https://medium.com/starrocks-engineering/how-to-reduce-snowf....