Re:400TB Single Cluster: OceanBase Powers Kwai`s Core Business
by jinqueeny on Friday December 27, 2024 @11:11PM
(#42528477)
Sorry if the post sounds too marketing-heavy. I'm sharing this case study because it addresses a common technical challenge: scaling a high-traffic application (Kwai, a video platform) from traditional MySQL to a distributed database system. It describes the journey from managing 300+ MySQL shards to consolidating into a single 400TB cluster using OceanBase, handling over 1 million QPS during peak loads. While the post is oriented towards promoting OceanBase, it resonated with developers because it provides specific technical details about real-world scaling problems, architecture decisions, and performance metrics that many engineering teams face when growing beyond traditional MySQL setups.
Re:400TB Single Cluster: OceanBase Powers Kwai`s Core Business
by theideaofcoffee on Friday December 27, 2024 @11:26PM
(#42528560)
This comment pegs my AI-slop-o-meter scale high. No real human genuinely talks like this. Perhaps sharing specific problems you ran into in your single host to distributed sql journey that this product solved for you would be useful? That would be much more interesting than a post about a marketing page and Someone Else’s Problem.
Re:400TB Single Cluster: OceanBase Powers Kwai`s Core Business
by pkkkzip on Friday December 27, 2024 @11:20PM
(#42528536)
this reminds me of stackoverflow's vertical scaling with a fail over server.
drastically simplifies scaling when you are dealing with only one machine with huge amount of RAM and maximum cores.
I think the most you can get now is 192x2 cores and double digit TB of RAM
but more economical is proably 128x2 cores with single digit TB of RAM
its pretty insane how much traffic you can throw at a single machine like this espeically if you use optimized languages like Go or Rust to handle traffic.
there's still use for distributed computing but 99.8% of websites do not really need it.
Re:400TB Single Cluster: OceanBase Powers Kwai`s Core Business
by a012 on Saturday December 28, 2024 @01:15AM
(#42529004)
This is what we did for a now defunct service where it’s scaled to a biggest available machine at that time (less than a TB of memory) and the db was mostly for a freaking giant table.
Re:400TB Single Cluster: OceanBase Powers Kwai`s Core Business
by teleforce on Saturday December 28, 2024 @04:33AM
(#42529760)
For clarification OceanBase is a distributed database offering from the Ant group from Alibaba that also owns and operates Alipay.
There's another related storage solutions by Huawei namely OceanStor but the database or storage engine is HPDA or Huawei High Performance Data Analytics [1].
[1] OceanStor Pacific Storage for High-Performance Data Analytics .
drastically simplifies scaling when you are dealing with only one machine with huge amount of RAM and maximum cores.
I think the most you can get now is 192x2 cores and double digit TB of RAM
but more economical is proably 128x2 cores with single digit TB of RAM
its pretty insane how much traffic you can throw at a single machine like this espeically if you use optimized languages like Go or Rust to handle traffic.
there's still use for distributed computing but 99.8% of websites do not really need it.
There's another related storage solutions by Huawei namely OceanStor but the database or storage engine is HPDA or Huawei High Performance Data Analytics [1].
[1] OceanStor Pacific Storage for High-Performance Data Analytics .
https://e.huawei.com/eu/topic/storage/high-performance-data-...