willpeter
Re:A high-performance, distributed, data integration framework
on Sunday October 30, 2022 @11:31AM (#33395302)
Easy-to-use tool to quickly sync or transfer large amounts of data based on Flink or Spark
Re:The modern data workflow orchestration platform – Apache DolphinSch...
on Wednesday October 26, 2022 @05:04AM (#33341323)
Apache DolphinScheduler is a cloud-native platform with powerful visual interfaces. Compared with other open source orchestration platform, it has lower latency, higher concurrency and more stability, which has been produced and used by more than 1000 companies, including IBM, Accenture, Walmart, VMware, SheIn, Cisco.
The key features for DolphinScheduler are as follows:
- Easy to deploy, we provide 4 ways to deploy, such as Standalone deployment,Cluster deployment,Docker / Kubernetes deployment and Rainbond deployment
- Easy to use, there are 3 ways to create workflows:
- Visually, create tasks by dragging and dropping tasks
- Creating workflows by PyDolphinScheduler(Python way)
- Creating workflows through Open API
- Highly Reliable,
DolphinScheduler uses a decentralized multi-master and multi-worker architecture, which naturally supports horizontal scaling and high availability- High performance, its performance is N times faster than other orchestration platform and it can support tens of millions of tasks per day
- Supports multi-tenancy
- Supports various task types: Shell, MR, Spark, SQL (MySQL, PostgreSQL, Hive, Spark SQL), Python, Procedure, Sub_Workflow, Http, K8s, Jupyter, MLflow, SageMaker, DVC, Pytorch, Amazon EMR, etc