News for hackers. Stuff that matters.

DSOfficial

A ClickHouse outage triggered 200+ retries and a DolphinScheduler deadlock. Here’s how to troubleshoot, recover, and prevent task pileups.
How does Apache DolphinScheduler handle long-held task group slots? Explore consistency correction, normal release, and force start.
Master big data workflows with Apache DolphinScheduler on Ubuntu via Docker! Build & run visual DAGs effortlessly.
Run Apache DolphinScheduler with Spark on Kubernetes! Learn cloud-native deployment, shared storage, and production-ready best practices.
Re:Generalized OIDC Authentication in Apache DolphinScheduler on Thursday October 23, 2025 @03:00AM (#45679012)
Revolutionizing enterprise-grade workflow orchestration! Apache DolphinScheduler now has generalized OIDC authentication. Simplify user management, enhance security & integrate seamlessly with any identity provider. Check out the details.
Re: on Friday August 29, 2025 @03:15AM (#45061143)
Last week, the tech world was shaken by a headline-grabber: Databricks has surged past the $100 billion valuation mark.

And remember, this isn’t a consumer internet giant, nor is it SpaceX—it’s a software company that builds the data and AI infrastructure layer. In a capital market environment dominated by caution, Databricks has managed to defy gravity, pushing the ceiling of what a unicorn can be worth. The obvious question is: why? What exactly gives Databricks the right to this number? I did some digging. Let’s look at why Databricks is valued at $100 billion—and more importantly, what software entrepreneurs might take away from it. https://dev.to/chen_debra_3060b21d12b1b0/why-databricks-is-w...

Re:Apache SeaTunnel on Tuesday May 20, 2025 @11:21PM (#44047994)
Great news! Apache SeaTunnel has surpassed 8,500 #GitHub Stars! A big shout-out to our amazing community for the continued support and contributions. Join us on our journey to make data synchronization easier and more powerful! #OpenSource #BigData
Re: on Monday May 19, 2025 @10:57PM (#44037333)
Just extracted a lightweight, delay-capable EventBus from Apache DolphinScheduler! Supports delayed & event-driven execution Inspired by DolphinScheduler's elegant design Easy to plug into any Java system Dive in
Re: on Wednesday April 16, 2025 @04:06AM (#43702779)
Submission Deadline: April 21!

The countdown is on for #CommunityOverCode Asia 2025 CFP!

Got a story, project, or insight to share with the open-source world?

Submit your talk now

#Apache #OpenSource #TechConference #CFP

Re: on Wednesday April 16, 2025 @04:02AM (#43702729)
Experiencing CPU spikes with Apache DolphinScheduler? One team solved it instantly by tweaking memory & thread settings — no more crashes, even at peak load! #DolphinScheduler #BigDataOps #OpenSource #DevOpsTips
Re: on Thursday April 03, 2025 @03:50AM (#43566265)
Join #GSoC2025 with DolphinScheduler! Enhance OIDC authentication for a leading workflow orchestrator. Open to students&devs! Proposals: https://issues.apache.org/jira/browse/GSOC-284 Mentors: [email protected] / [email protected]
Re: on Thursday April 03, 2025 @03:09AM (#43565906)
Just dropped: Deep dive into Apache DolphinScheduler's Master Module! Discover how it handles billion-level tasks with slot calculation, command processing, and DAG workflows.
Re: on Thursday March 27, 2025 @10:53PM (#43500988)
@NetEaseMail slashes dev time by 67% with #DolphinScheduler! Handling 50k+ daily tasks with optimized workflows & 1-click recovery. Now eyeing AI-powered data governance & Ops automation. #BigData #DataEngineering Smooth migration guide: https://medium.com/@ApacheDolphinScheduler/netease-mails-mig...
Re: on Thursday February 13, 2025 @04:59AM (#43034427)
Once upon a time, when people in the tech circle mentioned DS, what came to mind was a lively “little dolphin” - Apache DolphinScheduler (abbreviated as DS). This distributed task scheduling system, born in 2019, was once the “dream job” for big data engineers with its visual DAG interface, multi-tenancy support, and deep integration with the Hadoop/Spark ecosystem. However, the wheels of fate began to spin rapidly in early 2025: Hangzhou AI company DeepSeek (Deep Quest) burst onto the scene with its inference model R1 and open-source large model V3, touting “low cost and high performance.” Not only did it sweep the capital market, but it also redefined the acronym “DS” overnight - from now on, “DS” was no longer a scheduling tool but the “new darling” of the AI circle.
Re: on Monday February 10, 2025 @03:28AM (#42998049)
The software landscape is transforming!Affordable large language models, like #DeepSeekR1, are shaking things up. The future belongs to the monkey-troop-like collaborative software ecosystem.
Re: on Monday December 23, 2024 @10:55PM (#42499635)
Intriguing insights on data workflow management! Dive into the detailed comparison of #DolphinScheduler and #SeaTunnel against #AirFlow and #NiFi. Find out the best fit for your big data challenges. Full article: https://dev.to/chen_debra_3060b21d12b1b0/dolphinscheduler-an... #DataEngineering #opensource
Re: on Thursday December 05, 2024 @01:52AM (#42325671)
Join the Apache DolphinScheduler community for an online meetup on December 17th, 2024, at 2:00 PM (GMT+8), as we explore Big Data Scheduling Technology across traditional and cutting-edge applications! Register now: https://www.slidestalk.com/DolphinScheduler/12_17 #Dolphinscheduler #BigDataScheduling #Meetup
Re: on Wednesday November 20, 2024 @01:08AM (#42191203)
Delve into Quartz, the mighty open-source Java framework for scheduling tasks, and its dynamic partnership with DolphinScheduler through QuartzExecutorImpl. Find out how they work together to orchestrate workflows and manage timings in our in-depth exploration.
Re: on Monday November 18, 2024 @09:37PM (#42179606)
An interesting #opensource #LLM project on #github, saving valuable time in the big data era
Re: on Tuesday November 12, 2024 @05:06AM (#42114234)
Explore how Apache DolphinScheduler enhances MapReduce task management! #ApacheDolphinScheduler #MapReduce #BigData #DataEngineering #opensource
Re: on Monday November 04, 2024 @09:52PM (#42048190)
Dive into integrating Arthas with DolphinScheduler to monitor specific API calls and analyze performance data. This setup enhances task scheduling reliability and maintainability—take your workflows to the next level! #DolphinScheduler #Arthas #PerformanceMonitoring
After several months, #ApacheDolphinScheduler is back on the GitHub Trending weekly list, now reaching 12,820 stars! Big thanks to the community for all the support and contributions!

Older »