News for hackers. Stuff that matters.

debrachen12

Re: on Thursday June 29, 2023 @03:24AM (#36517453)
Use Apache SeaTunnel, Milvus, and OpenAI, we can achieve more accurate book title similarity searches through large language models, bringing significant technological breakthroughs to the field of book search, while also providing valuable references for semantic understanding. I hope this can provide some inspiration to everyone.
Re: on Monday June 05, 2023 @02:49AM (#36193229)
Apache SeaTunnel, a cloud-native high-performance massive data integration tool, has recently graduated to become a Top-Level Project (TLP) of the Apache Software Foundation (ASF).
Re: on Wednesday May 31, 2023 @11:39PM (#36147346)
The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 open source projects and initiatives, announced today that SeaTunnel has graduated from incubation and is now a Top-Level Project (TLP). SeaTunnel is a cloud-native, high-performance, massive data integration tool initially created by open source community members, data experts, and developers in China.
How can I deploy it?
Awesome work!
Re: on Thursday November 24, 2022 @03:42AM (#33729038)
After analyzing and selecting the data synchronization requirements of hundreds of billions of data, Chinese Churinga(CNCR) eventually decided to use DolphinScheduler for task scheduling and also needed to periodically schedule DataX, SparkSQL, and other methods for massive data migration. They use DolphinScheduler in daily big data jobs to reduce the daily operations and maintenance workload.
we introduce the creation of a machine-learning workflow based on a YAML file. We also use DolphinScheduler to call Amazon SageMaker for a stock training and prediction system.
Re:SeaTunnel Will Support CDC as a Feature Soon on Thursday November 03, 2022 @11:42PM (#33461863)
CDC is crucial to data synchronization, and we need SeaTunnel to support it as a feature. Here below is the reason why SeaTunnel needs this feature and my overall design for it.
Is it open-sourced?
A good view on big data modern stack, very interesting.
Re: on Monday March 14, 2022 @04:47AM (#30669931)
Recently, Apache DolphinScheduler announced the official release of version 2.0.5. This version has carried out some functional optimizations, such as optimizing the fault tolerance process of Worker, adding the function of re-uploading files in the resource center, and making several bug fixes. https://medium.com/@ApacheDolphinScheduler/release-news-apac...
Re: on Monday March 07, 2022 @02:13AM (#30585013)
This week, two open-source projects under WhaleOps Tech’s operation have received frequent good news. The Apache Software Foundation (hereinafter referred to as ASF) held its annual member meeting and elected a new board of directors. Apache SeaTunnel Champion Jiang Ning was elected as a new director of ASF.

https://medium.com/@ApacheDolphinScheduler/good-news-apache-...

Re: on Monday February 21, 2022 @03:46AM (#30413365)
Hello the community! After having adjusted ourselves from the pleasure of Spring Festival and re-devote to the busywork, study, and life, Apache DolphinScheduler is here to wish everyone a prosperous and successful journey in the Lunar year of the Tiger. In this early spring, Apache DolphinScheduler’s first Meetup in 2022 is also coming soon. I believe that the community has already been looking forward to it! In this Meetup, four guests from our user companies will share their experience when using Apache DolphinScheduler. Whether you are a user of Apache DolphinScheduler or a sideliner, you can definitely learn a lot from it. We are looking forward to more friends joining us in the future, and more and more users will share practical experiences with each other. The first show of Apache DolphinScheduler in February 2022 will be broadcast on time at 14:00 pm on February 26th. You can also click https://www.slidestalk.com/m/679 to go directly to the live broadcast reservation interface!

For details of the schedule, please refer to https://medium.com/@ApacheDolphinScheduler/sign-up-to-apache...

Re: on Monday February 14, 2022 @09:57PM (#30342124)
The main purpose of E2E test is to verify the integration and data integrity of the system under test and its components by simulating real user scenarios, so as to expand the testing scope, ensure the health and stability of the system, and reduce testing workload and costs to a certain extent . Therefore, the community decided to conduct E2E automated testing.

For more details, please refer to https://medium.com/@ApacheDolphinScheduler/this-is-an-apache...

Re: on Monday January 17, 2022 @03:43AM (#29964593)
Kerberos is a computer network authorization protocol used to authenticate personal communications securely over a non-secure network.

This article will guide you through testing Kerberos-authenticated HIVE connections in DolphinScheduler from scratch, including the solutions to clear some obstacles encountered in real-world environments.

Details please refer to https://medium.com/@ApacheDolphinScheduler/testing-kerberos-...

0 0 Comments

Share

Re: on Wednesday January 05, 2022 @01:15AM (#29805784)
Hello, 2022! 2021 is the past, and we will set sail in 2022! In the past year, we are used to getting along with the COVID-19 pandemic However, the cold winter under the epidemic didn’t stop the enthusiasm for contribution from the community
Re: on Thursday December 30, 2021 @01:45AM (#29734298)
Based on the Apache DolphinScheduler, the cloud computing and big data provider Eavy Info has been serving the business operations in the company for more than a year. Combining with the government affairs informatization ecological construction business, Shandong Eavy Info built the data service module of its self-develop Asset Data Management and Control Platform based on Apache DolphinScheduler. How do they use Apache DolphinScheduler? Sun Hao, the R&D engineer of Evay Information, shared their experiences on their business practice.

Details refer to:https://medium.com/@ApacheDolphinScheduler/eavy-info-builds-...

Re: on Monday December 20, 2021 @10:12PM (#29633374)
Good news! Apache DolphinScheduler 2.0.1 version is officially released today! In this version, DolphinScheduler has undergone a microkernel + plug-in architecture improvement, 70% of the code has been refactored, and the long-awaited plug-in function has also been emphatically optimized. In addition, there are many highlights in this upgrade, such as a one-click upgrade to the latest version, “de-ZK” in the registration center, and new task parameter transfer functions, etc.
At the recent Apache DolphinScheduler Meetup 2021, Zheqi Song, the Director of Youzan Big Data Development Platform shared the design scheme and production environment practice of its scheduling system migration from Airflow to Apache DolphinScheduler. This post-90s young man from Hangzhou, Zhejiang Province joined Youzan in September 2019, where he is engaged in the research and development of data development platforms, scheduling systems, and data synchronization modules. When he first joined, Youzan used Airflow, which is also an Apache open source project, but after research and production environment testing, Youzan decided to switch to DolphinScheduler. How does the Youzan big data development platform use the scheduling system? Why did Youzan decide to switch to Apache DolphinScheduler? The article below will uncover the truth.
Founded in 2014, Bigo has aimed to provide audio and video services on a global scale since its establishment. Facing the huge data size generated by more than 400 million users, the Computing Platform constructed based on the Apache DolphinScheduler scheduling system by the Bigo big data team processes 7W+ jobs daily now.
Unicom Digital Tech Big Data Scheduling System Re-development Based on DolphinScheduler, one use case analysis on the big data scheduling system remold of our key user.Down pointing backhand indexDown pointing backhand indexDown pointing backhand index
Hello community, good news! After nearly 10 months of joint efforts by more than 100 community contributors, we are happy to announce the release of Apache DolphinScheduler 2.0 alpha. This is the first major version of DolphinScheduler since it entered Apache. It has undergone a number of key updates and optimizations, which means a milestone in the development of DolphinScheduler.
On October 22, 2021, we are excited to announce the release of Apache DolphinScheduler 1.3.9. After a month and a half,Apache DolphinScheduler 1.3.9 brings StandaloneServer to users with the joint efforts of the community. StandaloneServer is a major update of this version, which means a huge leap in ease of use, and the details will be introduced below. In addition, this upgrade also fixes two critical bugs in 1.3.8.
Looking forward to Apache DolphinScheduler 1.3.9!