91dylwyn
Re:Why Typhoon Instead of Airflow?
on Thursday October 27, 2022 @03:12PM (#33362535)
Is there a better way than Airflow?
After giving it much thought and having many lunch time conversations with my coworkers I became convinced that all my issues with Airflow stemmed from the same root cause: task isolation. What in theory looked like a good thing ends up hindering good software practices....
Ask HN: Delta Lake vs. others (e.g. Iceberg etc.)
Can anyone explain if the recent DeltaLake 2.0 open source from DataBricks is providing something much better than you get with Iceberg? Does it being open source provide much as you basically need DataBricks to run it properly don't you? When would you choose it over other options?
Re:Show HN: I made a website for you to travel the world in first-pers...
on Sunday April 10, 2022 @11:28AM (#30977901)
Maybe you could you make it so people can add a place. I guess upload to youtube, geo-tag it. It would be fun to get to add your place to the game.
I would like to try with a VR headset
Re:Show HN: I made a website for you to travel the world in first-pers...
on Sunday April 10, 2022 @11:23AM (#30977839)
This is pretty fascinating!!
Re:Ask HN: What creative field will be safe from AI?
on Sunday April 10, 2022 @10:55AM (#30977446)
AI is the best at chess but you would not pay to see two AI play chess while its still interesting to watch Magnus win or lose.
So, likely just because you can make AI works of art, it doesn't mean people will value them equally.
Re:Play chess, earn money
on Saturday January 29, 2022 @05:52PM (#30131427)
Somebody made money... its now $ 1,040
Did someone win?
Re:Play chess, earn money
on Saturday January 29, 2022 @05:51PM (#30131410)
If no ads and no member fee...where is the money from? I don't quite get it. Nice graphic tho.
Re:Show HN: Plotting New York using a collision dataset
on Saturday January 29, 2022 @05:04PM (#30131018)
Fun end picture of the city when opacity is set- nice!
Show HN: UI to allow your team to easily create Airflow DAGs with no code
Re:Don't you lecture me with your thirty dollar website
on Saturday January 29, 2022 @04:27PM (#30130665)
Love it! theuselessweb found me http://eelslap.com/
My fav from back of the day http://www.davesweboflies.com/
Re:Basic income would not reduce people’s willingness to work
on Monday January 17, 2022 @03:06PM (#29971133)
This seems much more sensible than extra money to those already comfortably off.
Re:Ask HN: Is it a good idea to use AWS Lambda for ETL Pipelines?
on Monday January 17, 2022 @01:34PM (#29970001)
Are you finding any pain points? How did you manage the 15 minute window in case of large amounts of data running over time?
How do you manage the DAGs on it? For example creating dependencies between tasks and monitoring and rerunning tasks?
Ask HN: Is it a good idea to use AWS Lambda for ETL Pipelines?
We are working on an OSS for ETL (Typhoon data orchestrator) and we are thinking about using AWS Lambda as the backend. The benefit is that it has a built in scheduler and we don't need to build our own. We can also spawn multiple Lambdas to scale concurrency.Do you think this would be a sensible option? Would the 15 minutes running time be a deal breaker?