dianamp
Re:High-quality data is the best compute multiplier
on Friday October 09, 2026 @02:06PM (#50024447)
Yeah! This is so underrated relative to the overall impact.
Re:Data Wrangling at Slack
on Wednesday December 07, 2016 @11:51PM (#13128477)
Good luck and have fun! :D
Re:Data Wrangling at Slack
on Wednesday December 07, 2016 @11:32PM (#13128417)
We've considered HDFS, but we really liked the idea of having compute only clusters and have our data kept completely separate. Clusters failure happen and having data on S3 makes us worry less if a cluster goes down. Just spin up a new one and you're good to go.
There is a bit of more latency when using S3 compared to HDFS, but it's not bad and the benefits overcame that. We do have a couple of jobs that store some intermediate results in HDFS, but in the end everything lands in S3.
We encountered a few issues with S3 at the beginning mostly around the eventual consistency, but nothing that could not be fixed.
Data Wrangling at Slack
Bye Bye Vine
Re:USCIS received over 236,000 H-1B petitions
on Tuesday April 12, 2016 @07:44PM (#11484563)
So true! Been there myself! And have friends that are getting told by big tech companies that they don't do H1Bs anymore and prefer to do L1s and people working from their Europe offices.
Re:Alluxio, formerly Tachyon hits 1.0
on Tuesday February 23, 2016 @04:56PM (#11162674)
Nice!
JAWS – Spark SQL REST Server opensourced by Atigeo
Re:Ask HN: Who is hiring? (December 2013)
on Tuesday December 03, 2013 @09:31AM (#6840440)
Hi, the backend developer position sounds very interesting, by any change do you accept h1b?