ssk2

Databricks | Onsite | SF or MTV | Full-Time

We have an open headcount on my team at Databricks (Compute Lifecycle) for a senior or staff engineer! Typically this means 5+ years of experience. For this role, we're also looking for experience with distributed systems (and ideally experience running Kubernetes at scale). We're based out of San Francisco and Mountain View. Databricks has returned to office, so applicants must be able to commute to one of these two offices at least two days a week.

Databricks started out building a platform based on Apache Spark. Over the last few years, we've built an all encompassing platform for enterprises to make use of all structured and unstructured data. This includes tools to allow all users (technical or not) to quickly query and visualise data. Lately, we've integrated machine learning and generative AI technologies into the product - acquiring MosaicML last year, a leader in LLM infrastructure.

Our team maintains several systems that automate cluster provisioning at scale and allow us to manage hundreds of Kubernetes clusters that power Databricks' serverless products across all 3 major clouds. This includes heavy collaboration with MosaicML to support our new GenAI products.

Tech stack wise, we primarily use Go (for Kubernetes facing components) and Scala for Databricks' internal services. We're one of the biggest users of Cluster API, EKS and AKS.

This team is one of the strongest I've been fortunate to work with and the complexity of the environments we work in and the products we support means that there's a lot to learn!

If you're interested and could be a fit, please send me a resume at [email protected].

(Unfortunately I am not involved with university recruiting and cannot refer candidates for these roles - please keep an eye on https://www.databricks.com/company/careers/university-recrui... for those.)

Airbnb | Remote (North America) | Staff/Senior Software Engineer, Engineering Manager | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We joined the CNCF end user community last year, and shared our perspective on the Kubernetes ecosystem as part of a panel discussion [1].

I’m specifically helping hire for our Cloud Infra team, looking for engineers with experience with: + Kubernetes + Istio/other service meshes + Golang + Disaster recovery + Large scale infrastructure (think Borg, or Tupperware) + Load balancing and edge/cloud networking + CDNs

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply on our site [2]. Note that most of these Cloud roles are not yet listed on our site. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

[1] https://www.infoq.com/articles/kubernetes-trends-and-challen...

[2] https://airbnb.com/careers

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We are members of the CNCF end user community, and I recently shared my perspective on the Kubernetes ecosystem as part of a panel discussion [1].

I’m specifically helping hire for our Cloud Infra team, looking for engineers with experience with:

+ Kubernetes

+ Istio/other service meshes

+ Golang

+ Disaster recovery

+ Large scale infrastructure (think Borg, or Tupperware)

+ Load balancing and edge/cloud networking

+ CDNs

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply on our site [2]. Note that most of these Cloud roles are not yet listed on our site. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

[1] https://www.infoq.com/articles/kubernetes-trends-and-challen... [2] https://airbnb.com/careers

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We recently joined the CNCF end user community, and I recently shared my perspective on the Kubernetes ecosystem as part of a panel discussion [1].

I’m specifically helping hire for our Cloud Infra team, looking for engineers with experience with:

+ Kubernetes

+ Istio/other service meshes

+ Golang

+ Disaster recovery

+ Large scale infrastructure (think Borg, or Tupperware)

+ Load balancing and edge/cloud networking

+ CDNs

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply on our site [2]. Note that most of these Cloud roles are not yet listed on our site. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

[1] https://www.infoq.com/articles/kubernetes-trends-and-challen...

[2] https://airbnb.com/careers

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports Airbnb's mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We have positions open for staff and senior staff engineers across the following teams:

+ Data Platform

+ ML Infrastructure

+ Developer Productivity

+ Cloud Infrastructure

+ Search Infrastructure

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply here: https://airbnb.com/careers. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We recently joined the CNCF end user community, and I recently shared my perspective on the Kubernetes ecosystem as part of a panel discussion [1].

I’m specifically helping hire for our Cloud Infra team, looking for engineers with experience with:

- Kubernetes

- Istio/other service meshes

- Golang

- Disaster recovery

- Large scale infrastructure (think Borg, or Tupperware)

- Load balancing and edge/cloud networking

- CDNs

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply on our site [1]. Note that most of these Cloud roles are not yet listed on our site. If you're interested in learning more, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in (if one exists online), and a resume to [email protected].

[1] https://www.infoq.com/articles/kubernetes-trends-and-challen... [2] https://airbnb.com/careers

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We recently joined the CNCF end user community, and I recently shared my perspective on the Kubernetes ecosystem as part of a panel discussion [1].

We have positions open for senior, staff and senior staff engineers across the following teams:

+ Data Platform

+ ML Infrastructure

+ Developer Productivity

+ Cloud Infrastructure

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply on our site [2]. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

[1] https://www.infoq.com/articles/kubernetes-trends-and-challen...

[2] https://airbnb.com/careers

The Verge reported on this [1] in 2017-

> Bloomberg says that early staffers “had an unusual compensation system” that multiplied staffers' salaries and bonuses based on the performance of the self-driving project. The payments accumulated as milestones were reached, even though Waymo remains years away from generating revenue. One staffer eventually “had a multiplier of 16 applied to bonuses and equity amassed over four years.” The huge amounts of compensation worked — for a while. But eventually, it gave many staffers such financial security that they were willing to leave the cuddly confines of Google.

[1] https://www.theverge.com/2017/2/13/14599186/google-waymo-sel...

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We have positions open for senior, staff and senior staff engineers across the following teams:

+ Data Platform

+ ML Infrastructure

+ Developer Productivity

+ Cloud Infrastructure

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply here: https://airbnb.com/careers. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports our mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We have positions open for senior, staff and senior staff engineers across the following teams:

+ Data Platform

+ ML Infrastructure

+ Developer Productivity

+ Cloud Infrastructure

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply here: https://airbnb.com/careers. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports Airbnb's mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering organisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We have positions open for senior, staff and senior staff engineers across the following teams:

+ Data Platform

+ ML Infrastructure

+ Developer Productivity

+ Cloud Infrastructure

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply here: https://airbnb.com/careers. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

Airbnb | Remote (preferably West Coast), Seattle, WA or San Francisco, CA | Senior Staff/Staff/Senior Software Engineer | Full-Time | airbnb.com

Airbnb Infrastructure is hiring for several key positions - we're looking for talented engineers across a wide variety of domains to build the platform that supports Airbnb's mission to create a world where anyone can belong anywhere.

Airbnb is at a pivotal point in its growth and maturity as an engineering orgnaisation. We run some of the largest Kubernetes clusters in the industry, have open sourced some extremely successful data tools (e.g. Superset), and are rapidly adopting newer technologies like Flink and Istio.

We use Terraform, AWS and Chef extensively. My team (Cloud Infra > Compute Infra) mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

We have positions open for senior, staff and senior staff engineers across the following teams:

* Data Platform

* ML Infrastructure

* Developer Productivity

* Cloud Infrastructure

Note that these positions are for experienced engineers and it helps if you have 5+ years of experience and/or have acted in a similar capacity in prior roles.

Check out all open roles and apply here: https://airbnb.com/careers. If preferable, you can also send me a brief note about why you'd be a good fit, a link to the role you're interested in, and a resume to [email protected].

Airbnb | Remote (preferably West Coast) or San Francisco, CA | Senior Staff/Staff Software Engineer | Full-Time | airbnb.com

My team (Cloud Infrastructure) is hiring for a staff and a senior staff level software engineer to focus on our internal platform as a service built upon Kubernetes. Over the last three years, this has progressed from a pilot project to running almost all of our production services at immense scale.

Besides Kubernetes, we are in the process of rolling Istio out, and use Terraform, AWS and Chef extensively. Our team mostly uses Golang but it helps to have familiarity with (or a lack of fear of) Java and Ruby, which most of our product teams use.

Airbnb has a large number of engineers and users, which creates interesting technical and organisational challenges.

The job opening for senior staff engineer is here (and a staff posting will follow soon): https://careers.airbnb.com/positions/2245198/

Note that these positions are for highly experienced engineers and it helps if you have 8+ years of experience and/or have acted in a similar capacity in prior roles.

Please send me a brief note and resume at [email protected] or apply online.

When I left last August, we had around 600 in engineering, around 900 in product and engineering.

Engineering was mostly based in SF but with satellite offices in London and Hamburg. It looks like they were expanding in a large way into a Toronto office too.

Infrastructure was around a fifth of that total but included verticals like search and machine learning infrastructure.

Yelp turned off their last datacentre in early 2018 (IIRC) - and is entirely on AWS.

Airbnb | Staff Software Engineer, Cloud Infrastructure | Full-time | Onsite | San Francisco/San Jose, California

Airbnb is looking for experienced staff level software engineers to join our Cloud Infrastructure team. Cloud Infrastructure is a foundational team that manages the compute clusters, cloud deployments and service discovery infrastructure used to serve traffic to the millions of global Airbnb users. We work with approximately 1,500 product engineers within Airbnb and run many tier 0 services that have strict requirements for scale and robustness.

I've worked at a few companies over my career so far and it was surprising to me when I joined Airbnb last year that we have managed to hold onto such a strong sense of mission and such a welcoming culture. For a company of thousands of people, that is no easy feat.

On the engineering side, we run some of the largest Kubernetes infrastructure by footprint in the industry and that comes with some fascinating technical and organisational challenges.

You can apply online directly here: https://careers.airbnb.com/positions/2103004/, or email me at [email protected].

Mostly time. For workloads that are pseudo interactive (e.g. continuous integration), you're saving developers a 10-20 minutes here or there. That can add up to be quite a bit for a medium to large organisation.

Scaling happens at two levels: both for the deployments and for the size of the cluster itself. Clusterman operates on the cluster itself.

If you scale just the number of pods, if there isn't enough available capacity in the cluster, they are unable to deploy until that capacity brought online. By emitting a signal to increase the number of nodes in the cluster just before we think that capacity is about to be needed, we can ensure that the new pods are launched near instantly when the deployment is actually scaled up. This is mostly useful when you're scaling by large increments (i.e. hundreds of pods) that far exceed the spare capacity available in your cluster.

Mother's Day was a surprising one for me! A lot of people take their parents out for brunch. Same goes for Valentine's Day.

Small point of clarification: the DC/OS SDK works well for these applications when running on DC/OS. It did not work well when run on open source Mesos, which is what Yelp uses.

Usually around dinner time in our big metro areas :)

Former Yelp engineer here (I left recently). Clusterman was built about ~18 months ago for our primarily Mesos and Marathon based workloads. At the time, the Kubernetes cluster autoscaler was in its infancy and there was no comparable solution that worked for Mesos.

Yelp is currently migrating to Kubernetes which is why support for Kubernetes autoscaling was recently added to Clusterman.

Yelp | SRE/Infrastructure Engineer/Engineering Manager | Full Time | San Francisco, CA | Onsite / Remote in North America | https://yelp.com

Yelp is a popular review website with over 60 million monthly active users on our desktop site alone.

We're looking for SREs and Infrastructure engineers to join our Production Engineering and Compute Infrastructure teams. We're also looking for an engineering manager for Compute Infrastructure. Remote is fine for ICs but we'd prefer the manager is onsite.

I'm one of the hiring managers on the Compute Infrastructure team. We're a diverse group of ~15 spread across California, London and Hamburg. We handle the clusters that run the stateless services that power the website, along with all of our Spark and batch processing infrastructure. Technologies we use include: Mesos and Marathon (we're one of the biggest open source users of Marathon), Kubernetes, Docker, Terraform and Puppet. Work on our team is very high impact and supports a large number of both internal and external users.

I'd love to talk to you if you're interested in a role on our team and have either experience in distributed systems, reliability engineering or any of the specific technologies we mention. Right now we're looking for engineers with 2 or more years of experience to join our team[1]. If you're interested, please drop me an email to [email protected] with a brief note about why you're interested in this role and a resume.

[1] If you have recently graduated, we'd love to consider you for our university program: https://www.yelp.com/careers/teams/college-engineering

Yelp | SRE/Infrastructure Engineer | Full Time | San Francisco, CA | Onsite / Remote in North America | https://yelp.com

Yelp is a popular review website with over 60 million monthly active users on our desktop site alone.

We're looking for SREs and Infrastructure Engineers to join our Production Engineering and Compute Infrastructure teams.

I'm one of the hiring managers on the Compute Infrastructure team. We're a diverse group of ~15 spread across California, London and Hamburg. We handle the clusters that run the stateless services that power the website, along with all of our Spark and batch processing infrastructure. Technologies we use include: Mesos and Marathon (we're one of the biggest open source users of Marathon), Kubernetes, Docker, Terraform and Puppet. Work on our team is very high impact and supports a large number of both internal and external users.

I'd love to talk to you if you're interested in a role on our team and have either experience in distributed systems, reliability engineering or any of the specific technologies we mention. Right now we're looking for engineers with 2 or more years of experience to join our team[1].

If you're interested, please drop me an email at [email protected] with a brief note about why you're interested in this role and a resume.

[1] If you have recently graduated, we'd love to consider you for our university program: https://www.yelp.com/careers/teams/college-engineering

DC/OS has a unique packaging mechanism, and cannot be found as a traditional Linux package. (Personally, I'd love to see that happen at some point but for simplicity's sake, it's distributed as a single binary - this allows us to upgrade the bits on nodes near atomically).

There are three ways to install it: 1) Cloud provider specific templates (e.g. CloudFormation / Azure Resource Manager. GCP support is on the backlog)

2) The ssh installer (this has a UI and a CLI). I believe this doesn't work well for larger installations because of the one to many issue.

3) The advanced install method. This generates a binary that is copied to each node. You can integrate this with Puppet, Chef, Ansible and so on.

https://dcos.io/install/ and https://dcos.io/docs/ has more details on this.