sumbry

Having spent a chunk of my career in Platform Engineering, I feel what platform teams are about to go through. They're going to get stuck running self-hosted Inference workloads and it's going to come out of the blue. The training/inference worlds are completely different than what we're used to with very deep complex stacks and technologies that you'll have to come up to speed with very quickly.

The catalyst will probably come from outside of Engineering; finance asking for ways to control token spend, Security asking for more control over our data, even engineers asking for more control over model output (and stability of tools/harnesses,etc).

Either way the day of reckoning is coming. We're doing this ourselves at Upbound and decided to share our journey along the way. I do think SRE for Models is going to pop-up at some point as we learn how to reliably operate these things at scale but as we've already learned, getting the model up and running is the easy part.

SMS refuses to die because it's the serial console of the internet.

It is pretty common knowledge why they did this. Original plans for the BART had it traveling over the Golden Gate Bridge into Marin County. Due to high winds across the bridge, they wanted a wider gauge to prevent trains from tipping.

If you do the 5 why's and are honest you'll discover the real root cause is not having a backup and restore procedure that is programmatic.

All outages are blameless. It's always a process failure or lack of a proper system or tool.

If you're seriously worried about the NSA logging things - I'd be more concerned with the device you carry on your person every day, with a microphone, two cameras, a GPS device, 24 hour battery, and probably 10x the processing power of the Alexa.

It seriously is no worse than OK Google or Siri and you should seriously be more worried about your cell phone.

The questions I ask are designed to help me learn about the actual environment I'd potentially be walking in to. Understanding the technical parts are easy, grok'ing the environment is a bit tougher. You're making a decision for potentially years after only a few hours of input.

1. If there is one thing you could get the company to stop doing today, what is it?

2. On that note you're in tech, there are numerous opportunities everywhere, what are the top three things that keep you here?

3. What's the job you want after this one? Why? How is the company helping you get there?

4. What do you and your manager discuss in your 1:1s? What do you and your reports discuss in your 1:1s?

5. Name a mistake you made in the past year. What did you learn from it? How did your team and/or manager respond?

6. Same question as above but name a success.

Someone once said that the public cloud was never about virtualization but rather automation. I would contend that the container revolution we're going through now isn't really about containers but ultimately distributed system platforms. And if it changes the way many have traditionally approached problems great!

My main point is that we're really just at the beginning of all of this. I'm not picking on Docker (I'm actually embracing containers) but rather pointing out that many of these next gen solutions are only tackling the basic use cases right now. While I'm excited there is so much more work left to be done. Implementing many of these solutions still requires a bunch of engineering effort and I'd like to see more turnkey solutions. The developer end is definitely getting easier and more productive but the operational end is getting more complex and still not solving some use cases everyone has. How many different ways can different companies implement HA MySQL for example?!

There are tons of other platforms out there that have recognized and solved many of this class of problems for years but the Cloud and microservices are actually starting to make this worse as adoption skyrockets. Platforms are not really a new thing, we went through this with JVM Platforms a ~decade ago, Heroku style PaaS ~five years ago, and now containers + cloud today.

I guess this is what progress looks like :)

We're undertaking a similar project and the latency is almost negligible. In fact the latency is lower bridging between classic and vpc in the same availability zone than between two classic availability zones.

Truckers don't like to change gears. It's much more efficient for them to stay in a lower gear and coast at a consistent speed than it is to constantly change gears and brake.