Ask every enterprise that existed prior to 2018? Even for small and mid-size enterprises, manual deploys aren't that huge of a deal. Automation with some bash scripts can get you pretty far. Especially if you're only managing a monolith or two. I've worked in some platforms that absolutely defied automation due to their proprietary nature. It just meant that we'd need a few people to spend 1-2 hours every two weeks to run a deployment. That cost added up over even a year or two is probably less than paying for a redesign and replatform.

I've also done deploys just using managed services like ECS/Fargate or Heroku where we just build an artifact and push it to the host with a script. A stack that's a load balancer, stateless app server then a DB and/or file store can be provisioned manually once then not really worried about for a long time.

To be clear, I was more asking how people achieve the sort of fast bootstrapped dev environments the parent comment describes using k8s, especially where you’re not necessarily talking about simple topologies like a monolith plus a database.

Option 1: Everyone develops on the CI/CD platform.

Ops gives devs "disposable environments" to do their testing in. Basically, GitHub + Jenkins + Terraform + AWS. I've used this to stand up real infrastructure every time a PR is opened, and pipelines run against the infra. Code and infra match each other because they both exist in the same branch/repo (monorepo; you can of course do multi-repo, just takes more coordination). It's all destroyed as soon as the PR is closed, but you can also keep it up to do dev work against. You can also keep one copy of the latest master branch up at all times (the "dev" or "cert" or "test" infrastructure) as a shared test environment. Downside: you have to have access to the network (which is in AWS, so that's not so difficult). Upside: the dev environment always mimics production, devs don't need to do anything to stand it up.

Option 2: A bunch of mocks, and scripts to stand up stuff in Docker, using docker-compose or something else. This becomes a bit of a problem to manage with lots of devs, though, and docker-compose is not a good model for production deployments.

Thank you for describing option 1, which is really what I was picturing but seemed ambitious enough that maybe nobody tried it.

Tilt is pretty great for dev workflows on k8s, they have affordances for debuggers, fast restarts without a full docker build, docker push, k8s delete pod workflow.

https://tilt.dev

Depends on what cloud services you use. I am very opposed to using any cloud service that I can't self host in some way or another. That's a sure fire way to end up screwed. The main cloud services I use at work are:

    1. S3/Azure Blob Store
    2. DocumentDB/CosmoDB/Cloud Atlas
    3. Aurora/A for PG DB
    4. ElastiCache/some complex name in azure

All of these data stores can be hosted locally using minio for s3, MongoDB for the mongo implementations, and postgres/mysql for the RDBMSs, redis/memcached for ElastiCache.

If there was ever a case that I needed to ever use a real - full vendor lockin - solution I would hide it behind an API of my own and configure that API to store data locally. For example, the Azure Blob Store local container is completely broken for any sensible usage and I've been on an github issue thread for ~3 months waiting for them to resolve the issue (you're forced to link your netns to that container to talk to it as the application clients only works over 127.0.0.1). So essentially I have the following settings:

    - SERVICE_STORAGE_DRIVER=azure/fs
    - SERVICE_FS_LOCATION=/data
    - SERVICE_AZURE_CONTAINER=....
    - SERVICE_AZURE_....=....

So in development it runs in fs mode, in prod it runs in azure mode, you mock the azure blob client in unit test to make sure that code works.

Immutable infrastructure using Nix/NixOS.

I have used this. It has its benefits but you have to be ready to walk into alpha-ville (alpha-quality stuff) and take the pain that comes with it.

NixOS, however, I will not recommend as your dev environment. Use a regular distro and use NixOS for your containers.

Baroque organically grown scripts of arcana.

This is the correct answer for 99% of brownfield projects, especially those that didn't begin life on a cloud provider.

Define the same scripted startup on whatever floats you boat and run it.

We used to be able clone 400 VMs in 1 hour, sanitize all customer data and refresh a full dev environment from m prod in an hour. TBs of data.

But let's all stand around and claim k8s is not your flavor of the day solution to a very old problem, but one that caries a huge amount of complexity and so much network effry that it scares the pants off most people.

I use a git repo and pull in the changes to an ec2 instance. We also use git repos for our lambda functions.

Also our lambda functions are managed by bash scripts we created to interact with aws cli.

Everything is packages in RPM, so simple dnf install on bare metal or in container is enough.

Use Pulumi or Terraform.