I have used remote dev machines just fine, but my workflow vastly differs from many of my coworkers: terminal-only spacemacs + tmux + mosh. I have a lot of CLI and TUI tools, and I do not use VScode at all. The main GUI app I run is a browser, and that runs locally.

I have worked on developing VMs for other developers that rely on a local IDE such. The main sticking point is syncing and schlepping source code (something my setup avoids because the source code and editor is on the remote machine). I have tried a number of approaches, and I sympathize with the article author. So, in response to "Devs need to create the software tooling to make remote dev less painful. I mean, they're devs... making software is kind of their whole thing." <-- syncing and schlepping source code is by no means a solved problem.

I can also say that, my spacemacs config is very vanilla. Like my phone, I don't want to be messing with it when I want to code. Writing tooling for my editor environment is a sideshow for the work I am trying to finish.

Me as well, specially in the days that there was only a UNIX dev server for everyone.

It was never an issue to use X Windows on them, with hummingbird on my Windows thin client.

I guess a new generation has to learn the ways of timesharing development.

I am hardly a dev but occasionally have had to do some or some scripting or web stuff and have really loved VSCode and using the remote SSH support to basically feel like I’m coding locally. Does that not work for your devs?

That puts a "scary" VSCode blob on the remote-server. Some orgs do not like that, even if it's a "work" class box.

UNIX, and other competing timesharing systems of the time, have always been remote first, with Windows catching up with Citrix, followed by RDP, and nowadays finally headless as well.

Nowadays Web frontends and SSH/cloud shell, have replaced what used to be X Windows / telnet / rsh, but the underlying workflows aren't much different than running an IDE / emacs /vi / joe /... from a UNIX development server in a 1990's office.

The funniest (?) thing to me about all this: we're still hoping, if we do things right, to replicate the technology (terminals) from 50 years ago.

I honestly don't understand why nobody has simply invented some software to solve this problem, after 50 years.

Add to it the whole TUI fashion, as if we weren't doing them already 50 years ago.

We did, it is called GUI and language REPLs, like Smalltalk and Interlisp-D development enviroments, with graphical based terminals, not dependent on replicating virtual teletypes.

Still something that seems problematic to take off the way it should.

We have a project which spawns around 80 Docker containers and runs pretty OK on a 5 year old Dell laptop with 16GB RAM. The fans run crazy and the laptop is always very hot but I haven't noticed considerable lags, even with IntelliJ running. Most services are written in Go though and are pretty lightweight.

> Most services are written in Go though and are pretty lightweight

That's probably the difference. Throw elasticsearch, kafka and a bunch of Java services in and you'll be easily exhausting your RAM (at least at startup).

16GB of RAM is hardly a gargantuan amount in 2024 - it's not unreasonable to expect someone running a local dev environment to have a more practical amount. I wouldn't buy (or recommend) any machine for dev work with less than 64GB in 2024.

>I wouldn't buy (or recommend) any machine for dev work with less than 64GB in 2024.

Honestly, I think it depends on what kind of languages/frameworks/tools you use. Go is very lightweight, and you can safely run hundreds of Go services under 16 GB no problem (and with a crappy CPU, too). Python/Java/PHP etc. on the other hand, are much more wasteful. Our Go shop only now is considering maybe buying 32GB dev machines...

It has OpenSearch, RabbitMQ, Redis, MySQL

> the stack always grows to the point that a dev can no longer test it on their own machine

So the solution here is to not have that kind of "stack".

I mean, if it's all so big and complex that it can't be run on a laptop then you almost certainly got a lot of problems regardless. What typically happens is tons of interconnected services without clear abstractions or interfaces, and no one really understands this spaghetti mess, and people just keep piling crap on top of it.

This leads to all sorts of problems. Everywhere I've seen this happen they had real problems running stuff in production too, because it was a complex spaghetti mess. The abstracted "easy" dev-env (in whatever form that came) is then also incredibly complex, finicky, and brittle. Never mind running tests, which is typically even worse. It's not uncommon for it all to be broken for every other new person who joins because changes somewhere broke the setup steps which are only run for new people. Everyone else is afraid to do anything with their machine "because it now works'.

There are some exceptions where you really need a big beefy machine for a dev env and tests, maybe, but they're few and far between.

> So the solution here is to not have that kind of "stack".

Reminds me of my favorites debugging technique. It's super fast: Don't write any bugs!

Code is a liability. Not writing code is one of the best things a developer can do on any particular day, aside of course from deleting it ;)

Quite an effective approach:

https://github.com/kelseyhightower/nocode

What a boring trite reply. All of this is analogous to badly written spaghetti code. And yes, you can absolutely avoid all of this if you know what you're doing.

Not trite at all. Bad code and bad architectures are a reality. You can fix them in theory, but that takes a lot of time and needs to be done incrementally. In the meantime, you have to live with the problem at hand.

That kind of mess sounds super dangerous from a production perspective too.

With things that messy it's fairly likely there would be dependency loops or problems (thundering herd, etc) trying to get things going from a cold start.

ie after a complete outage or similar for whatever reason

"Most teams/products I have been involved in, the stack always grows to the point that a dev can no longer test it on their own machine"

Isn't this problem solved by CICD? When the developer is ready to test, they make a commit, and the pipeline deploys the code to a dev/test environment. That's how my teams have been doing it.

This turns a 1 hour task into a 1 day task. Fast feedback cycles are critical to software development.

I don't quite understand how people get into the situation where their work can't fit on their workstation. I've worked on huge projects at huge tech companies, and I could run everything on my workstation. I've worked at startups where the CI situation was passing 5% of the time and required 3 hours to run, that you can now run on your workstation in seconds. What you do is fix the stuff that doesn't fit.

The most insidious source of slowness I've encountered is tests that use test databases set to fsync = on. This severely limits parallelism and speed in a way that's difficult to diagnose; you have plenty of CPU and memory available, but the tests just aren't going very fast. (I don't remember how I stumbled upon this insight. I think I must have straced Postgres and been like "ohhhhhhhhh, of course".)

It's likely you haven't come across these use cases in your professional career, but I assure you its very common. My entire career has only seen projects where you need dozen to hundreds of CPU's in order to have a short feedback loop to verify the system works. I saw this in simple algorithms in automotive, to Advanced Driver Assistance Systems and machine learning applications.

When you are working on a software project that has 1,000 active developers checking in code daily and require a stable system build you need lots of compute.

There's a lot of folks in startups who think 100 devs is a large org and can't comprehend the scale at which '100% tests pass' stops being a build blocker. I've migrated from such an org to a late stage startup and 'tests must pass' even if fifty engineers are blocked with their PRs and the release train is fully halted. 'But our pipelines must be green' no they don't, at least not all of them.

You just need faster tests. ;)

Also, if you're booting kernel or device drivers you need the hardware. Some of this is not desktop hardware.

When you want to run the whole test suite, yes.

When you're developing and only need to touch 0.1% of the product and 0.001% of the code, that's a total and complete waste of time.

Really? I can't imagine not running the code locally. Honestly, my company has a micro services architecture, and I will just comment out the docker-compose pieces that I am not using. If I am developing/testing a particular component then I will enable it.

How tightly coupled are these systems?

The stack(factory) must grow.

> Most teams/products I have been involved in, the stack always grows to the point that a dev can no longer test it on their own machine, regardless of how big the machine is.

It doesn't have to be like that. I've worked on a 10MLOC codebase with 500+ committers - all perfectly runnable locally, on admittedly slightly beefy dev machines. It's true that systems will grow without limit unless some force exists to counter this, but keeping your stack something you can sanely run on a development machine is well worth spending some actual effort on.

It depends on the application. I've worked at places where the main application being written needs 128GB to run. Another where it's computationally intense and will take hours for a single run on anything but the most massive machines. Another where the dataset is huge and simply being remote makes the bandwidth requirements untenable unless you live across from a data center. I've seen teams just accept their workflow will take 15 minutes each time they run their app because they're copying stuff from remote each time. Remote dev gives you as beefy a machine as you want, with the fastest network and lowest bandwidth.

But that's only the resource problem. Another problem I have seen my entire career, is devs can't keep their machines configured the same. They have different model laptops, they don't pin their app versions, they configure and install things by hand. Each time they change something by accident it takes them hours, days, sometimes weeks, to get it working again. That also can lead to bugs developing the app, which wastes a huge amount of time.

And then there's the fact that their local copy runs completely differently than it does in production. This leads to the app being written with certain assumptions about how it runs, that turn out to be false in production. I've seen this lead to catastrophe, as well as just weeks to months of wasted time, trying to track down issues. This is an undeniable, existential issue.

Finally, it's rare for local setups to be secure. Often devs get too much access from their local machines, and this is stolen by infostealer malware and compromise happens. A protected remote environment is easier to secure. A lot of development is hampered by all the crappy corporate security tools that's on laptops now. Remote dev allows you to bypass all that and have a fully working yet protected network without restrictions.

Is remote dev a pain? Right now, yeah, because nobody has made it be less painful. So of course it's easier on the local machine. But it's not ideal. I'm sure eating with a spoon was more painful than eating with your hands, until forks were popularized in the 18th century. Change took a long time, but I think most of us prefer the change once new tools became widely available.

> Remote dev gives you as beefy a machine as you want, with the fastest network and lowest bandwidth.

Yes and no. Realistically, the range between the beefiest possible remote server and the beefiest possible workstation is what, one order of magnitude? So in a growing environment doing remote dev will maybe let you kick the can down the road a year or two, but you'll still have to deal with whatever was causing your requirements to grow pretty soon.

> But that's only the resource problem. Another problem I have seen my entire career, is devs can't keep their machines configured the same. They have different model laptops, they don't pin their app versions, they configure and install things by hand. Each time they change something by accident it takes them hours, days, sometimes weeks, to get it working again.

> Finally, it's rare for local setups to be secure. Often devs get too much access from their local machines, and this is stolen by infostealer malware and compromise happens. A protected remote environment is easier to secure. A lot of development is hampered by all the crappy corporate security tools that's on laptops now. Remote dev allows you to bypass all that and have a fully working yet protected network without restrictions.

This isn't a remote versus local question, it's a question of how much control developers have over their environment and how much you manage and standardise what's installed. You can have a fully locked down local machine where developers can't install anything except a short whitelist (of course you may get some pushback) and you can have a remote VM where developers curl|sh whatever random repack of Python they wanted this week - I've seen both these things happen in practice.

Security junkware I sort of agree with you, but I think that's more of an artifact of bad laws/policies and if and when remote dev takes off we'll see just as much junkware on remote dev machines as on local ones.

> the stack always grows to the point that a dev can no longer test it on their own machine

Sounds like you have a different problem.

CPU resources required to run your stack should be very minimal if it's a single user accessing it for local testing idle threads don't consume oodles of cpu cycles to do nothing.

Memory use may be significant even in that case (depending on your stack) but let's be realistic. If your stack is so large that it alone requires more memory than a dev machine can spare with an IDE open, the cost of providing developers with capable workstations will pale in comparison to the cost of running the prod environment.

I have a client whose prod environment is 2x load balancer; 2x app server; 3x DB cluster node - all rented virtual machines. We just upgraded to higher spec machines to give headroom over the next couple of years (ie most machines doubled the RAM from the previous generation).

My old workstation bought in 2018 had enough memory that it could virtualise the current prod environment with the same amounts of RAM as prod, and still have 20GB free. My current workstation would have 80+ GB free.

In 95% of cases if you can't run the stack for a single user testing it, on a single physical machine, you're doing something drastically wrong somewhere.