- create vm instance ( setup firewall, users, network dna, stuff) - install : apache, runtime - install app.
You can also script the process with (for example) Terraform.
- packer for images
- terraform for prod infra
- vagrant for local dev
- pssh for ci/cd and config management
And in some cases I would say they'd be far more efficient with this setup.
Here is my suggestion for keeping it simple, by solely relying on skills you will need anyways: git and shell scripting.
- Write a shell script that executes all those commands you would run manually to set up your system.
- Put up a git repository in the `/` directory, `.gitignore` everything by default, and explicitly git-notignore all those configuration files and user files/maintenance scripts which are relevant for your environment. Use githooks to also track/restore file permissions and ownerships of these tracked files.
If you later want to port this into terraform, you can.
Every single answer goes straight to name dropping various specific techs/tools and I don’t doubt it really is the most simple way for each poster. But I think a much better answer, would be a bunch of questions that dig into what kind of problem you are trying to solve, what are the circumstances, skillset, limitations, budget, is it 1 tiny private app for fun or a couple of apps your team or even company depends on etc.
Because it depends on the specifics in _your_ situation and jumping straight to _a_ solution, without understanding _your_ problem, is imho part of the over-complexity-problem in our industry.
We humans are creative creatures. When we are doing the same thing for a long time, we tend to get bored and make change happen. We 'improve' things, we move things about. We 'make things better'. So we are not bored anymore.
Because nobody wants to hear "that feature isn't viable given our architecture."
So we build systems that can accommodate any possible future feature. No matter how unlikely. And no matter how much extraneous complexity it adds now.
In order to maintain simplicity you have to enforce simplicity. And tech business culture doesn't like that.
It happens because these technologies solve real problem for some subset of their users. Usually when people are saying that technology has become too complicated, it's because they happen to be looking from a perspective where the problems that the technology solves are invisible to them. That doesn't mean that the problem isn't important - it just means the problem isn't important for them. The easiest way to motivate this is to imagine the kinds of problems that need solving at large companies with thousands of employees. They look vastly different than the problems of someone like me working on a side project by myself.
e.g., TypeScript might not be useful for some single-person teams of developers. It's quite useful for teams of around 5, and it's unthinkable to work without it in teams of 100+. Does this mean that the single-developer team is wrong?
(I'd actually say the same thing about React. If you're a one-man team, go ahead, do whatever you want. But if you're working with 100 other engineers, god help you if you're all directly manipulating the DOM with vanilla JS.)
Most "complex" technology is solving many problems for many different smallish subsets of its users at the same time. To the average bystander, it can look like the technology has gained huge amounts of unnecessary complexity, but it hasn't, not really - it's just complexity which isn't directly relevant to you.
(OK, and yes, some other technology is just needlessly stupid, but...)
I feel that is frequently the issue, fragmentation and derivation caused by a lack of fundamental first principles thinking, willingness to tackle the issue from the ground up. The outcome is then talk of bolting on yet another tool or some fix that really only ads debt.
Honestly, I don't think it's even that bad for basic development (they literally have a CLI command that generates a very basic webapp immediately.)
Type annotation also serves as a form of 'hard' documentation. If it's wrong or no longer correct, it will make things turn orange in your IDE and make your type linting tool go bonkers. Even as a single developer this has tremendous benefits. It's much easier to come back to something you wrote months/years ago and get back into it rather than having to go through each function to see what it returns under every situation.
Even if you don't want to use TS in it's full glory, you can just rename your files with a `.ts`, turn strict mode off, run everything through the compilation step, and weed out a category of simple mistakes from that alone. Add in that you can slowly add in certain rules to reach a level of strictness as you need and I see no reason how a JS dev would be better off writing vanilla JS than at least using TS and only running the compile step. Of course, I think taking the time to learn and fully utilize typescript the best path forward, even those with absolutely no desire to learn it can utilize some of it's strengths by reading the docs for 15 minutes.
Far from an unnecessary complexity, it's because an incredibly helpful tool at all levels.
Because vaccines were so successful in their harm reduction to even elimination, there h
created a perverse belief that vaccines
were not effective, and then that they were actually harmful. Being ubiquitous and their effects the same, and the absence of their necessary disease.
Tools that solve problems experience a lifecycle that eventually if successful get weilded by people who never had firsthand experience of the problem.
Really a very large fractal problem space.
We envision a similar problem in autopilot system with 'safety' fall backs. For now, autopilots can disengage and let a pilot take control, but soon, we will run out of qualified pilots to disengage to, and no secure safety procedure.
I tried to do this at several companies and ended up feeling like a pariah. At the end of the day, engineers, especially younger ones, like to follow trends, "learn new things", pad their resumes, and build bespoke solutions. Trying to fight against this will only make you look like a troglodyte or a party pooper.
I know I'm right, as I'm now sure the devs telling me no were right, but I'll be pushed out same as they were.
Young blood is naive and cheap and plentiful so they will always have the momentum advantage. They're the ones that promise the moon to other departments and then stay up all night covering their tracks when they were wrong.
A lot of people value more enhancing their résumé than solving problems
I wrote about a hypothesis here: https://medium.com/@scott.stevenson/how-to-finally-make-some...
Most of us are scared by the ambiguity of actual creative work, so unless we are under the threat of deadline, we seek out "structured games" to play so that we can put off the anxiety of freeform work. These games are:
(1) Tool Game: Researching and setting up tools (2) Learning Game: Books, podcasts, courses (3) Maintenance Game: Cleaning up our desktop, desk, house, etc. (4) Process Game: Setting up new processes and following them
The knowledge that things are actually easy and straightforward means we actually need to do the creative part of the work--many people are not ready to face that knowledge.
I…I…I am triggered by this.
So it’s as if, people such as myself are on the hunt for artificial positive feedback loops that are functionally beautified masks of procrastination. Fueled by, perhaps a subconscious lack of confidence, or a sense of ignorance. Something to that effect.
So now imagine an entire department, teams, organizations all suffering from this? This can explain a lot.
Anyway, I find the usage of the term "game" to be quite odd. Why not "strategy?" I'm not sure I agree with the overall point, but enjoy how you've operationalized/categorized behavior in this way. There is value in that.
It's fine in moderation but tackling ambiguity regularly makes magic happen.
This is exactly what I have been doing for so long (~2 years), I get hyped about a new project, delve into the "best" way of solving the problem and encounter 4-5 shiny new things, spend an unreal amount of time scaffolding and using new tech to create a base framework for whatever I am doing, and then when I actually have to solve the problem my productivity stagnates and I procrastinate more.
Even now I am conflicted/lost on what to do, this response is tbh a cry for help from people who have encountered something like this. Any insight would be great.
A few answers already but let me add one: Complex technologies favour seniors who are capable of understanding the complexity, over juniors who might struggle more. And it's seniors who make the decisions on what technology to use, so I think it's "rational" if they (even subconsciously) make choices which benefit them. They don't want to get replaced by cheaper juniors.
Your whole post is spot-on :)
My two cents on the answer to this question:
- New tools / frameworks / libraries / etc often position themselves as “solving” some perceived deficiency of the prior generation of that thing. So it can be attractive to want to move over to something that solves a pain point. (Whether it actually does, or whether it just replaces it with a new pain point is another thing…) Monolith to microservices is an example.
- At least for me, if I’m starting a new project (especially if it’s in a space I’m not that experienced in), it’s tempting to choose what looks to be the simplest choice, which could be for example a niche database technology suited for my exact need (even if in reality a regular SQL db would be just fine).
- Engineers like to play with new toys!
You can also learn to leave work at work. Keep your side projects simple, if you have any. And rest easy knowing that at work the efforts of one person, you, create enough maintenance work for two people; enjoy that job security.
I highly disagree with everyone else. This is the answer why:
Because there's no clear or definitive answer to What's best the "best" architecture or system for building these things. The entire industry is full of people making up stuff without anyone truly knowing anything.
For the shortest distance between two points we can calculate an answer. there's a definitive optimum. The question of what's the best web service is infinitely more complex. The inputs are huge in number and even the definition of "best" is complex and hard to define... So in short we essentially have no mathematical theory for software design and technology that can help us define what is "best." It's the same problem they have in Art and Design.
So nobody actually knows what is better BUT they think they know so they build wrappers around bad technology or they build completely new frameworks NONE of which actually provably move the needle forward. In fact the needle can move backwards and nobody knows!
When something can't be calculated, it's "Designed." This is the difference between "design" and calculation. Anything that's designed is basically some human wandering around in an area almost aimlessly without ever truly knowing what is optimum. They just use their gut and find some sort of answer that somewhat works. This is the current and past state of the programming industry. That is why on some intuitive level we can see that a lot of the industry looks like it's moving backwards.
Let's say we do have an equation that can take all the components that go into web application development and give you a numerical rating on how good it is. If we had such a model, my best guess is that if we took a system from the past and compared it to a system now, the Modern system will actually have a LOWER numerical rating.
But that's just my guess. As of now, nobody can truly define what that numerical rating represents and nobody knows whether some new framework or design methodology actually increases the amount of points. We are doomed forever to exist in this endless design loop until someone finds a mathematical model that can prove and layout the big picture from a quantitative perspective.
What we need to do is not that complicated but making it so distracts from understanding the harder problem at hand while appearing to be working furiously.
The intelligence is in the end result of the engineering, but no one wants their work to be so simple it could have been done by a junior.
I wrote up my hypothesis here: https://hasen.substack.com/i/52275953/why-the-situation-pers...
Basically, programmers love complexity.
If they can't solve real problems, they will create fake problems so they can solve them.
They might not realize that's what they're doing.
Programmers love solving puzzles. It's why they program. The problem arises when they can't solve the real puzzle due to lack of experience (and lack of skill which comes from lack of practice), they solve secondary and tertiary puzzles that contribute nothing to the real problem at hand.
Instead of "How can I write 1000 lines of code to solve this problem?" they opt for "how can I configure these 20 libraries and glue them together so that a solution to the problem emerges from their interaction?"
> "How can I write 1000 lines of code to solve this problem?"
I opt for "Why are these libraries so complicated? I'll research all 20 libraries and take the parts I like and build a newer, simpler framework."
People who don't see the existing solutions as overly complex are content to stay at their current level and have no vision of solving bigger more difficult problems. They think they are already working at or near the peak of human possibility.
This can seriously limit your earning potential by limiting your career advancement. You are forced to create complexity monsters for promotions at the type companies I've worked at.
1. New Tools, in the modern era means Promotion, VCs, Money, Prestige. personal branding.
2. Creating complexity, in the name of needs for these Big Tech is a great way to create a moat for themselves.
3. Ideology, web development is possibly the 2nd worst places if it wasn't for Rust evangelism strike force. The idea that every single god damn thing should be in Javascript or SPA.
4. The Act of design before coding is gone. Especially in the era of Web Development. Lead by Google and Facebook which later spread across Silicon Valley and later across the world. An Act which is even seen as evil or forbidden.
It's just as bad. Different, but bad.
These teams lack technical leadership that is based on professional experience. This is the sort of lesson that is acquired after at least 10 if not more years of professional work, however the industry tests for algorithmic performance art on whiteboards to select software engineers.
1. Boredom. Everyone wants to be doing interesting work, and doing the same thing that's been done a million times before is probably pretty dull. Devs don't want to feel like code monkeys piecing things together assembly line style, even if that's exactly what's needed for many sites, apps, etc.
2. CV/Resume padding. Perhaps many people see every job as a stepping stone to the next one, and hope a complex solution will get them hired at a FAANG company or fast growing startup in future.
3. Expectations. Problems at large tech companies like Google are more likely to be documented online, since the folks working there tend to have more of an online/social media/blog presence than the folks doing 9-5s at more traditional companies. So people think "hey, if this is how others are doing it, this must be the proper way to do it", ignoring the fact that it's the proper way for a company expecting X billion users a year rather than a more reasonable number.
4. Ego/online reputation. Saying you used a fancy framework and hosting setup with microservices for your project gets a lot more votes on Reddit and Hacker News than saying you used say, basic HTML and CSS.
5. Lack of training/learning opportunities at work. Companies tend not to give workers much of a chance to learn new tech or work on interesting projects in work hours, which would provide them a place to experiment with these new technologies. So they try and integrate it into their normal work instead, even if it's completely the wrong solution for the problem at hand.
The answer to that question has been the same as long as I've been using react. Represent most of your app logic in pure functions. This has been the React team's recommendation, pretty much since the beginning.
Once you get hit to the idea of making everything a function, it pays to learn some functional programming design patterns. This increases your vocabulary for abstracting your application logic into a function.
Once almost everything lives in functions, your react components should really only care about managing the view.
The problem is that's true literally across the board. You move to the backend and you have to deal with people who fell in love with microservices and weird databases that they didn't need. You move to ops and you have to deal with k8s when a single container would do. If you try mobile app development, let me know, I've never tried but I doubt it's exempt.
Why does this happen? No idea, but it's not lost on me that a lot of these things are backed by huge companies with giant piles of money who are using them as marketing.
The only thing you can do (besides leave it all behind and open a taqueria) is be a voice for simplicity. What are we going to build? What tools make sense for it?
The problem I'm running into is a lot of people expect to use React in the same way many of us once relied on jquery. So if that's the case and we must use some React, what can I do to use it in the simplest and clearest way possible?