Re:Ask HN: How to learn how to scale software?
by poulsbohemian on Friday March 29, 2024 @03:19PM
(#39867927)
Whatever code you are handy with. You'll need land generator tools and profiling tools - and picking those out and developing skills with them is a step toward understanding scale too.
In the course of building those prototypes, you are learning something about scale too... what if I spawn a bunch of threads to handle this task at this point? What if instead I put each of those tasks on a queue and have a task handler? What if I take this engine piece and put in on separate hardware? What if I have a bunch of these engines running in parallel? What if I make this change to the DB connection? What if I remove the dependency on the DB entirely?
It's easy to say a lot of this stuff is basic computer science you should have learned in school, but that's not reality. There's a lot of emphasis put on algorithmic complexity and efficiency, there's a lot less put on systems design and architecture. You kinda have to do it in the wild to learn it.
In the course of building those prototypes, you are learning something about scale too... what if I spawn a bunch of threads to handle this task at this point? What if instead I put each of those tasks on a queue and have a task handler? What if I take this engine piece and put in on separate hardware? What if I have a bunch of these engines running in parallel? What if I make this change to the DB connection? What if I remove the dependency on the DB entirely?
It's easy to say a lot of this stuff is basic computer science you should have learned in school, but that's not reality. There's a lot of emphasis put on algorithmic complexity and efficiency, there's a lot less put on systems design and architecture. You kinda have to do it in the wild to learn it.