Somehow I still don't feel that way about StackOverflow. Google and random results though... yikes.
Somewhere around the time node got very popular, I started to notice a lot of impeccably branded (is that the right word? trendy maybe?) websites with tutorials that used all the right buzzwords to get me interested. Once I'd step through the content, it'd be very low quality. Despite the smooth lines and round edges, a lot of them were riddled with inaccuracies, assumptions, unabashed spelling and grammar issues, stolen content and lots of candid offtopic observations.. Not to mention they all loaded very slowly.
I used to joke about it mockingly during the early node days that you could judge the trajectory of a project based on the ratio of bytes dedicated to persona and branding vs actual content. Anything passing the 1:2 ratio generally didn't last long.
I don't think PHP makes for a great comparison, because the web ecosystem was still growing. Node was birthed in the age of Github, social media, Stack Overflow, and Google. PHP was released in 1995, when search engines were still stuck in their infancy and only a small percentage of the world was even using the WWW. 20 years in PHP's time is like maybe 5 years from 2009.
But... much of the earliest web work (PHP, etc) occurred in the early days of search. Example of quality code, best security practices, etc all were fairly .. rudimentary and hard to find.
Just because it took 20 years starting 20 years ago doesn't justify 20 years starting from 10 years ago. There are infinitely more and better resources for just about everything these days.
Err ... Netscape's LiveWire introduced server-side Javascript in 1996 even before Java became widely used on the server side. The module convention, the (synchronous) core modules of Node.js, and its canonical http middleware API and express.js/Connect/JSGI is from the CommonJS initiative, a co-op of 2000's SSJS framework developers [1].
And don't "um ackshually" me about saying "nobody." You know what I mean.
I thought of node as a victory for async more than anything else.
In terms of server-side development? Async is rather useless on the server side. Can't generate a web page or JSON blob with the results of a database query until you actually get those results from the database. I can see the application for something like websockets but I presume most Node sites aren't using those.
I figured its success was due to V8 being reasonably fast, plus a generation of front-end-centric developers coming along who had learned to use JS quite heavily on the front end and didn't realize or didn't care that better options already existed on the back end.
You'd be wrong, again. According to Ryan Dahl, creator of Node.js [1]:
> I think the combination of Chris Neukirchen’s rack plus how Nginx structured its web server with non-blocking IO led me to start thinking about how those two things could be combined. [...] So, when V8 came out, I started poking around with it, and it looked fascinating [...] and suddenly, I clicked that: oh! Javascript is single-threaded, and everybody is already doing non-blocking. [...] I think JavaScript plus asynchronous IO plus some HTTP server stuff would be a cool thing. And I was so excited about that idea that I just worked on it non-stop for the next four years.
[1]: https://mappingthejourney.com/single-post/2017/08/31/episode...
This is precisely why async (well, cooperative concurrency generally) is useful, whether on a server or otherwise. Blocking for things you can’t do yet is a waste of idle resources. There are plenty of other concurrency models which are also useful, but yielding/suspending when idle is almost always better than not. And the cases where it’s not generally have other problems.
You can think of the concurrency model like green threads with only one thread available, or actors in a single system process.
Edit to elaborate: yield/suspend is key here. The way it works is yielding/suspended calls have their outstanding work put behind the extant queue which proceeds until each next queued routine suspends or returns, and once that set of blocking work completes the routine which suspended can proceed until it yields again or returns. The obvious pathological case is long running work which doesn’t suspend. Which if it’s I/O or idle-heavy should be refactored to yield, if it’s compute-heavy it should be treated as an optimization or scaling target.
I don't know, I'm willing to accept that maybe I'm just not used to thinking of developing back-end web apps that way. But async still just doesn't seem as useful as it would in a GUI desktop app.
If 20 users log in at the same time node and php would behave very differently.
Php would start 20 processes/threads and execute each in independently.
Node would dispatch 20 Request events to the event loop in the same process.
I will not argue the advantage of server-side async, but here it is clear that node needs it, as those 20 database queries will be executed from the same thread. Without async the 20th user would have 20 times as much latency as the first.
Because I have two requests active at the same time and the second one which arrived shouldn’t wait for me to stand around doing nothing while I wait for the database to serve the first.
> Like I guess on a blog or something, you could load the user record of the current user and then load the blog post details at the same time, but then there's a chance that the user doesn't have permission to view that blog post and then the request for the post has gone to waste, right? It goes against my instincts to kill a request as quickly and directly as possible if something can't be accessed.
Here one user represents 2+ events, both of which should fail. This wasn’t part of my original explanation but yielding does give you the opportunity to fail both simultaneously. Albeit in this case it’s just concurrency broadly. Why is that against your instincts, if you know that one request’s failure implies the other will also fail? Why should that user wait in line to get the same error after another cycle of the same trial?
> I'm just not used to thinking of developing back-end web apps that way. But async still just doesn't seem as useful as it would in a GUI desktop app
Yep I get that. And async takes a bit to get used to in general. The way I think of it is: I have two kinds of work which need to be done, and a long line asking me to work. Some of the work needs labor (CPU), some needs patience and communication (IO). Everyone in line needs some of both.
If I do the first person’s immediate labor and wait for further communication, persons 1 and 2 are waiting for me to proceed. If I do the first person’s labor, send off an asynchronous request for feedback on that, I can do N people’s labor before I get any response. As long as no one spends too long making anyone else wait in queue everyone moves as fast as the queue’s capacity.
I really have no idea what you're saying here, so let me put it this way. I would code this hypothetical blog page access situation this way in PHP: Get user data from database. Do access checks on user data. See user does not have permissions to view blog article. Show 403 page.
Assuming I'm understanding what you're saying about async database requests, this is apparently "correct" in Node: Request user data and blog post data from database. If the blog post data comes first, wait. When the user data comes, do access check and show 403 page. If the blog post data hasn't come yet… it just gets ignored or something, I guess. Otherwise the data just gets thrown out…?
What's against my instinct is to make the request for the blog article data unless and until it is known that it will be needed.
The scenario you describe, performing multiple queries for the same request in tandem, certainly isn’t incorrect. It’s a less common design, but one which might make sense depending on the workload. An example where I’d choose that design: both queries are slow individually, but not significantly slower queried concurrently. Even if the query for blog post data turns out to be wasted work for one request, it might halve the time to serve another request.
When I say which is more common, it’s probably more a matter of the mental model of the human who implemented it. Fulfilling a request as a series of steps versus a set of concurrent steps only becomes a matter of correctness when one of those steps is dependent on another (and even then only if there’s no other reasonable way to address their dependency afterwards).
I’ll add that designing for concurrency this way often leads to better design overall. It helps guide towards minimizing dependencies (or at least minimizing shared state) between functions. Which in turn isn’t just easier to reason about, but also easier to scale should you want (say) to move some expensive function out to its own service or worker.
Maybe some pseudocode will help illustrate. Here is a hypothetical[1] implementation of the Node equivalent of the PHP design you described:
const handleRequest = async (request) => {
try {
const userData = await getUserData(request)
const blogData = await getBlogData(request)
return whatever(userData, blogData)
} catch (error) {
return errorResponse(error)
}
}
But because, in this hypothetical implementation, `getBlogData` doesn’t depend on `getUserData`, it could be written this way too:
const handleRequest = async (request) => {
try {
const [ userData, blogData ] = await Promise.all([
getUserData(request),
getBlogData(request)
])
return whatever(userData, blogData)
} catch (error) {
return errorResponse(error)
}
}
To your original wondering, why `await` at all? Well, for one because you have no choice. With very few exceptions, all IO is asynchronous in Node. And async colors[2] JS functions. More importantly, because JS is (mostly) single threaded and generally spawning new processes is either not available or expensive to do on demand. Getting the most processing out of a single process is the whole point.
Other environments will either accomplish something similar with threads, or spin up a process per request and depend on the OS for scheduling work. They’re all slightly different trade offs along the same general theme: make idle resources available while waiting. Node’s trade-off is that it has to handle a global (or process-wide anyway) task queue. Threaded approaches and similar put more scheduling control, resources, and complexity, in the hands of devs. Process-per-request eliminates the need to think about any of this, at the expense of cold start times and memory.
All of those trade-offs are reasonable depending on the workload and depending on the environment. Part of the reason `async` (the keyword and/or language facility) has gained popularity across a wide variety of use cases is that the combination of concurrent state complexity, cold start time, and memory pressure makes it appealing to model the problem in a way that almost looks synchronous.
1: Node devs stumbling upon this: yes I realize this isn’t the typical interface for a request callback. That’s intentional to keep it simple, but you can do it this way if the traditional callback hell interfaces are not your cup of tea… you just need a simple wrapper.
2: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...
I think the various “assume best intent” rules and adages would benefit greatly from a huge “assume limited familiarity when dismissiveness seems out of place” addendum. At least for technical topics where the stakes are “whoops we actually just disagree on the merits”.
sometimes i Google stuff i already know just to verify i remembered it correctly
lately i've been refreshing my memory on Vue JS and this website comes up a lot: https://thewebdev.info
the solutions i read there were often incorrect and misleading
i can't imagine becoming a developer today, when most of the stuff you read on the internet bullsh*t
of course you can browse the docs, but some docs are tedious to comprehend when all you're looking for is a simple one-line answer
Honestly, we are most of the way there but for three holdouts; 0 config auth, easily extensible admin, and migrations.
User stuff isn't so bad with bcrypt but I'd really like to be able to type `framework create superuser` and `framework create user` instead of fiddling with user access and re-imaging user groups and permissions for the hundredth time.
Admin stuff always seems to be something relegated to a separate tool. No, please, please, please include it in the framework.
Most frameworks have you choose your own ORM (which is fine if they consistently use the same one in documentation), but for some reason Node ORMS always treat migrations as an afterthought or second class citizen (if they even consider it). Changing table structures in development to add a new column? Half of them just say, sorry, we're going to have to blow that away. The rest have either manual migrations or partially automated migrations written in the SQL dialect you happen to be using during development. My favorite of the lot I've seen so far is Prisma and even they have recently indicated they don't want to support transactions in migrations because they don't want to give you a "false sense of security" (well yeah, how about a real sense of security?). To be fair, even in python land Django is really the only one to have ever gotten migrations 100% right.
Overall I guess it was a mixed experience, I missed the JS language (just because I know it well, Ruby has a lot more syntax) and Typescript but I liked having all the things like auth, migrations etc. which are not my speciality taken care of in a way that I could trust and easily Google thanks to it being so widely used.
I’m not 100% sure if I’d make the same choice again, the mental overhead of going to a language I don’t usually use to make changes is a downside as is the lack of type checking (though I believe there are better options in the Ruby world now), but on the other hand I could do the whole thing with very little code (relatively) and it was a good experience to see just how much work a well designed framework can save you.
I should add I did try Django but had my heart set on GraphQL (just because I wanted to try using it for real) and the options in the Python ecosystem didn’t scale well. Overall I liked Django’s ORM better and the Python language, but I felt like you got more out of the box with RoR, the ecosystem is higher quality, and I had to do fewer hacks to get stuff working. All depends on the scale of the project and how much code you want to write I guess.
So I decided to just build it all once so I could re-use it, and then I found that others had the same problem and are happy to pay a reasonable amount to have it solved for them, and now it's doing pretty okay for itself.
It did end up taking a lot longer and involving a lot more work than I expected, but I figure that's alright, it just means a more-reliable base to start from each time.
which in my book is a plus
Perhaps you can point out issues or improvements which could be possible by leveraging these strengths if javascript?
If you're looking for a traditional web framework (something similar to Flask or Sinatra) I would take a look at Hapi (which has no third party dependencies and is maintained by one of the original authors of the oAuth spec), or you could always try Express (the most popular Node framework), or Fastify (the fastest Node framework).
[1] https://nodejs.org/dist/latest-v18.x/docs/api/crypto.html#cr...
[2] https://nodejs.org/dist/latest-v18.x/docs/api/crypto.html#cr...
That said, I agree that in the Vercel/Next universe, everything on the backend seems to be an afterthought.