News for hackers. Stuff that matters.
Re:V8 adds support for top-level await by mstade on Tuesday September 24, 2019 @07:25PM (#21065806)
I think async/await probably makes more sense in a typed language, where a compiler can tell you when you're missing an await, or at least warn you about not dealing with potential side effects and error handling. For something like JavaScript, it'd make more sense to me to have the runtime always and implicitly await the result of async functions, and instead make developers explicitly say when they wish for the result to be async. For example, instead of:

    const data = await fetch()
    push(someData) // async, runs in the background
You would do:

    const data = fetch() // Runtime detects promise and awaits result
    async push(data) // the async keyword would return a promise and execute the push function asynchronously, allowing the next line to execute
In this fantasy world the "await" keyword would work anywhere and as you'd expect – awaiting the result of any promise:

    const data = fetch() // implicitly await
    await async push(data) // this would also be "synchronous" in that it suspends execution of subsequent code until `push` fulfills or rejects the promise, and so it'd have the same effect as implicit await
Point is you'd probably await that promise elsewhere, so you'd actually store away the return value of the `async` call and later on you'd `await`, or another example would be to await a block.

Promise rejections in implicit awaits would halt execution, just like a sync function throwing an error, so you wouldn't "miss" an error somewhere because runtimes swallow promise rejections. (Well, at least Node wised up eventually.)

This means there'd be no difference in function declaration between sync and async functions, it'd be determined by whether they return a promise or not which I think should be possible to statically determine by a JIT compiler in most cases, so not adding too much (if any) overhead.

Kind of a half baked thought, but point is I always felt the async/await thing was kind of backwards in JavaScript.

Re:V8 adds support for top-level await by minitech on Tuesday September 24, 2019 @07:33PM (#21065866)
Promises are nice in JavaScript because they’re just a value with no magic. Anything can create them, not just async functions. Generic/higher-order functions and so on can get involved without needing to know the difference. A proposal to introduce magic at calls sounds really awful, sorry.
Re:V8 adds support for top-level await by mstade on Tuesday September 24, 2019 @07:48PM (#21065997)
No magic is rich — they swallow errors, for one. In any case, I don't think anything I said precludes the creation of promises outside of async functions, in fact quite the opposite. The difference is that the runtime would implicitly await the resolution of a promise returned from a function (any function, there'd be no such thing as an "async function") and if you actually wanted things to progress asyncronously you'd simply add `async` before the statement and it'd return a promise (for keeps) of the value of the statement, which could be the return value of a function, or even something as simple as:

    const val = async (1 + 2)
    await val // 3
Maybe it's a genuinely dumb idea, for reasons I can't fathom, but it seems to me this would make working with async code much simpler than it is now, where if you forget an await you've probably introduced a bug, and the only way to know if you need await is to peruse docs (hoping that they're accurate) or judiciously sprinkle it everywhere.
Re:V8 adds support for top-level await by laughinghan on Tuesday September 24, 2019 @09:22PM (#21066629)
How is swallowing errors magic? try/catch does that and predates ES3.

To clarify your example, what about:

    function foo() {
      const val = async (1 + 2)
      return val;
    }
    const x = foo();
Does the `x = foo()` block/implicitly await?

> if you forget an await you've probably introduced a bug, and the only way to know is ... docs

Given that async is "contagious", I'm having a lot of trouble imagining a scenario where a codepath could run and appear to work, but actually be hiding a bug because you didn't realize something was async. Unless you're just saying that the bug would be apparent as soon as you ran the code, but the syntax checker wouldn't flag it for you?

Re:V8 adds support for top-level await by mstade on Wednesday September 25, 2019 @05:53AM (#21069227)
But try/catch makes swallowing errors explicit, you have to catch it to swallow it. With promises it's the opposite situation, errors will be swallowed unless you explicitly handle the rejection. At least the runtimes have grown up to show console output when there's an unhandled rejection, but man it was pretty dark for a while.

> Does the `x = foo()` block/implicitly await?

Yes, that's what I'd expect, because `foo` returns a promise.

> Unless you're just saying that the bug would be apparent as soon as you ran the code, but the syntax checker wouldn't flag it for you?

It may or may not be apparent even from running the code, consider the following:

    let data
    try {
        data = readFile('some-file.csv', 'utf8')
          .split('\n')
          .filter(l => !!l)
          .map(l => l.split(','))
    } catch (e) {
        data = []
    }
It's not particularly good code, granted, but it's also not a contrived example. What's the bug? Well, assuming `readFile` is sync, and `some-file.csv` is a nice csv file with no surprises in it, this should make data an array filled with data from said file. If the file is empty, it's just going to yield an empty array. If the file can't be read for some reason and `readFile` throws an error, it'll be an empty array.

Now make `readFile` return a promise instead and `data` will always be an empty array, regardless of whether the file exists, can be read, or otherwise all conditions for a happy path are met. Why? No `split` method on the promise.

The problem isn't promises of course, it's the fact that we're doing too much in a try clause, or at least not dealing with specific errors properly. But let's be honest – who hasn't seen (or even written?) code like this in the past?

It might've even worked fantastically well for a long time, years even, until someone comes around and changes `readFile` to be async, for reasons, and now it breaks in a subtle way and it's fun and games trying to find why `data` is always an empty array even though it seems everything should be fine. Actually, it doesn't even have to be `readFile` that changes, it might be a function that it depends on, causing bugs further up the call stack. It happens, no matter how semantic our versioning is.

If however the runtime would always implicitly await return values, and you'd have to explicitly mark statements as async to break out of that, then this code would continue to function regardless of whether `readFile` or one of its dependencies returns a promise or the actual file contents, because the runtime would deal with it. As it is now, you have to go and change all code that calls `readFile` to be async, so you can await, meaning anything further up the stack also needs to be async, so you can await. It's a bit of a foot gun I think.

In any case, it's just a thought and at best a half baked one at that. I just find the async/await semantics to be backwards, and while I've used it for a few years at this point I still keep running into dumb situations like the above. Maybe I'm just a bad programmer, I'm certainly not excluding that as a possibility. :o)

(Apologies for the wall of text.)

Re:V8 adds support for top-level await by laughinghan on Wednesday September 25, 2019 @04:28PM (#21075000)
> With promises it's the opposite situation, errors will be swallowed unless you explicitly handle the rejection.

Yes, I've been bitten by this too, it's certainly a drawback with the design of Promises. I wouldn't describe it as magic though, since both the implementation and the impetus are easy to understand.

> Actually, it doesn't even have to be `readFile` that changes, it might be a function that it depends on, causing bugs further up the call stack.

I don't think it's possible for readFile to become async without at least some change to it (possibly just adding 'async' and 'await' keywords, but at least some change), except maybe a rare case of a tail call.

> even though it seems everything should be fine

In what way would it seem that everything should be fine? If readFile() were changed from sync to async, wouldn't every single call to readFile() in the entire codebase need to be changed, just like if readFile() were changed from returning a string to returning a File object? It's not like most or even any at all of the calls to readFile() wouldn't need changing, then I could see how it might seem like everything would be fine.

> breaks in a subtle way

But this isn't a subtle bug, it completely breaks as soon as readFile() is changed from sync to async, right? No testing, automated or manual, of this codepath would work at all after the change, right? It's not like a cursory smoke test of this codepath seems to work fine, then I could see how the bug could seem subtle.

> I still keep running into dumb situations like the above. I still keep running into dumb situations like the above

You don't seem like a bad programmer, which is why I'm skeptical of the example you gave.

Re:V8 adds support for top-level await by mstade on Friday September 27, 2019 @11:44AM (#21093192)
> I wouldn't describe it as magic though, since both the implementation and the impetus are easy to understand.

That's fair, magic may have been a bit hyperbolic.

> [...] just like if readFile() were changed from returning a string to returning a File object

But that would change the semantics of the function, it literally changes the return type. My point is that adding `async` really just changes the meachanics of the function, not the semantics. If my function returned a string before, and I add `async`, it'll still return a string; just eventually. As a caller, I don't really care, I just want the darn string.

ometimes, as a caller, I do care, and that's exactly why I think have the caller decide when to run something async makes more sense. (There's a whole other discussion that could be had here about how JS promises are a poor async abstraction anyhow, but I digress.)

> But this isn't a subtle bug, it completely breaks as soon as readFile() is changed from sync to async, right?

No it's definitely subtle. In the example I gave, the code would use the default empty array value when there's an error reading the file, for whatever reason. For the happy path, it'll work just fine, though it probably wouldn't deal with invalid input very well. Change the mechanics of readFile to async though and it'll always return the default value, even though the semantics of readFile stays the same. It still returns a string, just eventually, but because the code expects a string, it'll always break because it gets a promise instead. Add `await` and it'll be fine, but now whatever function that code is in is async, and whatever function calls that needs to also `await`, ad nauseam.

> You don't seem like a bad programmer, which is why I'm skeptical of the example you gave.

Hey, thanks! :o)

To your point though, it's definitely representative of the kind of code I come across on a regular basis. Many a times have I had to help colleagues debug this kind of issue, and many a times have I shot myself in the foot in similar ways. In any case, JS async/await semantics are set in stone now, and it's probably too dynamic a language for something like implicit await to work (performantly) anyhow, as previously mentioned.

I appreciate you taking the time to discuss, it's nice being challenged on the actual topic, without it devolving into ad hominem nonsense. There are still good corners of the internet after all!

Re:V8 adds support for top-level await by minitech on Wednesday September 25, 2019 @01:34AM (#21067926)
> and the only way to know if you need await is to peruse docs (hoping that they're accurate) or judiciously sprinkle it everywhere.

Knowing whether what you’re calling is async is part of knowing what you’re calling at all. Sprinkling await everywhere to try to mask the difference is horrifying. (I kind of wish it didn’t work on non-thenables for that reason – half the time, it’s a bug.)

Re:V8 adds support for top-level await by mstade on Wednesday September 25, 2019 @06:36AM (#21069397)
But it can change under your feet – someone might change a function to be async, that used to be sync, and now your code is broken. The code does the same thing, it's just that it turned from returning a value to returning the promise of a value, and now your code is broken. Maybe it's not the function you're calling, but a function further down the stack, that you don't even know about.

It may be that the docs are bad and don't even tell you it returns a promise. Heck, maybe it only returns a promise on Tuesdays, or at random like some other commenter wrote – you'd have to then sprinke `await` there to make sure you're ok, even if most of the time you don't need it.

Re:V8 adds support for top-level await by laughinghan on Wednesday September 25, 2019 @04:43PM (#21075201)
> maybe it only returns a promise on Tuesdays, or at random like some other commenter wrote

But async/await makes this better, because a function marked async always returns a promise.

Re:V8 adds support for top-level await by minitech on Wednesday September 25, 2019 @03:49PM (#21074572)
> But it can change under your feet – someone might change a function to be async, that used to be sync, and now your code is broken.

Someone might change a function that returns a single value to return an array, and now your code is broken. Someone might rename the function, and now your code is broken. This is the nature of breaking changes, and the same solutions apply.

Re:V8 adds support for top-level await by mstade on Friday September 27, 2019 @11:48AM (#21093237)
Yeah but that changes the semantics of the function, whereas `async` arguably just changes the mechanics. The promise itself is not interesting, it's whatever value it (eventually) returns. My point is that a language that had implicit await would let you go on making function as async as you want them to be, and callers would be none the wiser. It'd also allow the caller to decide when to run things asynchronously and even truly defer values, something which JS promises can't do since they immediately execute, but that's an altogether different discussion.
Re:V8 adds support for top-level await by laughinghan on Tuesday September 24, 2019 @09:08PM (#21066561)
I like your idea, but I don't see how it could work in an untyped language. Consider:

    function foo() { return 1; }
    function bar() { return fetch('http://example.com'); } // implicitly async
    
    function qux() {
      const fn = Math.random() > 1/2 ? foo : bar;
      fn();
      return 1;
    }
Is qux() synchronous?
Re:V8 adds support for top-level await by anonytrary on Wednesday September 25, 2019 @12:07AM (#21067505)
The function is both synchronous and asynchronous until it is called by the caller. This is called Shannon's Cat.

Jokes aside, I'd like to add that just because a function returns a promise doesn't mean the caller will always want to wait for it to resolve. I think of `await` as a simple modifier that casts the return value from a `Promise<T>` into `T`. By getting rid of the modifier, we can no longer assume that the caller wants to implicitly wait.

Re:V8 adds support for top-level await by mstade on Wednesday September 25, 2019 @06:40AM (#21069412)
You're right of course, and I have zero data to back this up other than anecdotes from my own experience, but still I posit the most common case is that the caller wants to await the return value, not run the function asynchronously. This is why I think it'd make more sense to flip the semantics so you'd have implicit await, and have to explicitly mark the things you want to run asynchronously.
Re:V8 adds support for top-level await by eyelidlessness on Thursday September 26, 2019 @03:25PM (#21084522)
Implicit await would be a nightmare. You could never tell looking at a given block of code whether it yields to the event loop or not, introducing invisible concurrency problems. Even static types wouldn't solve this without looking at every value and function return type.
Re:V8 adds support for top-level await by mstade on Sunday September 29, 2019 @05:25AM (#21106048)
Would love to learn what concurrency issues you'd see with implicit await, that you presumably wouldn't see otherwise. You're probably right, I just can't think of any examples.
Re:V8 adds support for top-level await by anonytrary on Wednesday September 25, 2019 @11:35PM (#21077930)
I think there might be some confusion here. `foo()` will run asynchronously regardless of whether or not `await` is specified. I agree that in most cases, you will want to unbox the Promise. It is still very common to pass Promises around as values.
Re:V8 adds support for top-level await by mstade on Wednesday September 25, 2019 @06:17AM (#21069327)
The point isn't so much whether qux() is async or not, it's whether the caller wants it to execute before subsequent statements or not. In the current state of the world:

    const a = await qux()
    const b = qux()
    const c = await qux()
    const d = (await b) + 5
But if I had my little way:

    const a = qux()
    const b = async qux()
    const c = qux()
    const d = b + 5
The weird one is `d` of course, why would it implicitly await there? Because `+` tries to get the value of `b` in order to add `5`.

How the runtime would optimize this scenario I don't know, it could probably statically determine that qux() may be async and therefore determine it would have to implicitly await its return value even if it's just the number. Obviously this is a contrived scenario, but I'd rather pay a small runtime cost, than pay the cognitive overhead of remembering to await things everywhere. Like most people, I forget things...

Re:V8 adds support for top-level await by laughinghan on Wednesday September 25, 2019 @03:42PM (#21074511)
I think you missed my point. I understood how you inverted explicit and implicit, you saw me explain that to someone else who asked about Promise.all().

What I'm asking you is, how does anyone, be it the programmer, compiler, or runtime, know whether to block on qux() or not, given that it sometimes synchronously returns 1?

Are you saying that in your proposal, it would become part of the calling convention that any time any function at all returns (and no 'async' keyword was used), the callee would check if a Promise was returned and if so, return a continuation promise?

That is no small runtime cost.

In an untyped language like JS, the cases where static analysis could determine whether a function returns a Promise or not are vanishingly few, probably about the same as the cases where a function can be inlined. Remember, anyone at any point can overwrite Array.prototype.push to be anything they want, including a function that returns a Promise. Any function that takes and calls a callback might be calling an async function. Any math or string operation could be calling .valueOf() or .toString(), which might be async.

Re:V8 adds support for top-level await by Scooty on Tuesday September 24, 2019 @08:41PM (#21066388)
Making await implicit would make it difficult to manage parallel promises. You would either have to make an exception for Promise.all or add new syntax. It's also not unheard of to have hanging promises (fire and forget).

Types make it easier to catch mistakes early in general. Typescript has a compiler rule 'no-hanging-promises' to help avoid forgetting to await.

Re:V8 adds support for top-level await by laughinghan on Tuesday September 24, 2019 @09:00PM (#21066501)
I think you missed this part: "instead make developers explicitly say when they wish for the result to be async"

So using Promise.all() would be as simple as:

    const results = Promise.all(async fetch(url1), async fetch(url2))