joeycumines

Addendum: My first attempt would drift over time, if the receiver was slow.

Hurt my head a little, but it's fixed now, and I'm at least moderately confident it's correct - unit tested the behavior step by step using `ts-chan`, actually. Will probably use that test as another example use case.

Actually, it was fun, so I've made it into an actual package.

The one I slapped together works, but this one is better (fixes issues with actually stopping it properly): https://github.com/joeycumines/generator-ticker

On NPM as generator-ticker.

What makes data races not a "real race condition"?

Anyhow, do what you will. I don't get the impression that you are genuinely interested in this topic.

Sure do, I haven't actually used this one (I implemented it just then) but it should do the trick: https://gist.github.com/joeycumines/6206f2a6cd79875c7c164738...

Generators have lots of really nice uses, yep.

I'm not sure what specifically you were imagining, but I've added an example of how "vanilla JS" can achieve fan-in, using an AsyncGenerator: https://github.com/joeycumines/ts-chan/blob/main/docs/patter...

It uses one of the patterns suggested in a comment chain above, which I think is pretty neat, and wasn't one that readily occurred to me: https://news.ycombinator.com/item?id=38163562

I'm not making a case for using ts-chan for any situation where a simple generator-based solution suffices. I wouldn't call the example solution (in my first link) simple, but it's something I'd personally be ok with maintaining. Like, I'd approve a PR containing something similar without significant qualms, _if_ there was a significant enough motivator, and it was sufficiently unit tested. I might suggest `ts-chan` as an alternative, to make it easier to maintain, but wouldn't be particularly concerned either way.

That's all very subjective, though :)

First instalment of docs/examples complete, feedback appreciated: https://news.ycombinator.com/item?id=38183241

Concurrency != Parallel execution

An operation may take longer than a single tick of the event loop, and may have it's own rules regarding state transitions.

To be clear, I'm not saying "don't do any communication by sharing state", just that there are use cases where it's possible to make it much simpler to reason about.

As an example, you might control the state of "making a HTTP request to perform a search", within the frontend of a single page app that has a map, search filters, and results.

One strategy is to use a buffered channel (1 element), and, when the search filters are updated, drain then re-send the request to the channel.

The logic processing these requests would then just need to sit there, iterating on / receiving from the channel. It could also support cancellation, if that was desired.

(I'd imagine the results would be propagated via some other mechanism, e.g. to a store implementation)

I'm working on improving documentation and examples, still a WIP, but you may find this interesting: https://news.ycombinator.com/item?id=38183241

Hey HN,

Based on feedback on my last post: https://news.ycombinator.com/item?id=38156519

I've been working on better documentation and examples, aimed at JavaScript developers without significant knowledge of concurrency-related patterns.

The first pattern I've covered is "fan-in". Aside from a side-by-side comparison, in the pattern doc linked in the question, I also implemented an example of the pattern, a "log aggregator", which multiplexes (TCP) log streams, formatting and writing out lines to a file. Details in the example readme: https://github.com/joeycumines/ts-chan/blob/main/examples/pa...

I'd appreciate it if you could let me know what you think, and if this is a step in a helpful direction.

Thanks :)

Hey, that's a neat little trick to implement locking in JS, thanks.

I oversimplified my example perhaps - it also involved handling interruptions (certain system events), maintaining a lifecycle (set up and tear down), and scenarios where it allowed a certain subset of operations to be performed, while performing one of several operations. That last requirement was due to it using shell scripts to perform configuration of the system, and it needing to extract runtime and configuration information from the main daemon.

Still though, thanks very much for your comments, I've enjoyed reading them.

Fan-in is the other way, but yeah you can do batches of concurrent operations in JS fairly easily.

I still like "pipelining" over that, but that can also be implemented with just an iterator, pretty trivially.

I am working on better examples, but they are going to take me a while, at the rate I'm currently going.

To be clear, `ts-chan` is not intended to target any use case already addressed by promises or async/await.

You mentioned CSP so I'll assume you've got context re: that topic. I believe I understand your point re: synchronisation between threads, which is fair, but I'd point out that race conditions still exist in JavaScript - I'd even say they are common, at least in my experience. It is easiest to maintain the integrity of the internal state of complex data structures when only a single logical process can mutate that state at a time.

Example in a similar vein: Firewall daemon that accepts commands over RPC, and performs system configuration, in a linear, blocking fashion, to avoid blowing things up (say it runs `iptables` and/or `nft` commands, under the hood). It would be trivial to have a select statement, with a channel per command (or just one, perhaps), receiving the input payload. In JS, the response would probably be via callback, rather than a ping-pong channel recv then send, or the like.

It wasn't a firewall daemon (although it did interact with firewalld and more), but that's exactly a pattern I've implemented in Go, for a past employer. I don't imagine anyone is keen to implement such a thing in JavaScript, but it's a pattern that applies to anything that mutates state, especially if that state is fragile or complex.

I'd agree with you for personal projects, but there are plenty of companies etc using TS/JS for backend systems, my current place of work included.

Yep, JS is still single threaded, and all that.

I agree async iteration is pretty good - the `Chan` class implements both `Iterable` and `AsyncIterable`.

For quite a few scenarios, async generators are a better choice. They do have limitations, however. For example, you _can_ implement fan-in, but there's no easy mechanism to fan-in in a blocking manner, from sources that aren't known ahead of time. It's also hard to implement a "fair" mechanism, that doesn't bias certain inputs (like you might desire if multiplexing logs, for example).

I might be completely wrong, but the impression I got from your comment is that you haven't been exposed to many implementations using non-trivial concurrency :)

Fair call though, I guess. It doesn't really matter, but I'm certainly used to TypeScript and JavaScript.

From the concurrent prime sieve example?

That's just something I copy and pasted from the documentation for the benchmark package I used. Allegedly, it is to prevent "certain compiler optimisations", which is something I'm familiar with when it comes to benchmarking Go code, for example.

I lack specific knowledge as to whether it's actually necessary, sorry :)

Concurrency is just the ability to handle multiple tasks at the same time, to use my quickly googled definition.

JavaScript already supports concurrency, e.g. handling more than one HTTP request concurrently, despite the runtime being single-threaded in nature. Also provided by JavaScript is the ability to fork and join with other asynchronous tasks, e.g. using `Promise.all` to wait for multiple HTTP requests to complete.

The purpose of `ts-chan` is to make it easier to coordinate concurrent operations, via communication, which definitely isn't something JavaScript makes easy.

So part of this is addressing usability issues, and providing necessary boilerplate. I've modelled most of that after Go's channels.

Another is providing a mitigation for a common gotcha (well, it's gotten me), when attempting to implement behavior that involves multiple independent asynchronous operations, communicating with one another. Specifically, JS has a "microtask queue", an optimisation I believe was created to power async/await. Details / links to MDN here: https://github.com/joeycumines/ts-chan#the-microtask-queue-a...

I'll definitely keep that in mind, thanks :)

If you want it to run on exiting the function in reverse order, something like Python's context managers would probably be necessary.

I've dabbled in that too, though I don't actually use it for anything, currently: https://github.com/Mcsavvy/contextlib/pull/1

That one is actually pretty easy in JavaScript - in the past I've implemented the same sort of behavior using async generators.

(I also like Go's time.Ticker behavior quite a lot)

Sure, I intend to give it a shot.

I will say though, I personally get the impression that attempts at "concurrency" in JavaScript (in production code) are quite rare, which I attribute to how difficult it is.

That is to say, I don't know if there really _is_ a "default pattern".

I think y'all have very fair points, for the record.

Unfortunately it is a lot easier to write documentation for an audience that shares the same context / background / experience. The README was written with an audience in mind consisting primarily of those familiar with Go (or more convoluted "communicating sequential processes" implementations), who were frustrated that things that are very easy in Go, are so much harder in JS. It's not something I considered deeply, but I was imagining that it was unlikely that someone would be searching for "channels in JS" without a base level of understanding.

I work/have worked with some pretty talented people, but (in the past) I've found it difficult to convey the value of the sorts of patterns that `ts-chan` is intended to enable, to those without first-hand experience with such patterns.

Documentation is hard :P

Ah, that example is demonstrating an internal mechanism used to ensure the high-level asynchronous API calls take at least one tick of the event loop to complete.

There is no `result` variable - I was just attempting to convey that "some async stuff happens, then it waits if we can't verify that it's not the same tick as when the call started". The "async stuff", in the actual implementation, is sending or receiving.

I actually considered including one in the README, but there were (are) lots of other topics to consider.

The tl;dr is that there is some cross over, but the use cases and patterns they support are largely different. Message channels are two-way, channels are one way. (I believe) message channels are 1-1, while `ts-chan`'s channels are n-n, though each send corresponds to a single receive.

The primary use case I aim to address is coordination between independent, asynchronous operations (concurrency in JavaScript). Having an analogue to Go's "select statement" is fairly key to that, and I am not quite sure how to articulate the value it provides, but it does not overlap with message channels.

Example using a single channel: `ts-chan` may be used to implement a set of workers, which accept requests from n sources (e.g. incoming http requests in a web server), then perform some operation, as a mechanism to bound the concurrency of that operation. Personally, I find it much easier to model it as:

1. Start the desired number of workers

2. Wire up workers to await incoming requests then process them, in a loop

3. Send requests to said workers

Rather than (for example) tracking the number of running operations, starting a new worker only if possible, after enqueuing the operation, then having the workers need to operate like "check for any work, increment number of operations, perform operation, check for any work (loop), otherwise decrement number of operations".

The `ts-chan` pattern (which is, really, just a Go pattern) also makes it much easier when the requirements are more complex, e.g. if the requester needs to wait for a response, and might timeout before the operation is even started - in which case the operation should be cancelled.

I should probably include demonstrative examples for all that, heh.

I've been considering adding a `select` function, but I was concerned with the overhead of recreating the select case each time, and wanted to reserve it for a time once I've mulled over my options (I'm aiming to avoid making breaking changes, even though it's pre-v1).

With receives, there's a value attached, and that presents some difficulties regarding typing (for TypeScript).

You have given me an idea, however: I _might_ be able to wrangle together a mechanism that avoids making it a two-step deal (get the index, then resolve the type). It'd probably support only a subset of the functionality, though.

It's on my list to add a simple example to the README using the `Chan` class. Communicating with a single channel (with or without abort support) is very simple, and I'm happy with the API for that.