bogaczio

I've heard that they'll be releasing a cheap Splunk-like service over CloudWatch for log searching, which will hopefully alleviate the issue.

I would say that as long as you have a small enough number of types where implementing this for each of them (and each combination of them) is manageable... you should. The absolute worst thing about this is losing type safety (and I helped write it). The time to use this is IF this kind of design could simplify your code overall AND the number of types and their combinations is simply too much to code. Say you have four input types, two intermediary types, and one output type, and more than one step in between them, then you'd be writing about 15 times the code. We actually had a situation like this, which is why it came in handy. If you're not in this situation though, I would always advocate retaining type safety.

I think the performance issue is addressed, in the second paragraph, because you're quite right that it would create a slowdown. This library is quite simple, and it's mostly an exploration of a potential design pattern, not intended, nor recommended, as a generic solution.

The channel ownership isn't so much of an issue, at least the way I understand the question, since currently each Operator creates its own "output" channel, and is effectively the producer for that channel. The way this works in practice is that you instantiate an input channel, combine multiple operators together, and when your input is done, you close that input channel, which closes all of the subsequent channels in turn (because of the for ... range ch being used). That said, in practice I have used a separate channel for error reporting, which is lightly addressed near the end of the article.

The point about annotating channels is very good, I'll probably be making a change to incorporate that here soon.

Thanks for reading, and I appreciate the good feedback!