News for hackers. Stuff that matters.
Re:The JavaScript Pipeline Operator by ConcernedCoder on Thursday November 08, 2018 @12:08PM (#18407797)
That's all well and good ( increasing readability ) but the problem remains that each step has to finish before the next step can begin... sometimes the entire dataset won't fit into memory/machine/whatever...

More useful, IMHO, would be a way to EASILY compose a true pipeline:

  const
    _pipe = (a, b) => (arg) => b(a(arg)),
    pipe = (...ops) => ops.reduce(_pipe)

...but have the behavior work like unix pipes ( a stream ), nodeJS supports this concept at it's most basic level using the pipe() abstraction, although you have to supply methods which handle being pipe'd to, and from... an example:

  const crypto = require('crypto');
  // ...
  fs.createReadStream(file)
    .pipe(zlib.createGzip())
    .pipe(crypto.createCipher('aes192', 'a_secret'))
    .pipe(reportProgress)
    .pipe(fs.createWriteStream(file + '.zz'))
    .on('finish', () => console.log('Done'));

*ripped from: [source](https://medium.freecodecamp.org/node-js-streams-everything-y...)

Imagine reading a 100gb json file line-by-line via ajax on the client, and feeding into the pipeline of transformative methods -- iteratively introduce data in one end of the pipe, and gathering the results at the other end, and creating some visualization like a graph or whatever... without ever having to have the entire thing in memory at once...

Re:The JavaScript Pipeline Operator by piedar on Thursday November 08, 2018 @01:58PM (#18408867)
Have you seen https://github.com/labs42io/itiriri? It does lazy queries on iterables, like IEnumerable from C#.

    import { query } from 'itiriri';

    function* fibonacci() {
       let [a, b] = [0, 1];

      while (true) {
        yield a;
        [a, b] = [b, a + b];
      }
    }

    // Finding first 3 Fibonacci numbers that contain 42
    const result = query(fibonacci())
      .filter(x => x.toString().indexOf('42') !== -1)
      .take(3);

    for (const e of result) {
      console.log(e);
    }

    // outputs: 514229, 267914296, 7778742049
Re:The JavaScript Pipeline Operator by dvlsg on Thursday November 08, 2018 @03:07PM (#18409500)
I wrote a lib that does this too. It's been a while, but using generators tended to be way slower than just using arrays, except in the most obvious cases (array of 1000000 elements, only take 5, no sorting involved, etc). Maybe that's changed. It's been a while since I've checked.