News for hackers. Stuff that matters.
Re:Aggregate streaming data in real-time with WebAssembly by jgraettinger1 on Tuesday August 24, 2021 @08:25PM (#28296699)
A similar strategy we [1] are pursuing is to use `reduce` JSON Schema annotations to make it easy to define generalized reductions over your JSON data types [2].

Taking the examples from the article, equivalent schemas might be:

  type: object
  properties:
    mySum:
      type: number
      reduce: { strategy: sum }
Or even:

  type: object
  properties:
    myDeeplyAggregatedMap:
      type: object
      reduce: { strategy: merge }
      additionalProperties:
        reduce: { strategy: sum }
Use of WASM is really interesting. We've been exploring it as a means for powering user-defined reduction strategies, in cases where the built-in strategies are insufficient.

[1] https://estuary.dev [2] https://docs.estuary.dev/reference/catalog-reference/schemas...

There are no comments on this comment yet.