kn7
Reactor-pubsub: A minimalist reactive Google Pub/Sub Java 8 driver
If I am not mistaken the majority of the PL/SQL glue is owned by Gert, though you might recall better. Quite some VCS history was lost while migrating from SVN to Git. ;-)
The reason we are "replicating" the entire data is to 1) determine the affected products and 2) re-execute the relevant configurations (facets, synonyms, etc.) while making retroactive changes. (For instance, say someone has changed the PL/SQL of "leeftijd" facet.) Here, the storage is required to allow querying on every field, for (1), and on id, for (2). While id-based bulk querying is (almost) supported by every ETL source, querying on every field is not. Hence, we "replicate" the sources on our side to suffice these needs. Actually, the entire point of the post was to explain this problem, but apparently it was not clear enough.
For your remarks on event sourcing and BI, I am a little bit puzzled. I will need some elaboration on these remarks. We do have event sourcing on our side (that is how we can replay in case of need) and BI is not really interested in ETL data. Maybe I misunderstood you?
I am also confused by how you relate scheduling/running PL/SQL jobs via Hadoop, Spark, Flink, etc. Did you see the link to Redwood Explorer I shared in the post?
Using Elasticsearch as the Primary Data Store in an ETL Pipeline
Inter-Microservice Communication Fatigue [in Java]
HRRS: Record and Replay HTTP Requests in Java EE and Spring Applications
Performance of Linux IP Aliased Network Interfaces
Chat Application with Play Framework and KnockoutJS
Facebook Login and Secure module integration in Play Framework
WAPI: A generic C API to (wireless) network interfaces in GNU/Linux systems
WAPI (Wireless API) provides an easy-to-use function set to configure wireless network interfaces on a GNU/Linux system. One can think WAPI as a lightweight C API for iwconfig, wlanconfig, ifconfig, and route commands. (But it is not a thin wrapper for these command line tools.)