- hybrid transactional and analytical processing (HTAP), coined by Gartner, - hybrid operational and analytical workloads (HOAP), by 451 Research - Translytical, by Forrester

If that's the solution you want to explore, TiDB (https://github.com/pingcap/tidb), the open source distributed scalable HTAP database, might be able to help you. ETL is no longer necessary with TiDB’s hybrid OLTP/OLAP architecture.

Here is a use case about how it helps the largest B2C fresh produce online marketplace in China to acquire real-time intelligence:

https://www.datanami.com/2018/02/22/hybrid-database-capturin...

Here is a tutorial about how you can try TiDB/TiSpark on your own laptop using Docker Compose: https://www.pingcap.com/blog/how_to_spin_up_an_htap_database...

Disclaimer: I work for TiDB.

The past solution of the separate operational database and data warehouse poses great challenges for real-time analytics because it needs either data pipeline or the ETL process which could be the bottleneck of being "real-time", not to mention the waste of time, efforts and human resources maintaining multiple data warehouses. It was impossible for real-time analysis because, in the past, you would need a data pipeline, or message queue with the equivalent throughputs with your OLTP database, which I believe does not exist.

However, whether to adopt this hybrid solution depends on your specific usage scenario. For cases where users want to do real-time analysis in their data warehouse upon the same data table as in their OLTP database, TiDB is your choice.

But I will say that many moons ago when I did actually write stuff for Mongo. The oplog was a god send. You can "tail" the oplog, and get every transaction in near real time. We used this for updating Elasticsearch indexes etc in what is basically realtime, without having to poll or modify existing code at all.

[0] https://docs.mongodb.com/manual/changeStreams/

Seems to me that it’s on your devs to explain to the business why their poor technical choices now necessitate a substantial additional investment to get a usable solution. When they could have just used Postgres, and they knew it.

Quite often they will have also read somewhere that joins are slow and have managed to convince themselves that the solution is to avoid relational databases altogether. Or maybe that's just how they justify it.

You can also use something like NiFi which supports MongoDB and will allow you to shift data out to Avro/Parquet on HDFS/S3 for your data scientists to use.

As for BI stack. Not sure what you mean. There are hundreds of tools which blend data management and data analysis. You can do this with Hortonworks (Atlas + Spark) or Alteryx for example.

Problem is that it doesn’t work for analysis and we are still using our old platform, which works just fine.

Then they’ll build their BI models in some high level drag and draw system, and pay extra whenever they realize they didn’t get everything they needed in a cube.

The only place I’ve seen actual data scientists is at the university or at the 100% software companies that sell both the solution and the data cube. I’ve never met a real world analytic who could actually code. :p

You’d want to keep a separate dB for your analytics either way though, as they typically eat up quite a lot of load and you don’t want that to interfere with your production environment when you don’t have to.

So may one only use it with a subscription?

I got mongodb postgres foreign data wrapper[0] working in a previous life.

[0]: https://github.com/EnterpriseDB/mongo_fdw

Why not query the data directly in MongoDB?

Edit: changed "offline" to "non-realtime"

But even if you did that, you'll find that you'll need joins and aggregations that are painful to do in Mongo yet trivial to do in a system that is designed for them.