So the main issue with Spark is that the streaming is not that great. In general a Spark cluster persé is not bad, but I see more and more people using Apache Beam and then just use a runner that either they already have or that fits them best. https://beam.apache.org/