What we mean by that rule (general purpose data streaming engine) is just, that the product must be usable for other use cases than just this challenge. You can write your own data streaming engine, but it should be able to handle a wide variety of use cases, not just the challenge use case.

For instance, simply writing a program that loads 1 billion bytes into memory, iterates them and sums them, would not count as a "general purpose data streaming engine". But you don't have to use Spark, Kafka or something like that. You can write your own.