Re:Streamz: Python pipelines to manage continuous streams of...
by SpaceManNabs on Thursday March 26, 2020 @10:37PM
(#22700038)
Nice. This would help solve the issues of deploying pytorch/tensorflow models in a streaming environment (Apache Beam do not make it easy). I am curious as to how performant this is when compared to Flink or Spark Streaming/Structured Streaming. When you deploy deep learning models in pyspark, you run into performance issues from serializing from jvm to python.
edit: just found out flink now has a python api! so include it along in the comparison. not sure if the apache flink api also has serialization overhead.
edit: just found out flink now has a python api! so include it along in the comparison. not sure if the apache flink api also has serialization overhead.