> Is it expected to be slow?
Probably, yes. The slowness is not on the Streamlit end, but on the Replicate API end. The docs for the 13b API [0] say:
> Predictions typically complete within 9 seconds.
Whereas for the 70b API [1]:
> Predictions typically complete within 18 seconds. The predict time for this model varies significantly based on the inputs.
Also how recent is the training data? When I asked what’s the date today? I received 22nd of March 2018.
Interesting choice of using streamlit for interface.