Re:Tachyon – Memory Centric Reliable Distributed Storage
by jnaour on Thursday November 05, 2015 @08:53AM
(#10513260)
Anyone has experience with this project? We are currently thinking about using it in our cluster with Spark Streaming instead of an external DB (MongoDB)
[ Parent
| on HN ]
Re:Tachyon – Memory Centric Reliable Distributed Storage
by doug1001 on Thursday November 05, 2015 @07:50PM
(#10517198)
we use it, but (after reading the homepage linked to at the bottom) pretty clearly outside the core use case. In particular, we use it for off-heap caching, which we configure through org.apache.spark.StorageLevel. We monitor via tachyoneFolderName which as you probably know is a spark context (sc) attribute (eg, ./bin/spark-shell, then type "sc."[Tab] to find it).