nickaggarwal

I agree, if you go with the wrong solutions, it can inflate the costs

With Fuse you can make it transparent for the Application, it just exposes the mount with all the files. When your application reads them, it's pulled from Object storage, while az-copy is a utility to copy it to your disk

This was our exact use case backing store from nfs to object store. You can cloud cloud-specific mount providers and use a thin client in the middle

Will test out Weka, Thanks for sharing

Yes there is AWS FSx with lustre in the blog..that might be worth checking out

If you want de couple model loading and distribution, If you do it in application when you need application pulling from bucket can be slow

This is a great guide