One use case that comes to mind is backups. I can have a zpool created backed by a S3 vdev and then use zfs send | zfs recv to backup datasets to S3 ( or the billion other S3 like providers)

Saves me the step of creating an instance with EBS volumes and snapshotting those to S3 or whatever

haven't done the math at all on whether that's cost effective, but that's the usecase that comes to mind immediately