Re:Small File Archives in MinIO
by laurencerowe on Wednesday February 16, 2022 @05:47PM
(#30366900)
While not zip S3 does now support SELECT into a number of datatypes. https://docs.aws.amazon.com/AmazonS3/latest/userguide/select...
[ Parent
| on HN ]
Correct, but it pretty much forces a scan through the file. For AWS, you pay for scanning the file, even if only a few results are returned. Currently this is $0.002 per scanned GB, plus request and transfer fee.
For MinIO you "pay" in CPU time and disk iops to scan the file.
So for retrieving single values it is a very inefficient/expensive method.