News for hackers. Stuff that matters.
Re:We Saved $500k per Year by Rolling Our Own "S3" by codedokode on Sunday October 26, 2025 @10:20PM (#45716759)
With s3, you cannot use ls, grep and other tools.

> Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and...

Wow that's a lot to learn before using s3... I wonder how much it costs in salaries.

> With what network topology?

You don't need to care about this when using SSDs/HDDs.

> With what access policies?

Whichever is defined in your code, no restrictions unlike in S3. No need to study complicated AWS documentation and navigate through multiple consoles (this also costs you salaries by the way). No risk of leaking files due to misconfigured cloud services.

> With what backup strategy?

Automatically backed up with rest of your server data, no need to spend time on this.

Re:We Saved $500k per Year by Rolling Our Own "S3" by rajamaka on Sunday October 26, 2025 @10:32PM (#45716840)
> You don't need to care about this when using SSDs/HDDs.

You do need to care when you move beyond a single server in a closet that runs your database, webserver and storage.

> No risk of leaking files due to misconfigured cloud services.

One misconfigured .htaccess file for example, could result in leaking files.

Re:We Saved $500k per Year by Rolling Our Own "S3" by codedokode on Monday October 27, 2025 @12:17AM (#45717338)
> One misconfigured .htaccess

First, I hope nobody is using Apache anymore, second, you typically store files outside of web directory.

Re:We Saved $500k per Year by Rolling Our Own "S3" by pestaa on Monday October 27, 2025 @04:12AM (#45718463)
Why nobody should use Apache? I rediscovered it to be great in many use cases. And there's llms to help with the config syntax.
Re:We Saved $500k per Year by Rolling Our Own "S3" by codedokode on Monday October 27, 2025 @04:27AM (#45718535)
Performance not great compared to nginx.
Re:We Saved $500k per Year by Rolling Our Own "S3" by AdieuToLogic on Sunday October 26, 2025 @11:59PM (#45717280)
>> No risk of leaking files due to misconfigured cloud services.

> One misconfigured .htaccess file for example, could result in leaking files.

I don't think you are making a compelling case here, since both scenarios result in an undesirable exposure. Unless your point is both cloud services and local file systems can be equally exploited?

Re:We Saved $500k per Year by Rolling Our Own "S3" by Nextgrid on Sunday October 26, 2025 @11:38PM (#45717194)
With bare-metal machines you can go very far before needing to scale beyond one machine.
Re:We Saved $500k per Year by Rolling Our Own "S3" by inlined on Sunday October 26, 2025 @10:37PM (#45716872)
It sounds like you’re not at the scale where cloud storage is obviously useful. By the time you definitely need S3/GCS you have problems making sure files are accessible everywhere. “Grep” is a ludicrous proposition against large blob stores
Re:We Saved $500k per Year by Rolling Our Own "S3" by coderintherye on Sunday October 26, 2025 @10:27PM (#45716803)
I mean you can easily mount the S3 bucket to the local filesystem (e.g. using s3fs-fuse) and then use standard command line tools such as ls and grep.
Re:We Saved $500k per Year by Rolling Our Own "S3" by hallman76 on Sunday October 26, 2025 @10:48PM (#45716926)
I inherited an S3 bucket where hundreds of thousands of files were written to the bucket root. Every filename was just a uuid. ls might work after waiting to page though to get every file. To grep you would need to download 5 TB.
Re:We Saved $500k per Year by Rolling Our Own "S3" by codedokode on Sunday October 26, 2025 @10:36PM (#45716861)
It's probably going to be dog slow. I dealt with HDDs where just iterating through all files and directories takes hours, and network storage is going to be even slower at this scale.