The Geocities Torrent (~1TB of awesomeness)

Re:The Geocities Torrent (~1TB of awesomeness) by jacquesm on Wednesday October 27, 2010 @06:29PM (#1840737)

I would be very surprised if there is anything in there that is not in reocities yet but I will certainly do a comparison.

I'll have to run that against all the deletion requests as well for this content because chances are fairly large that lots of it has already been removed at the request of the owners.

[ Parent | on HN ]

Re:The Geocities Torrent (~1TB of awesomeness) by a1k0n on Wednesday October 27, 2010 @06:40PM (#1840768)

Have you tried de-duping files to save space? I imagine there's a finite number of MIDI files and "under construction" GIFs that could just be symlinked to save a ton of space...

[ Parent | on HN ]

Re:The Geocities Torrent (~1TB of awesomeness) by jacquesm on Wednesday October 27, 2010 @06:51PM (#1840795)

Space is not the problem, I've got plenty of that. The bigger problem is that the underlying filesystem is having some problems dealing with the total number of files.

Total disk space usage on the array that holds reocities is about 10.5 T (that includes a master copy though) and runs to 233 million files.

[ Parent | on HN ]

Re:The Geocities Torrent (~1TB of awesomeness) by bobf on Wednesday October 27, 2010 @07:21PM (#1840903)

Have you considered implementing a NoSQL solution to store the files? I've successfully done something similar with HBase.

[ Parent | on HN ]

Re:The Geocities Torrent (~1TB of awesomeness) by jacquesm on Wednesday October 27, 2010 @07:50PM (#1840992)

No, but that's a good idea, I'll definitely look in to that when the next round of reocities improvements rolls around.

[ Parent | on HN ]