News for hackers. Stuff that matters.
Re:Principles of Content Addressing (2014) by btrask on Tuesday August 04, 2015 @05:08PM (#10006299)
I'm happy to answer any questions about this article or the implementation I've put together.
Re:Principles of Content Addressing (2014) by lisper on Tuesday August 04, 2015 @08:51PM (#10007411)
You say you've been working on your CAS system for two years. What are the challenges you are facing that is making it take that long? Because a simple CAS is trivial: just hash the file, rename it to the hash, and serve it up via a normal HTTP server. What is making it hard?
Re:Principles of Content Addressing (2014) by btrask on Wednesday August 05, 2015 @12:23PM (#10010888)
Here's what I've written up for a toy content addressing system I've been writing in Python:

"If it's so easy, why did it take us 3 years and 10,000 lines?" "Because we wrote it in C."

And because the real system:

- Tracks file meta-data (including tags and full-text indexing)

- Has a way to find files/hashes without file or hash (search)

- Uses a database instead of hardlinks everywhere

- Supports hash algorithm plugins (not technically done yet)

- Supports more hash formats (short, base-64, etc.)

- Is ACID (and coordinates transactions between DB and file system)

- Supports multiple repos per user (at arbitrary paths)

- Handles hash collisions (aside from the primary hash)

- Has a basic user interface

- Has a sync protocol