Re:Principles of Content Addressing (2014)
by btrask on Tuesday August 04, 2015 @05:08PM
(#10006299)
I'm happy to answer any questions about this article or the implementation I've put together.
[ Parent
| on HN ]
Re:Principles of Content Addressing (2014)
by lisper on Tuesday August 04, 2015 @08:51PM
(#10007411)
You say you've been working on your CAS system for two years. What are the challenges you are facing that is making it take that long? Because a simple CAS is trivial: just hash the file, rename it to the hash, and serve it up via a normal HTTP server. What is making it hard?
Re:Principles of Content Addressing (2014)
by btrask on Wednesday August 05, 2015 @12:23PM
(#10010888)
Here's what I've written up for a toy content addressing system I've been writing in Python:
"If it's so easy, why did it take us 3 years and 10,000 lines?" "Because we wrote it in C."
And because the real system:
- Tracks file meta-data (including tags and full-text indexing)
- Has a way to find files/hashes without file or hash (search)
- Uses a database instead of hardlinks everywhere
- Supports hash algorithm plugins (not technically done yet)
- Supports more hash formats (short, base-64, etc.)
- Is ACID (and coordinates transactions between DB and file system)
- Supports multiple repos per user (at arbitrary paths)
- Handles hash collisions (aside from the primary hash)
- Has a basic user interface
- Has a sync protocol