Serializable, Lockless, Distributed: Isolation in CockroachDB

Re:Serializable, Lockless, Distributed: Isolation in Cockroa... by LoSboccacc on Thursday May 05, 2016 @12:27PM (#11637505)

Thanks! That's fantastic to hear even if it's going to materialize later/eventually/never at least it's great to know the need is recognized.

Disregarding write amplification issues for a sec, would it make things better to split binaries in 64k chunks and have them in a chunk table keyed by name and offset?

[ Parent | on HN ]

Re:Serializable, Lockless, Distributed: Isolation in Cockroa... by hnkimb3558 on Thursday May 05, 2016 @01:09PM (#11637920)

I'm not really sure which strategy would benchmark best between 64K chunks, 1M chunks, or even 8M chunks. I think this requires some experimentation. Pushing them all through as 64K chunks has a lot of overhead, and you'd reap the full write amplification. Could you tell me a bit more about your use case? You can email me spencer at cockroachlabs.com.

I've been meaning to work on the "CockroachDB Egg Store" (my disgusting name for a blob storage subsystem) as a Free Fridays side project for a while, but have been distracted with all manner of other enticing options. There aren't enough hours in a week...

[ Parent | on HN ]