Re:Ditto: Real-time sync for apps even without the internet
by techdragon on Saturday September 24, 2022 @12:43AM
(#32959797)
Thanks for the reply! I had a very quick look at the documentation and didn’t see the eviction information. That’s excellent to know it’s already a built in capability. As for space, don’t worry if you’ve never heard of CCSDS, it’s just yet another standards group for various things, the key thing is that it’s delta based and already designed to handle non-TCP-IP based data transfer mechanisms. One of the big challenges I’ve found with bringing Linux and other modern software into an environment where “normal” (you can get a 24/7 IP link, but it’s unusual for satellite to maintain an open link like that) space communication is that a lot of useful modern open source tools is just not designed for a non IP based networking environment, so much so that you dig into things like delay tolerant networks and you’ll find mechanisms for trying to fake TCP-IP over the top of them just to help accomodate this software. Having a mechanism to replicate important data on the basis of just a transmission mechanism agnostic “packet” of delta update data, would be a huge boon for low cost satellite systems.
Im definitely going to have to dig in deeper and check out the rust stack! I’ll be sure to get in touch about the various CCSDS data transmission related stuff once I get there, since if you’ve looked at Link16 I’m sure you’re probably interested in adding more aerospace communications link standards/mechanisms.
To help users with eviction, we are adding support for dynamic queries, for example, using "now" for a date in the query to aid in creating TTL logic. In addition, to built-in TTL as well.
Satellite comms are definitely a great fit for another reason, since Ditto's replication is "delta-based" it will only sync differences. Our protocol is designed to overlay on top of custom arbitrary links, including unreliable ones. Not familiar with CCSDS, but we have explored Link16, and even built-in we replicate over BLE GATT.
Hope that helps and perhaps we can get Ditto into space!
[1] https://docs.ditto.live/ios/common/concepts/syncing-data#evi...