News for hackers. Stuff that matters.
Re:A distributed Posix file system built on top of Redis and S3 by 14u2c on Tuesday January 12, 2021 @10:36PM (#25757714)
Out of curiosity how do you use a DNS service as a database?
Re:A distributed Posix file system built on top of Redis and S3 by QuinnyPig on Wednesday January 13, 2021 @12:31AM (#25758582)
Excellently. I use it as a database excellently: https://www.lastweekinaws.com/blog/route-53-amazons-premier-...
Re:A distributed Posix file system built on top of Redis and S3 by hilbertseries on Wednesday January 13, 2021 @02:16AM (#25759173)
OOC, what are the downsides of storing service discovery information like this in TXT records? As opposed to using ZK, consul, etcd or a more standard solution?
Re:A distributed Posix file system built on top of Redis and S3 by QuinnyPig on Wednesday January 13, 2021 @02:22AM (#25759204)
There are better tools for the job now. This is basically a misuse of a service instead of using things that are purpose built.

I mean, your inventory is either a grep or a zone transfer this way, for one.

Re:A distributed Posix file system built on top of Redis and S3 by paulfurtado on Wednesday January 13, 2021 @05:07AM (#25760179)
A zone transfer or grep really isn't so terrible, even for someone who doesn't know much about dns it's just a quick memorized dig away. HTTP or grpc aren't really better CLI UX and if this is your infra you can just write a cli to display your dns-driven information cleanly.

Storing key-value pairs in TXT records is easily parseable in any language if you use a simple delimiter. This isn't even an abuse of DNS, the protocol has been used to serve arbitrary data forever.

All the other modern services like consul, etcd, zk, etc require keeping a quorum of servers happy and have pretty heavy clients. By contrast, it's so hard to take down DNS and spinning up a new server is as easy as copying a zone file to a new server and the new server could even be running a totally different dns implementation because zone files are so standardized. Plus, your tooling can directly parse zone files with whatever dns library you were using and have a trivial way to dump the data without any server at all.

DNS can be replicated in arbitrary configurations and everything supports DNS caching for really high HA.

When you do want dynamic discovery and don't want to implement direct zone file generation, there's always CoreDNS which has plugins for so many datasources.

And if you don't want to host it, there are tons of DNS providers with great uptime.

At this point, I really can't think of any solution for service discovery that's better than DNS for most cases. Especially since the majority of service discovery solutions end up returning hostnames instead of IP addresses so you're already taking a dependency on DNS. Other solutions only really add benefit if you need to store tons of metadata or take advantage of things like leader election, etc

Re:A distributed Posix file system built on top of Redis and S3 by casselc on Wednesday January 13, 2021 @02:37AM (#25759297)
If only AWS Cloud Map wasn't $0.10/resource outside of ECS you could reasonably have your API cake and query it using DNS too.
Re:A distributed Posix file system built on top of Redis and S3 by dionian on Wednesday January 13, 2021 @09:42AM (#25762520)
What’s a good example of a better tool
Re:A distributed Posix file system built on top of Redis and S3 by geekthattweaks on Tuesday January 12, 2021 @10:49PM (#25757805)
You use text records and have a key value pair in a record.