News for hackers. Stuff that matters.
Re:DNS Key Value Storage by ignoramous on Monday January 04, 2021 @09:29AM (#25631733)
You'd think DNS KV Store's a "nice figment of imagination", but one of the teams I worked for did use DNS TXT records genuinely as a (faster) way to reconcile state between the control plane and the data plane.

It sounded ugly on-paper, but the design was approved by everyone in the chain of command for what I presume was for lack of other viable options given the architecture.

And in ironic turn of events, instead of using DNS for service discovery, another set of engineers in the team opted for EC2 metadata (for example, security groups) instead.

Re:DNS Key Value Storage by inopinatus on Monday January 04, 2021 @04:26PM (#25638317)
It should not be surprising to anyone that long-established protocol standards are capable and versatile. DNS and LDAP are robust tree-structured replicated attribute stores; mix in Kerberos for a complete foundation of general purpose directory services. NNTP supplies a straightforward Gossip protocol for eventually-consistent distributed pub/sub. MQTT is still good for lightweight telemetry. I have services I authenticate with via Lamport's mostly forgotten scheme (S/KEY). And so on.

There is a current fad of HTTP+JSON as a generic protocol substrate, but it's almost always a mediocre fit for the problem at hand. Go read the RFC archives, there's diamonds to be found.

On the other hand, and perhaps even by the same token: using EC2 metadata isn't a completely terrible idea if it's the infrastructure you already have and the semantics line up with specific needs.

Re:DNS Key Value Storage by OJFord on Monday January 04, 2021 @09:56AM (#25632059)
I did something similar (for personal use) for a sort of 'peer discovery' where load balancing didn't actually matter, I just wanted to pick one of N without knowing where they are ahead of time, using IPVS: https://github.com/OJFord/ipvs-txt-lb

To be honest I'm not sure if it's a total hack or a reasonable use as intended, or somewhere in between. But it got it done, and is at least more elegant than hard coding something arbitrary which may change (just not too frequently), I think.

Re:DNS Key Value Storage by tyingq on Monday January 04, 2021 @06:31PM (#25639752)
DNS SRV records seem appropriate for that.
Re:DNS Key Value Storage by alexfrydl on Monday January 04, 2021 @09:59AM (#25632089)
Using TXT records for this kind of thing is not that unusual. For example, Kubernetes's external-dns uses TXT records to track which entries it manages.
Re:DNS Key Value Storage by strictfp on Monday January 04, 2021 @01:25PM (#25635671)
I've used the AWS metadata trick as well. I kind of liked the solution; better use something that the machine is already relying on than adding something extra.

DNS is also usually a much better option than the framework-specific service discovery mechanism that every framework seems to be destined to reinvent. I think the problem is that most computer scientists aren't very well-versed in networking, and DNS doesn't look like the best match at first glance.

Re:DNS Key Value Storage by aECBNhWl4sqXr2 on Monday January 04, 2021 @03:50PM (#25637726)
I've done the same - tag EBS volumes with metadata rather than having to maintain a separate datastore for a tiny amount of information. Worked well.