dnsmasq can read from /etc/hosts so you can automate it just using `curl` (or other standard UNIX tools):

   # the first command is a one time only job
   cp -vip /etc/hosts /etc/hosts.default

   # the following is the automation.
   curl http://someonewhocares.org/hosts/hosts --silent > /etc/hosts.sowc
   cat /etc/hosts.default /etc/hosts.sowc >> /etc/hosts

The other benefit of this is you can then just do a `dnsmasq reload` (since the hosts file is an external configuration file) rather than the full `dnsmasq restart` that dnsgate performs.

Silently putting data fetched over unauthenticated HTTP into your hosts file doesn't sound like a great idea.

Good point. Sadly that's pretty much how all adblockers keep up-to-date (albeit at least with TLS). Though pragmatically even if the update was manual, you still need to place some trust in the site when dealing with files of that size. Granted it would be better if those resources served TLS, but that would only prevent MITM and not the file being dodgy from source (eg site hacked).

Thinking a little more about the problem you raised, I guess you could grep it for IPs that aren't localhost (ie exit the shell script before writing the new hosts if `grep` returns a zero exit code). Aside that, I'd genuinely be interested in any ideas you have for keeping the file updated in a secure way (if at all possible).

https://gist.github.com/Holzhaus/ed4ac1675a57f11c3057 contains a simple reverse grep for doing this.

You should still have a modicum of trust towards the host you're pulling from, as they can also insert something to potentially break the parsing code, which opens up a whole 'nother host of issues.

True. In dnsgate's case the domains are extracted and guaranteed (real hostname validation is on the todo) to point to 127.0.0.1 (or whatever '--dest-ip' is).

But having the entries in /etc/hosts should on Unix in most cases cause your computer to find it there right away without DNS lookup. Are you using dnsmasq locally on the host computer or to serve other computers as well? In the latter case, I see the point (though if I were to use dnsmasq I wouldn't intermix the hosts file).

There wouldn't be any point running dnsmasq only for a local machine :)

RE intermixing hosts file: dnsmasq supports reading from additional hosts files other than /etc/hosts (thank to creshal for that suggestion: https://news.ycombinator.com/item?id=10580082)

I run dnsmasq locally (and silently redirect all traffic from a certain uid with an iptables rule) to do adblocking and ... minimal parental control on my son's laptop. it is pretty great, I don't have to worry about him using the neighbor's wifi to get around my rules. ;)

Another benefit is it caches DNS lookups.

is this on a PC or a router?

On a laptop. Otherwise I wouldn't know the uid. All users (including services) are allowed outgoing port 53/udp, except my son's user.

dnsmasq has an addn-hosts parameter, so you don't have to thrash your main hosts file.

I'm trying to think of a use case for addn-hosts in dnsgate. If the user is not using dnsmasq, then they are stuck with /etc/hosts, if they are using dnsmasq then they might as well use a dnsmasq syntax file so '--block-at-tld' can be used. dnsmasq.conf has an include syntax:

echo "conf-file=/etc/dnsgate/generated_blacklist.conf" >> /etc/dnsmasq.conf

Oh that's handy. Would also make the update even easier since you no longer need to preserve any host-specific entries so a simple one liner in `curl` or `wget` would be enough.

Thanks for the recommendation.

dnsmasq can read /etc/hosts, but /etc/hosts format can't do * blocking. So if you are using /etc/hosts (with dnsmasq) '--block-at-tld' wouldn't work.

Moving /etc/hosts to /etc/hosts.default is much better than the 'hosts example'. Updated it. Thanks. Still experimenting with 'dnsmasq reload' but if it works as advertised that's more general than restarting the service.

That doesn't return NXDOMAIN though, that returns localhost, no?

Correct. And it would be a problem if something was listening on 80/443/whatever. That's why '--dest-ip' can be set.

from man dnsmasq:

  -B, --bogus-nxdomain=<ipaddr>
 Transform replies which contain the IP address given into "No such domain" replies.
 This is intended to counteract a devious move made by Verisign in September 2003
 when they started returning the address of an advertising web page in response to
 queries for unregistered names, instead of the correct NXDOMAIN response. This
 option tells dnsmasq to fake the correct response when it sees this behaviour. As
 at Sept 2003 the IP address being returned by Verisign is 64.94.110.11

Is there another way to get dnsmasq to return NXDOMAIN? If I am reading this correctly, one would need to do 'dnsgate --dest-ip 127.2.2.2' and then tell dnsmasq '--bogus-nxdomain=127.2.2.2'.

Only just followed up on this thread. Yes. My dnsmasq.conf contains:

  servers-file=/usr/local/etc/dnsmasq-servers.conf

and then that file contains a lot of lines like:

  server=/taboola.com/

The docs don’t tell you that this results in NXDOMAIN for the server whose IP you didn’t specify. It seems to work!

Much better! Fixed: https://github.com/jakeogh/dnsgate/commit/62bfa78ac27586d048...

That's fantastic, thank you for the tip!