...making Linux just a little more fun!

What's up with Google Public DNS?

By Rick Moen

A few days ago, out of nowhere, Google, Inc. announced that it was offering a new free-of-charge service to the public: robust recursive DNS nameservice from highly memorable IP addresses 8.8.8.8 and 8.8.4.4, with extremely fast responses courtesy of anycast IP redirection to send the query to the closest network cluster for that service (like what Akamai and similar CDNs do for Web content).

This service needs some pretty considerable funding for hardware and bandwidth, especially if its usage scales upwards as Google intends. Reportedly, it also developed a totally new recursive-only nameserver package, name not disclosed, in-house. According to my somewhat casual research, the software was probably written by Google developer Paul S. R. Chisholm, who doesn't seem to have any prior record of writing such software.

That is remarkable, to begin with, because recursive nameservers are notoriously difficult to write correctly, such that the history of open source DNS servers is littered with failed recursive-nameserver projects. (By contrast, the other types, authoritative, iterative, and forwarder, are much easier to write. Please see my separate article, "The Village of Lan, A Networking Fairy Tale", about the differences.)

The obvious question is, what's in it for Google? At first, I guessed it to be a research/prestige project, an opportunity for Chisholm and other technical staff to try their hand at managing major network infrastructure. It also already serves as DNS foundation for some internal Google services, such as the WiFi network for visitors to the Google campus, and for the free public WiFi network Google offers to residents of Mountain View, California, near its headquarters. That didn't really seem sufficient to justify the outlay required in expenses and technical resources, though.

My friend Don Marti suggested an alternative idea that sounds more promising: It's a "spamtenna", a distributed data-collection system that classifies DNS clients: those doing high numbers of DNS lookups of known spam-related hostnames, and regular DNS clients that don't. Then, other hostnames resolved more often by the first group than the second are noted as also probably spam-involved. Google can use that database of suspect hostnames in anti-spam service, to aid GMail antispam, etc., a usage model consistent with Google's privacy policy.

Is Public DNS an attractive offering? Does it have privacy problems? Yes and no.

For most people, probably even most Linux Gazette readers, DNS is just something automatically taken care of, which means it tends to get referred out to the local ISP's recursive nameservers. I shouldn't be unkind to ISPs, who do their best, but ISP recursive servers tend to be pretty bad. In general, they have spotty performance, often play tricks such as redirecting queries or ignoring time-to-live timeouts on DNS data, are security liabilities (on account of pervasive "cache poisoning" by malign parties elsewhere on the Internet), and of course are an obvious point where data collects about all of your DNS queries -- what you and your users ask about, and what is replied. Of course, your ISP already sees all all your incoming and outgoing packets -- but, unlike Google, are contractually obliged to deal with you in good faith and look after your interests.

That is one of the problems with Google's Public DNS offering, and, likewise, with its other free-of-charge services such as GMail (and, for that matter, with pretty much any unpaid "Web 2.0" service): Any contractual commitment to users is either non-existent or extremely weak. They say they have a strong privacy policy, with information about which of your IP addresses queried their servers being erased after 24-48 hours, but the fact is that you're just not a paying customer, so policies don't mean much, completely aside from the possibility of them being changed on a whim.

Another problem I've already alluded to: What software does it use, and is it reliable and correct? Google is telling the public nothing about that software, not even its name, except that it was developed in-house and is not to be open source in the foreseeable future. Most of the really good recursive nameserver packages were developed as open source (in part) to get the benefit of public debugging and patches. The rest took a great deal of time and money to get right.

One common comparison is to David Ulevich's OpenDNS free recursive service, which is competently run and keeps users away from malware/phishing hostnames. Unless you use the paid version, however, OpenDNS deliberately breaks Internet standards to return the IP of its advertising Web server whenever the correct answer is "host not found" (what is called NXDOMAIN in DNS jargon). Google Public DNS doesn't suffer the latter problem.

A third alternative: None of the above. Run your own recursive nameserver. For example, NLnet Labs's open-source Unbound is very easy to set up, requires no maintenance or administration, takes only about 11 MB RAM, and runs on Windows, Linux, or Mac OS X. It would be silly to run such a network service on every machine: Instead, run it on one of the machines of your home or office's LAN, and have all of the machines' DNS client configurations -- /etc/resolv.conf on Linux machines -- point to it. That way, there's a single shared cache of recently queried DNS data shared by everyone in the local network.

(Note: Most people have their machines' "resolv.conf" contents, along with all other network settings, populated via DHCP from a local network appliance or a similar place, and configure that to point clients' DNS to the nameserver machine's IP. For that reason, the nameserver machine should be one to which you've assigned a static IP address, and left running 24/7.

Hey, OpenWRT wireless-router firmware developers! Make me happy and put Unbound into OpenWRT, rather than just Dnsmasq, which is a mere forwarder, and Just Not Good Enough. Dnsmasq forwarding to Unbound, however, would be really good, and putting recursive DNS in the same network appliance that hands out DHCP leases is the ideal solution for everyone. Thanks!)

It seems that hardly anyone other than us networking geeks bothers to have real, local nameservice of that sort -- a remarkable example of selective blindness, given the much snappier network performance, and greater security and control, that doing it locally gains you for almost no effort at all.

In fact, what's really perplexing is how many people, when I give them the above suggestion, assume either that I must be exaggerating the ease of running a recursive nameserver, or that "professional" options like their ISP, OpenDNS, or Google Public DNS must actually be faster than running locally, or both. Let's consider those: (1) When I say there's nothing to administer with recursive nameserver software, I mean that quite literally. You install the package, start the service, it runs its cache by itself, and there are no controls (except to run it or not). It's about as foolproof as network software can get. (2) When your DNS nameserver is somewhere on the far side of your ISP uplink, then all DNS queries and answers must traverse the Internet both ways, instead of making a local trip at ethernet speeds. The only real advantage a "professional" option has is that its cache will be already well stocked, hence higher percentage of cache hits relative to a local nameserver when the latter first starts, but then there will be essentially no difference and the local service will thereafter win because it's local.

Why outsource to anyone, when you can do a better job locally, at basically no cost in effort?

Recursive Nameserver Software for Linux

Late Addendum: The Real World Sends Rick a Memo

This might be you: You have one or more workstation or laptop on either a small LAN or direct broadband (or dialup) connection. You're (quite rightly) grateful that all the gear you run gets dynamic IP configuration via DHCP. The only equipment you have drawing power on a 24x7 basis is (maybe) your DSL modem or cable modem and cute little network gateway and/or Wireless Access Point appliance (something like a Linksys WRT54G). You don't have any "server" other than that.

Mea culpa: Being one of those oddballs who literally run servers at home, I keep forgetting you folks exist; I really have no excuse.

Shortly after I submitted the first version of this article to LG, a friend on my local Linux user group mailing list reminded me of the problem: He'd followed my advice and installed Ubuntu's Unbound package on his laptop, but then (astutely) guessed it wasn't being sent queries, and asked me how to fix that. His house had the usual network gateway appliance, which (if typical) runs Dnsmasq on embedded Linux to manage his Internet service, hand out DHCP leases on a private IP network, and forward outbound DNS requests to his ISP nameserver.

As I said above, it would be really nice, in such cases, to run a recursive nameserver (and not just Dnsmasq) on the gateway appliance, forwarding queries to it rather than elsewhere. That's beyond the scope of my friend's immediate needs, however. Also, I've only lately, in playing with OpenWRT on a couple of actual WRT54G v.3 devices received as gifts, realised how thin most such appliances' hardware is: My units have 16 MB total RAM, challenging even for RAM-thrifty Unbound.

My friend isn't likely to cobble together a static-IP server machine just to run a recursive nameserver, so he went for the only alternative: running such software locally, where the machine's own software can reach it as IP 127.0.0.1 (localhost). The obstacle, then, becomes DHCP, which by design overwrites /etc/resolv.conf (the DNS client configuration file) by default every time it gets a new IP lease, including any "nameserver 127.0.0.1" lines.

The simple and elegant solution: Uncomment the line "#prepend domain-name-servers 127.0.0.1;" that you'll find already present in /etc/dhcp3/dhclient.conf . Result: The local nameserver will be tried first. If it's unreachable for any reason, other nameserver IPs supplied by DHCP will be attempted in order until timeout. This is probably exactly what people want.

(If, additionally, you want to make sure that your DHCP client never requests additional "nameserver [IP]" entries from the DHCP server in the first place, just edit the "request" roster in /etc/dhcp3/dhclient.conf to remove item "domain-name-servers".)

Second alternative: Install package "resolvconf" (a utility that aims to mediate various other packages' attempts to rewrite /etc/resolv.conf), and add "nameserver 127.0.0.1" to /etc/resolvconf/resolv.conf.d/head so that this line always appears appears at the top of the file. This is pretty nearly the same as the first option.

Last, the time-honoured caveman-sysadmin solution: "chattr +i /etc/resolv.conf", i.e., setting the file's immutable bit (as the root user) so that neither your DHCP client nor anything else can alter the file until you remove that bit. It's hackish and unsubtle, but worth remembering for when only Smite with Immense Hammer solutions will do.


Talkback: Discuss this article with The Answer Gang

Bio picture Rick has run freely-redistributable Unixen since 1992, having been roped in by first 386BSD, then Linux. Having found that either one sucked less, he blew away his last non-Unix box (OS/2 Warp) in 1996. He specialises in clue acquisition and delivery (documentation & training), system administration, security, WAN/LAN design and administration, and support. He helped plan the LINC Expo (which evolved into the first LinuxWorld Conference and Expo, in San Jose), Windows Refund Day, and several other rabble-rousing Linux community events in the San Francisco Bay Area. He's written and edited for IDG/LinuxWorld, SSC, and the USENIX Association; and spoken at LinuxWorld Conference and Expo and numerous user groups.

His first computer was his dad's slide rule, followed by visitor access to a card-walloping IBM mainframe at Stanford (1969). A glutton for punishment, he then moved on (during high school, 1970s) to early HP timeshared systems, People's Computer Company's PDP8s, and various of those they'll-never-fly-Orville microcomputers at the storied Homebrew Computer Club -- then more Big Blue computing horrors at college alleviated by bits of primeval BSD during UC Berkeley summer sessions, and so on. He's thus better qualified than most, to know just how much better off we are now.

When not playing Silicon Valley dot-com roulette, he enjoys long-distance bicycling, helping run science fiction conventions, and concentrating on becoming an uncarved block.

Copyright © 2010, Rick Moen. Released under the Open Publication License unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 170 of Linux Gazette, January 2010

Tux