A recent post by Krebs On Security about a mastercard mistake in their nameservers got me thinking. I recently got access to the .com zone file, I could just… grep for this common type of mistake.
Towards the bottom of the article, he mentions that there is an awsdns-06.ne registered some time ago that fits a similar bill. This sparked a particular interest, because I happen to know that AWS uses an absolute ton of domains for their dns infrastructure, each would have to be registered individually in order to mitigate this issue.
I downloaded the .com zone file and ran a quick grep to look for specifically this type of issue. E.g. domains that have a .ne nameserver. The same exact issue could reasonably apply to .co instead of .com, but there is no .or to make the mistake with .org.
rg -e "\.ne\.$" 2025-01-22-com.zone | tee 2025-01-22-niger_nameserver_results
After this finished processing the 22GB zone data for .com, I was left with 1820 potential sites of interest.
➜ wc -l 2025-01-22-niger_nameserver_results
1820 2025-01-22-niger_nameserver_results
From here, it’s time to attempt an A record lookup on every nameserver in the list. My thought here is that if it’s an intentional, valid nameserver in the .ne domain, it will return an answer for an A record – if it doesn’t return an A record, chances are pretty high that this is a misconfiguration. A quick look at RFC 2181 confirms my guess that NS records should not be CNAMEs, so this check should be pretty valid (the whole IPv6 of it aside).
I wrote a simple python script using dnspython to iterate through the list of records in the zone, extracted the impacted domain as well as the nameserver address that matched our previous grep, and then attempted to resolve an A record. If it resolved, great, do nothing. If it didn’t resolve, add it to a dictionary and add the record that references it to the list of records pointing to that nameserver. The code looks roughly like this:
from collections import defaultdict
import dns.resolver
import sys
def extract_domain(line):
"""Extract the first value (domain) and last value (nameserver domain) from the given line."""
parts = line.strip().split()
if len(parts) >= 5:
return parts[0], parts[-1].rstrip('.')
return None
def process_file(file_path):
failed_domains = defaultdict(list)
resolver = dns.resolver.Resolver()
resolver.nameservers = ["8.8.8.8"]
with open(file_path, 'r') as file:
for line in file:
impacted_domain, domain = extract_domain(line)
if domain:
if domain in failed_domains:
failed_domains[domain].append(impacted_domain)
else:
try:
resolver.resolve(domain, 'A')
except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN, dns.exception.DNSException) as e:
failed_domains[domain].append(impacted_domain)
for k,v in failed_domains.items():
print(f"{k}," + ','.join(v))
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python main.py <input_file>")
sys.exit(1)
input_file = sys.argv[1]
process_file(input_file)
I then redirected the output of this to a file and found 858 unique invalid .ne nameservers being referenced (as of the time of writing, 2025-01-22) in the .com zone. Of these, awsdns based nameservers accounted for 285 unique invalid nameserver records, with 64 unique apex domains referenced. Based on a quick search on Gandi, that’ll only be somewhere in the range of $38,400 worth of Niger domains for AWS to squat if they want to protect their users against this.
Fallout
Going through the list of unique domains, there are a wide variety of types of domains impacted. Law firms and doctor’s offices, mail services, farms, motorcycle repair shops, home remodeling services, flower shops, and the list goes on. But perhaps most entertainingly, a surprising number of small tech companies and cybersecurity services. Not big ones, mind you, but there’s just something funny about a company positioning itself as a security service and having invalid nameservers configured.
I will not be making the list of invalid nameservers or the list of impacted domains available at this time. The sheer number of results is high enough that I can’t really know whats in here or what the impact could be to users of these services.
How do we prevent this?
Based solely on the number of impacted domains, 1820 impacted domains out of some 150 million + domains (and 404564680 invidual NS records) in the .com zone is not a huge deal. But on the flip side, I can’t really think of a reason why any registrar would ever let their users configure nameservers that don’t exist. I feel like registrars could almost completely fix this problem if they wanted to.
Of course, there are tons of registrars and they would each have to fix it to prevent their customers from making mistakes. On the other hand, since the nameservers for a domain have to be registered with the TLD operator anyways, the TLD operators could fix this. There are only 1445 TLDs at the time of writing, according to IANA. And a bunch of TLDs are all operated by the same entities, so probably a considerably smaller number of entities that would actually need to make the changes.
But it does present some interesting logistical challenges. I could put a valid DNS name in for my nameserver and it could be valid for days or weeks or years, then become invalidated at some point in the future. What would I expect to happen then, if registrars or TLD operators did choose to do something to mitigate this issue? Would they let me know and have me remove it? Would they remove it for me automatically? I don’t really know.
Ultimately, I think there are things that could be done to reduce the risk of these misconfigured nameservers. But on the flip side, there are an infinite number of ways to mess up your nameservers, and this is just one. Is it worth the effort? I don’t know that either. But I do know that if we continue to just rely on every domain administrator to keep getting it right, every once in a while one of them will get it wrong. So I’m just trying to think of technical controls we could implement to mitigate that.
Outro
It’s very cool that Philippe Caturegli found this happening on a mastercard domain. That’s a big find in terms of impact. But unfortunately it’s also just a drop in the bucket when it comes to these sorts of invalid DNS configurations.
I’m working on tooling to help enable and enrich this sort of research through my company, Room 641A.