DNS Security for Defenders: DNSSEC, DoH and Registrar Hardening

DNS is the address book the entire internet trusts, and that trust makes it a prime target. If an attacker can change where your domain points, redirect your mail, or forge answers to your users' queries, they bypass most of your other defences before a single packet reaches your application. This defender-focused guide explains three distinct layers of DNS protection — DNSSEC for data authenticity, encrypted DNS (DoH and DoT) for query privacy, and registrar and account hardening for control-plane security — and shows how to detect tampering and harden each layer without breaking resolution.

Why DNS is such a high-value target#

DNS sits at the start of nearly every connection, and it was designed without authentication or confidentiality. Three properties make it attractive to attackers. First, a successful redirect is invisible: victims type the correct name and still land on hostile infrastructure. Second, DNS controls more than web traffic — it steers mail delivery (MX), certificate issuance validation, and service discovery, so control of DNS can enable email interception and even fraudulent TLS certificates. Third, the control plane (your registrar and DNS hosting accounts) is often weakly protected compared to production systems, making account takeover an efficient path to domain hijacking.

DNSSEC: authenticating the data, not the channel#

DNS Security Extensions (DNSSEC) add cryptographic signatures to DNS records so a validating resolver can prove that an answer genuinely came from the zone's owner and was not modified in transit. Each zone signs its records (RRSIG), publishes signing keys (DNSKEY), and a chain of trust is established from the root down through DS records at each parent. What DNSSEC gives you is origin authentication and integrity: it defeats cache poisoning and forged answers. What it does not give you is confidentiality — queries and answers remain readable on the wire — and it does not protect against a hijack of the account that lets an attacker legitimately re-sign the zone. Deploy it, but understand its scope.

Encrypted DNS: DoH and DoT for query privacy#

DNS over HTTPS (DoH) and DNS over TLS (DoT) encrypt the query between the client and the resolver, preventing on-path observers from reading or tampering with lookups. This closes a real privacy and integrity gap on untrusted networks. For defenders, encrypted DNS is double-edged: it protects your users, but it can also hide malicious lookups and command-and-control if endpoints bypass your resolver. The defensive posture is to provide an encrypted resolver you control and to steer or block unsanctioned public DoH so that DNS telemetry stays visible to your security stack. Encrypted DNS and DNSSEC are complementary: one protects the channel, the other authenticates the data.

The control plane: registrar and DNS account security#

The most damaging DNS incidents are usually not clever protocol attacks but plain account takeover. If an attacker phishes or credential-stuffs your registrar login, they can repoint nameservers, change MX records to intercept password resets, and request certificates for your domain. Hardening the control plane means enforcing phishing-resistant MFA on registrar and DNS-hosting accounts, enabling registrar lock (and registry lock for critical domains, which requires manual out-of-band verification to change), restricting who holds these credentials, and monitoring for unexpected changes. Treat these accounts with the same rigour as domain admin in your directory.

Attack surface and techniques to understand#

Defenders should recognise the failure modes at a conceptual level. Cache poisoning injects forged answers into a resolver's cache; DNSSEC validation and source-port randomisation are the countermeasures. Domain hijacking changes registration or nameserver data through account compromise; registrar/registry lock and MFA are the countermeasures. Subdomain takeover occurs when a DNS record still points to a de-provisioned cloud resource an attacker can re-claim; the fix is disciplined DNS hygiene and dangling-record scanning. Resolver abuse such as amplification uses open resolvers for DDoS; do not run open recursion. Knowing which control addresses which technique prevents wasted effort.

Detection: monitoring your own records#

The single most valuable DNS detection is change monitoring on your own zones. Periodically snapshot your authoritative records — NS, MX, A/AAAA, TXT, CAA, DNSKEY and DS — and alert on any diff you did not initiate through your change process. Watch nameserver (NS) and MX changes especially closely, because those are the high-impact hijack indicators. Monitor Certificate Transparency logs for certificates issued for your domains that you did not request, which can reveal a hijack or an unauthorised issuance. Track DNSSEC validation status so you learn immediately if signatures expire or the chain breaks, which causes outages that look like attacks.

Detection: query telemetry and passive DNS#

On the resolver side, log and analyse DNS queries from your environment. High-value detections include queries to newly registered or low-reputation domains, unusually long or high-entropy subdomain labels that suggest DNS tunnelling or exfiltration, spikes in TXT or NULL record queries, and endpoints reaching external DoH providers that bypass your sanctioned resolver. Enrich with passive DNS to see historical resolution of a domain and pivot during investigations. Feed resolver logs into your SIEM and baseline normal behaviour so beaconing intervals and rare destinations stand out. The goal is to keep DNS a source of visibility, not a blind spot.

Mitigation and hardening#

Bring the layers together into a hardening programme. Sign your zones with DNSSEC using modern algorithms and automate key rollover so signatures never expire unexpectedly. Publish a CAA record to restrict which certificate authorities may issue for your domain, cutting off unauthorised issuance. Enable registrar lock, and registry lock for your most critical domains. Enforce phishing-resistant MFA and least privilege on every registrar and DNS account. Run a validating, encrypted internal resolver, and steer clients to it while blocking rogue DoH. Regularly scan for dangling records that enable subdomain takeover, and remove DNS entries as part of decommissioning any cloud resource.

The DoH visibility dilemma#

Encrypted DNS deserves special attention because it changes your detection posture. When endpoints use public DoH, their lookups are invisible to network-based DNS monitoring, which malware increasingly exploits for stealthy command-and-control. The balanced answer is not to ban encryption but to own it: deploy your own DoH/DoT resolver, configure endpoints and browsers to use it via policy (canary domains and enterprise settings exist for exactly this), and detect or block connections to unsanctioned DoH endpoints at the egress. That preserves user privacy against outsiders while keeping the telemetry your defenders need.

Common pitfalls#

DNSSEC misconfiguration is the classic self-inflicted outage: an expired RRSIG or a mismatched DS record makes your whole domain unresolvable for validating resolvers, so automate rollover and monitor expiry. Forgetting CAA leaves any CA free to issue for you. Leaving registrar accounts on SMS-based MFA keeps them phishable. Assuming DNSSEC provides privacy is wrong — it does not encrypt anything. Assuming DoH provides authenticity is equally wrong — it secures the channel to the resolver but not the data's origin. And decommissioning a cloud service without removing its DNS record is the number-one cause of subdomain takeover. Each of these is avoidable with process discipline.

Hardening checklist#

1. DNSSEC signing enabled with automated key rollover and expiry monitoring. 2. CAA records published to constrain certificate issuance. 3. Registrar lock on all domains; registry lock on business-critical ones. 4. Phishing-resistant MFA and least privilege on registrar and DNS-hosting accounts. 5. Automated change monitoring on NS, MX, A/AAAA, TXT, CAA, DS records with alerting on unexpected diffs. 6. Certificate Transparency monitoring for your domains. 7. Internal validating, encrypted resolver with rogue-DoH steering/blocking and query logging into the SIEM. 8. Dangling-record scanning and DNS cleanup wired into decommissioning.

FAQ: does DNSSEC replace TLS or encrypted DNS?#

No. These solve different problems and you generally want all three. DNSSEC authenticates DNS data so you can trust the answer you received, but it sends that answer in the clear. Encrypted DNS (DoH/DoT) hides the query from network observers but does not prove the data's origin. TLS secures the connection to the destination after DNS resolution. A robust posture layers DNSSEC for integrity, encrypted DNS for query privacy, and TLS with CAA-constrained certificates for the connection itself.

FAQ: is DoH bad for enterprise security?#

DoH is a tool, not a threat by itself. It genuinely protects users on hostile networks. The risk to defenders is loss of DNS visibility when endpoints use external DoH resolvers that bypass corporate monitoring and can conceal malware traffic. The answer is to run your own encrypted resolver, push clients to it by policy, and detect or block connections to unsanctioned DoH providers at your egress. Done this way, you keep both user privacy against outsiders and the internal telemetry your detection depends on.

Conclusion#

DNS security is a layered discipline, and the biggest mistakes come from assuming one control covers everything. DNSSEC authenticates the data and defeats forgery; encrypted DNS protects the query in transit; registrar and account hardening protects the control plane where the most damaging hijacks actually happen. Around all three, continuous monitoring of your own records, Certificate Transparency and query telemetry turns DNS from a silent single point of failure into an early-warning system. Sign your zones, lock your registrar, own your resolver, and watch your records — and you remove one of the most powerful pivots an attacker can use against you.