DNS Privacy (DoT & DoH) & Enterprise Security

Written by: George Grzyb, Principal Engineer
Connect with George on LinkedIn

Domain name system (DNS) privacy and security are two considerations with competing goals. In Ransomware Series Part 3, we discussed the importance of DNS to prevent the initial attack vector for ransomware and malware propagation. Today, we will go a bit deeper into the newer DNS protocols, which further muddy the debate of privacy versus security within the enterprise. DNS over transport layer security (TLS) (“DoT”)  and DNS over HTTPS (“DoH”), undermine enterprise security for the benefit of privacy. EDNS0 Client Subnet (ECS) erodes privacy for distributed cloud-based resources. Let’s look at how these standards work and the DNS security practitioner’s options in defense or offense.


The Problem with DNS

The challenge with DNS is the ability to screen and selectively block outbound requests along with some level of monitoring or alerting. Without an adequately architected or configured environment, many out-of-band methods of communication will facilitate DNS exfiltration.

Consider the following: If you are using a commercial DNS solution, are all recursive requests forced to go solely through these name servers? If legacy non-decommissioned or rogue DNS name servers exist on the network, users may still use their services to bypass the approved DNS name servers. Another typical blind spot is the edge firewall configuration (or lack thereof). Clients can still access public DNS directly if you are not blocking all outbound DNS (UDP/53 and TCP/53) to the Internet and are only permitting the approved DNS name servers. The purpose of a commercial DNS solution is to provide an approved DNS choke point, allowing for inspection and control of both user and server DNS communications.

How is the DNS solution architected for semi-trusted or DMZ environments? Users and servers in these security zones can use sanctioned recursive DNS name servers or public resolvers. More often than not, security misconfigurations permit unfiltered DNS requests. All clients, users, and servers must resolve internal or Internet-based names to access resources. However, malware such as ransomware utilizes these same channels, and it is imperative to have the appropriate controls to monitor, block, and alert malicious activities. Now, even if you have taken all necessary steps to control enterprise DNS traffic, DoT and DoH have been gaining traction and making things more difficult. This is yet another way for DNS communications to occur while bypassing security controls.


DoT and DoH

DoT and DoH are standards for encrypting DNS to access the Internet securely. These standards evolved for several reasons, including geopolitical pressures requiring users to surpass security controls and general privacy requirements due to imposed Internet service provider (ISP) tracking or application and streaming services providing selective content availability. Additional privacy is acceptable for users that may need to get past government controls, or perhaps when you cannot watch the New England Patriots stream on your smart TV while in the New York City market (as a fellow New Englander, I feel this pain when visiting my parents in New Jersey). However, for employee users within an organization, this is a blind spot for network and security professionals.

Keep in mind that malware, ransomware, and other malicious tracking software tend to use these same standards, and sometimes their bespoke implementations, to exfiltrate intellectual property (IP) and command-and-control (CC) communications via DNS traffic. Let’s tackle each standard individually to understand better how we can implement the appropriate network and security controls.


DNS over TLS

DoT essentially uses TLS to encrypt user datagram protocol (UDP) based DNS queries. Generally, DoT will use TCP/853 between client and server for establishing a TLS session and tunneling DNS queries.

The solution is to ensure appropriate next-generation firewalls (NGFW) are deployed and configured to use application awareness or protocol enforcement features with a zero-trust approach to Internet connectivity. Block DNS (UDP/53 and TCP/53) and DoT (TCP/853) early in the firewall ruleset and only permit approved DNS name servers to access the Internet for DNS.



DoH uses HTTP or HTTP/2 protocols to encrypt DNS traffic. Unlike DoT, an application running on TCP/853, you cannot distinguish DNS traffic from HTTP traffic when encrypted using HTTPS. To make life even more complicated, many web browsers now enable this functionality by default. No DNS solution is available to tackle this issue directly and effectively.

Most enterprises already have a dedicated firewall policy for HTTP and HTTPS, with perhaps some antivirus scanning and URL inspection. So, why not add some logic to delve deeper into the application running over the web (TCP/80 and TCP/443) protocols? Many firewalls have built-in application filters or profiles for identifying DoH traffic that must be enabled and added to the appropriate web browsing firewall rules.


NGFW for Both DoT and DoH Control

Since many edge firewalls tend to be more liberal in outbound communications, it is imperative to have explicit rules to block TCP/853 for DoT in addition to UDP/53 and TCP/53 for all clients except sanctioned DNS name servers. This way, DNS traffic can be inspected and controlled for compliance, regulatory, or security threats at a singular DNS choke point. Application awareness profiles should be used to prevent tunneling and ensure that, for instance, port TCP/8080 is used for proxying and not for tunneling DNS or SSH traffic. Without such application-based inspection, a benign firewall rule may still allow outbound private email or instant messenger communication (your author may have known about such capabilities with certain proxy manufacturers back in the day).

In the case of DNS over HTTP, application awareness profiles should be applied to block DNS traffic when detected over TCP/80 or TCP/443. TCP/443 should be decrypted with financial and medical exceptions in place using web or URL filtering. Taking these NGFW configuration steps is how enterprises prevent DNS unfiltered exfiltration, which leads to data loss, ransomware command and control, redirection to malicious sites, and other cybersecurity threats. If we do not take this approach, DNS exfiltration becomes a game of whack-a-mole as we try to inspect every new DNS standard that comes around the corner purporting enhanced user privacy at the cost of enterprise security.


EDNSO Client Subnet

Finally, let’s discuss the ECS DNS extension. ECS was initially proposed in 2011 by Content Delivery Network (CDN) operators to allow DNS recursive servers to pass along a portion of the client IP address. Public DNS providers are generally very distributed in fashion. Depending on what server handles a DNS request, the local domain name server (LDNS) may be in a different region, as seen by the final authoritative DNS server.

Internet-based authoritative DNS servers have relied on Google, OpenDNS, and other public DNS provider source IP addresses to determine the geolocation DNS response. Most enterprises do not forward their unresolved DNS requests directly to the Internet root servers but to public DNS providers. Using ECS, a portion of the client IP address, generally the first three octets (i.e., 30.0.0.x), is encoded within the DNS request options field. In this way, authoritative DNS name servers obtain the externally routable IP address subnet of the DNS client to steer clients to locally geolocated resources. This is advantageous as application architectures continue adopting globally distributed DNS systems, varied compute resources in different regions, CDNs, etc. However, ECS may not be desirable when an enterprise wants to hide the origin subnet of its DNS requests. Instead, they could forward their DNS requests to a public DNS provider of choice or even a contracted Internet DNS service with ECS disabled within their recursive DNS resolver configurations.

Considering ECS options regarding your local recursive DNS name server and any external public recursive DNS servers is critical. Enterprise DNS name servers performing recursion to the Internet can support the enabling or disabling of ECS. Of course, you also need to ensure that if using public DNS providers for recursion, they support (or don’t support) ECS based on your preference. ECS may be desired to provide a better cloud services experience for users at the expense of potential tracking.


Security Over Privacy?

If you value security above privacy in the enterprise environment, you’ll want to ensure that backdoor communications of DNS are stopped. Some solutions can be deployed and configured using a combination of NGFWs and enterprise DNS products. Nexum is ready to discuss your DNS needs and expectations while providing secure and manageable solutions. Our expert engineers can review your DNS architecture and security controls to give you a better understanding of the DNS communications leaving your enterprise.

Check Out More Resources

Nexum Resources

Enterprise Logging Best Practices

Each quarter, the managed security team at Nexum shares insights from our first*defense SNOCC. In this post, we decided to share some general logging best practices that are likely to benefit every organization.

Read More »