DNS Zone Explorer
Mix.install([
{:jason, "~> 1.4"},
{:domainname, "~> 0.1.5"},
{:whois, "~> 0.2.1"},
{:kino, "~> 0.14.1"},
# {:kino, "~> 0.14.1", github: "livebook-dev/kino"}
])
Overview
External Dependencies
A few external programs are used by DNS Explorer, in discovering and teasing out subdomains. The following tools have all been used in discovery at some point:
-
OWASP Amass, In-depth attack surface mapping and asset discovery, with environment variable
LB_AMASS_BIN
pointing to the binary’s location -
assetfinder
, Find domains and subdomains related to a given domain, with environment variableLB_ASSETFINDER_BIN
-
DNSRecon, DNS Enumeration Script, with environment variable
LB_DNSRECON_BIN
-
Domain Security Scanner, Scan domains and receive advice based on their BIMI, DKIM, DMARC, and SPF records, with environment variable
LB_DSS_BIN
-
puredns
, Puredns is a fast domain resolver and subdomain bruteforcing tool that can accurately filter out wildcard subdomains and DNS poisoned entries, with environment variableLB_PUREDNS_BIN
-
subfinder
, Fast passive subdomain enumeration tool, with environment variableLB_SUBFINDER_BIN
-
Sublist3r, Fast subdomains enumeration tool for penetration testers, with environment variable
LB_SUBLIST3R_BIN
At present, however, the tool actively uses only the Domain Security Scanner (dss
) and Subfinder tools. Both are easily installed from their respective GitHub repositories. Once installed, set the DSS_BIN
and SUBFINDER_BIN
‘secrets’ in this Livebook, prior to running a domain scan.
Domain entry and parsing
To begin, we need user input specifying which domain to interrogate, storing the input into the domain
variable.
DNS Zone Explorer relies on the Kino
library to provide user interaction. Kino.Shorts
simplifies calling provided functions, so is is imported into the Livebook process up front.
import Kino.Shorts
As well as reading user input, Kino will at this point interrupt any further execution if the domain name input is left blank. Not doing this allows Livebook to try to evaluate (all) subsequent sections reliant on this input, resuling in Elixir throwing an error message. Delegating this right away to Kino is slightly more user friendly.
domain = read_text("Enter your domain name", default: "")
if domain == "", do: Kino.interrupt!(:error, "You must provide a domain name")
The read_user_input/1
function takes a domain_input
argument—the user input field constructed earlier can be passed in—immediately using the DomainName
library to construct a domain name structure, if the input conforms to the hostname RFC specification, throwing an error otherwise.
defmodule Dns.ZoneExplorer.UI do
def read_user_input(domain_input) do
DomainName.new!(domain_input, must_be_hostname: true)
end
end
Defining special variables
Email authentication is a collection of techniques aimed at providing verifiable information about the origin of email messages. Three authentication methods have been widely adopted: DKIM, SPF and DMARC. DKIM and DMARC are specified as DNS records, using specific subdomain labels, which we define in this section.
BIMI is a branding specification, and as it is bound to be more widely used, we will specify its own subdomain label requirement as well.
dkim_subdomain_label = "_domainkey"
dmarc_subdomain_label = "_dmarc"
bimi_subdomain_label = "_bimi"
Let’s define a set of DKIM selectors in dkim_selector_list
, as a valid selector is needed to query for a DKIM key.
Lists of known DKIM selectors, mined from a range of locations: articles and forum discussions on the internet:
- Email Provider Commonly Used DKIM Selectors
- List of most common DKIM selectors? (Reddit)
- The Handbook of Email Service Providers
known_dkim_selector_list = [
# Generic
"default", "dkim", "email", "mail", "x",
# Google
"google",
# Microsoft
"selector1", "selector2",
# ACOUSTIC DKIM SELECTOR:
"spop1024",
# ActiveCampaign
"dk",
# Avoccado.ai
"pic",
# Aruba.it
"a1",
# AWEBER DKIM SELECTOR:
"aweber_key_a", "aweber_key_b", "aweber_key_c",
# Blackbaud, eTapestry
"sm",
# CAMPAIGN MONITOR DKIM SELECTOR:
"cm",
# ConstantContact
"ctct1", "ctct2",
# CONTACTLAB DKIM SELECTOR:
"clab1",
# DOTDIGITAL DKIM SELECTOR:
"dkim1024",
# EMARSYS DKIM SELECTOR:
"key1", "key2",
# EMMA DKIM SELECTOR:
"e2ma-k1", "e2ma-k2", "e2ma-k3",
# Everlytic
"everlytickey1", "everlytickey2", "eversrv",
# Global Micro
"mxvault",
# GODADDY/MAD MIMI DKIM SELECTOR:
"sable",
# Hetzner
"dkim",
# HUBSPOT DKIM SELECTOR:
"hs1", "hs2",
# iCloud
"sig1",
# KLAVIYO DKIM SELECTOR:
"kl", "kl2",
# LISTRAK DKIM SELECTOR:
"key1",
# MailChimp/Mandrill
"k1", "k2", "mandrill",
# Mailerlite
"litesrv",
# MAILGUN DKIM SELECTOR:
"k1",
# MailJet
"mailjet",
# MAILPOET DKIM SELECTOR:
"mailpoet1", "mailpoet2",
# MAILUP DKIM SELECTOR:
"m101", "m102",
# MAPP DIGITAL DKIM SELECTOR:
"ecm1",
# MxVault
"mxvault",
# Nationbuilder, Sharpspring
"s1", "s2",
# NETCORE DKIM SELECTOR:
"nce2048",
# OMNISEND DKIM SELECTOR:
"smtp",
# Protonmail
"protonmail",
# ZenDesk
"zendesk1", "zendesk2"
]
further_dkim_selector_list =
[
k%N1,20%
class
s%L384,512,768,1024,2048%
m%L384,512,768,1024,2048%
smtpapi
dkim
bfi
spop
spop1024
beta
domk
key%N1,20%
dk
ei
yesmail%N1,20%
smtpout
sm
selector%N1,20%
authsmtp
alpha
v%N1,5%
mesmtp
cm
prod
pm
gamma
dkrnt
dkimrnt
private
gmmailerd
pmta
m%N1,20%
x
selector
qcdkim
postfix
mikd
main
m
dk20050327
delta
yibm
wesmail
test
stigmate
squaremail
sitemail
sel%N1,20%
sasl
sailthru
rsa%N1,20%
responsys
publickey
proddkim
my%N1,20%
mail-in
ls%N1,20%
key
ED-DKIM
ebmailerd
eb%N1,20%
dk%N1,20%
Corporate
care
0xdeadbeef
yousendit
www
tilprivate
testdk
snowcrash
smtpcomcustomers
smtpauth
smtp
sl%N1,20%
sl
sharedpool
ses
server
scooby
scarlet
safe
s
s%N1,20%
pvt
primus
primary
postfix.private
outbound
originating
one
neomailout
mx
msa
monkey
mkt
mimi
mdaemon
mailrelay
mailjet
mail-dkim
mailo
mandrill
lists
iweb
iport
id
hubris
googleapps
global
gears
exim4u
exim
et
dyn
duh
dksel
dkimmail
corp
centralsmtp
ca
bfi
auth
allselector
zendesk1
; search rules
; uncreative
dk%N01,20%
dk%N1,9%
dkim%N01,20%
dkim%N1,9%
dkim
proddkim
testdkim
%Ldkim,dk,testdkim,proddkim%%L256,384,512,768,1024,2048%
]
Let’s get locations of all subdomain enumeration tools we may need, into variables we can use.
# amass_binary = System.get_env("LB_AMASS_BIN")
# assetfinder_binary = System.get_env("LB_ASSETFINDER_BIN")
# dnsrecon_binary = System.get_env("LB_DNSRECON_BIN")
# puredns_binary = System.get_env("LB_PUREDNS_BIN")
# sublist3r_binary = System.get_env("LB_SUBLIST3R_BIN")
# dss_binary = System.get_env("LB_DSS_BIN")
Querying domain name information (WHOIS query)
defmodule DomainName.Querying do
def domain_name_query(domain_name) do
domain_name
|> DomainName.name()
|> Whois.lookup()
end
def get_critical_domain_info(domain_name) do
case domain_name_query(domain_name) do
{:ok, whois_record} -> whois_record
{_, _} -> {:error}
end
end
end
Using Erlang’s :inet_res builtin functionality
defmodule Dns.Query do
@doc """
Runs Erlang's `:inet_res:resolve/2` resource query on the provided domain.
## Returns
The resource query returns the following response:
```
%{
data: [...],
type: :...,
ttl: 9310,
domain: ~c"...",
func: false,
class: :in,
answer_source: "...",
label: "...",
query_tool: "inet_res"
}
```
"""
def get_record_info(dns_name, type, options \\ []) when is_atom(type) do
answer_source =
Keyword.get(options, :answer_source, "#{String.upcase(Kernel.to_string(type))} query")
label =
Keyword.get(options, :label, "")
dns_name
|> DomainName.name()
|> String.to_charlist()
|> :inet_res.resolve(:in, type)
|> case do
{:ok, res} ->
{:ok,
res
|> :inet_dns.msg()
|> Keyword.fetch!(:anlist)
|> Enum.map(fn i -> Enum.into(:inet_dns.rr(i), %{}) end)
|> Enum.map(fn i ->
Map.merge(i, %{
query_tool: "inet_res",
answer_source: answer_source,
label: label
})
end)}
{:error, reason} ->
{:error, reason}
end
end
def construct_subdomain_host(fqdn, subdomain_label),
do: DomainName.join!(subdomain_label, fqdn)
def filter_for_matching_type(response, type) do
case response do
{:ok, msg} ->
msg
|> Enum.reverse()
|> case do
[%{type: ^type} | _tail] -> msg
_ -> []
end
{:error, _} ->
[]
end
end
def resolves_to_already_seen?(record_list, domains_seen)
when is_list(record_list) and is_list(domains_seen) do
Enum.map(record_list, fn rec ->
Enum.map(domains_seen, fn domain ->
rec.data
|> DomainName.new!()
|> DomainName.equal?(domain)
end)
end)
|> List.flatten()
|> Enum.any?(fn i -> i end)
end
def get_a_records(fqdn) do
get_record_info(fqdn, :a) |> filter_for_matching_type(:a)
end
def get_aaaa_records(fqdn) do
get_record_info(fqdn, :aaaa) |> filter_for_matching_type(:aaaa)
end
def get_cname_records(fqdn) do
get_record_info(fqdn, :cname) |> filter_for_matching_type(:cname)
end
def get_ns_records(fqdn) do
get_record_info(fqdn, :ns) |> filter_for_matching_type(:ns)
end
def get_mx_records(fqdn) do
get_record_info(fqdn, :mx) |> filter_for_matching_type(:mx)
end
def get_ptr_records(fqdn) do
get_record_info(fqdn, :ptr) |> filter_for_matching_type(:ptr)
end
def get_soa_records(fqdn) do
get_record_info(fqdn, :soa) |> filter_for_matching_type(:soa)
end
def get_txt_records(fqdn) do
get_record_info(fqdn, :txt) |> filter_for_matching_type(:txt)
end
def get_dmarc_record(fqdn, dmarc_subdomain_label) do
construct_subdomain_host(fqdn, dmarc_subdomain_label)
|> get_record_info(:txt, answer_source: "DMARC query", label: dmarc_subdomain_label)
end
def get_dkim_records(fqdn, dkim_subdomain_label, selector_list)
when is_list(selector_list) do
dkim_subdomain = construct_subdomain_host(fqdn, dkim_subdomain_label)
Enum.map(selector_list, fn sel ->
construct_subdomain_host(dkim_subdomain, sel)
|> get_record_info(:txt, answer_source: "DKIM query", label: sel)
|> case do
{:ok, payload} -> payload
{:error, reason} -> {:error, reason}
end
end)
|> Enum.filter(fn
{:error, _} -> false
_ -> true
end)
end
def get_caa_records(fqdn) do
get_record_info(fqdn, :caa) |> filter_for_matching_type(:caa)
end
def get_srv_records(fqdn) do
get_record_info(fqdn, :srv) |> filter_for_matching_type(:srv)
end
def get_spf_records(fqdn) do
get_record_info(fqdn, :spf) |> filter_for_matching_type(:spf)
end
end
As any record query may return a CNAME
—or a _canonical name_—response, which the client will immediately query all the way until it encounters a non-CNAME response, we will receive a response list. This possibility needs to be checked, as it means getting multiple answer records, and this needs to be handled appropriately.
Two ways present themselves, the quick one being to discard all but the last record in the list, as this is likely to be the final one, and the one we actually want. Doing this, however, means eliding potentially useful information, at least for debugging, as it would not be clear how we came to the final record, without seeing the alias traversals within. In cases of DNS cache poisoning, or even Man-in-the-Middle (MITM) attacks, discovering unexpected CNAME
records is critical in helping spot vulnerabilities, so they should be kept and reported on for a full picture of the state of the zone. Showing the full redirection picture aids in upholding the principle of least astonishment, in that it shouldn’t cause surprise or consternation to systems administrators, or those developers who are not aware of every infrastructure decision taken to date.
On DNS zone enumeration
For security reasons, it is difficult to get a true state of a DNS zone, as this is information which would be used by attackers in finding exploits.
To enumerate subdomains of a domain, we will turn to several penetration testing tools.
Subdomain enumeration
We now call out to System.cmd
to run subfinder
on our domain. Note that ease of parsing is most important here, thus the tool is run with a few additional options:
-
-silent
so that the masthead message, progress and other messages are not printed tostdout
-
-json
to return a list of JSON-formatted results, for each enumerated subdomain -
-active
to display active subdomains only
PLEASE NOTE This command is run on the network, querying multiple internet sources, and as such is likely to take some time, typically needing 30 seconds or more. Please be patient and don’t interrupt it unnecessarily. For additional informative use, stderr
has been redirected to stdout
to show error messages, if any.
defmodule SubdomainDiscovery do
def scan_for_subdomains(domain) do
subfinder_binary = System.get_env("LB_SUBFINDER_BIN")
System.cmd(
"bash",
[
"-lc",
"#{subfinder_binary} -d #{Dns.ZoneExplorer.UI.read_user_input(domain)} -silent -json -active"
],
stderr_to_stdout: true
)
end
end
{subfinder_query_result, 0} = SubdomainDiscovery.scan_for_subdomains(domain)
subfinder_query_result =
subfinder_query_result
|> String.split("\n")
|> Enum.filter(fn rec -> rec != "" end)
subfinder_query_result_length = length(subfinder_query_result)
subfinder_query_result =
cond do
subfinder_query_result_length > 1 ->
[_head | tail] = subfinder_query_result
tail
true ->
[]
end
With query results saved to subfinder_query_results
, we now need to clean up the results by splitting away newline characters (\n
), filtering out empty strings, and parsing the JSON results.
enumerated_subdomains =
case subfinder_query_result do
[] ->
[]
_ ->
subfinder_query_result
|> Enum.map(fn rec -> Jason.decode!(rec) end)
|> Enum.map(fn m ->
m
|> Map.new(fn {k, v} -> {String.to_atom(k), v} end)
|> Map.merge(%{tool: "subfinder"})
end)
|> Enum.filter(fn %{host: host} ->
host != DomainName.name(Dns.ZoneExplorer.UI.read_user_input(domain))
end)
end
Core functionality
defmodule ZoneRootDomain do
defstruct name: "",
display_name: "",
depth: 0,
registrar: "",
nameservers: [],
created_at: nil,
updated_at: nil,
expires_at: nil,
queried_at: NaiveDateTime.utc_now(),
query_tool: "",
zone_tree: []
end
defmodule DnsRecord do
defstruct id: nil,
label: nil,
name: nil,
display_name: "",
inside_zone: false,
depth: nil,
weight: 0,
type: nil,
class: :in,
specialised_type: nil,
data: "",
structured_data: %{},
ttl: nil,
query_tool: "",
answer_source: "",
child_records: []
def construct_dns_record(dns_record) when is_map(dns_record) do
%DnsRecord{
id: Map.get(dns_record, :id, nil),
label: Map.get(dns_record, :label, nil),
name: Map.get(dns_record, :name, nil),
display_name: Map.get(dns_record, :display_name, ""),
inside_zone: dns_record.inside_zone,
depth: Map.get(dns_record, :depth, nil),
weight: Map.get(dns_record, :weight, 0),
type: dns_record.type,
class: Map.get(dns_record, :class, :in),
specialised_type: Map.get(dns_record, :specialised_type, nil),
data: Map.get(dns_record, :data),
structured_data: Map.get(dns_record, :structured_data, %{}),
ttl: dns_record.ttl,
child_records: Map.get(dns_record, :child_records, []),
query_tool: Map.get(dns_record, :query_tool, ""),
answer_source: Map.get(dns_record, :answer_source, "")
}
end
end
defmodule Dns.Core do
def get_host_name_without_domain(fqdn, zone_root_domain)
when is_struct(fqdn, DomainName) and is_struct(zone_root_domain, DomainName) do
fqdn
|> DomainName.without_suffix(zone_root_domain)
|> case do
{:ok, name} -> name
{:error, _} -> fqdn
end
end
def is_subdomain_of_zone?(fqdn, zone_root_domain) when is_struct(fqdn, DomainName) do
DomainName.ends_with?(fqdn, zone_root_domain)
end
def is_subdomain_of_zone?(fqdn, zone_root_domain) when is_bitstring(fqdn) do
fqdn
|> DomainName.new!()
|> DomainName.ends_with?(zone_root_domain)
end
def get_subdomain_labels(subdomain) do
DomainName.labels(subdomain) |> Enum.reverse()
end
def classify_txt_record_type(record, options \\ []) do
data_string =
cond do
is_list(record.data) -> Enum.join(record.data, " || ")
true -> record.data
end
cond do
is_record_spf?(data_string) -> :spf
is_record_payload_dmarc?(data_string) -> :dmarc
is_record_payload_dkim?(data_string) -> :dkim
true -> nil
end
end
defp is_record_spf?(data_string) when is_bitstring(data_string) do
spf_re = ~r/^v=spf1.*/
String.match?(data_string, spf_re)
end
defp is_record_dmarc?(data_string, fqdn, options \\ [])
when is_bitstring(data_string) and is_struct(fqdn, DomainName) do
case is_record_payload_dmarc?(data_string) do
true -> is_record_domain_dmarc?(fqdn)
false -> nil
end
end
def is_record_payload_dmarc?(data_string) when is_bitstring(data_string) do
String.match?(data_string, ~r/^v=DMARC1.*/)
end
def is_record_domain_dmarc?(fqdn) when is_struct(fqdn, DomainName) do
fqdn
|> DomainName.labels()
|> Enum.find(fn l -> l == "_dmarc" end)
|> case do
nil -> nil
_ -> :dmarc
end
end
defp is_record_dkim?(data_string, fqdn, options \\ [])
when is_bitstring(data_string) and is_struct(fqdn, DomainName) do
case is_record_payload_dkim?(data_string) do
true -> is_record_domain_dkim?(fqdn)
false -> nil
end
end
def is_record_payload_dkim?(data_string) when is_bitstring(data_string) do
String.match?(data_string, ~r/.*p=.*/)
end
def is_record_domain_dkim?(fqdn) when is_struct(fqdn, DomainName) do
fqdn
|> DomainName.labels()
|> Enum.find(fn l -> l == "_domainkey" end)
|> case do
nil -> nil
_ -> :dkim
end
end
def classify_record_chain_type(record_chain) when is_list(record_chain) do
case record_chain do
[head_link | []] ->
case Dns.Core.classify_txt_record_type(head_link) do
:dkim -> Map.merge(head_link, %{specialised_type: :dkim})
:dmarc -> Map.merge(head_link, %{specialised_type: :dmarc})
:spf -> Map.merge(head_link, %{specialised_type: :spf})
_ -> head_link
end
[head_link | tail_links] ->
cond do
Dns.Core.is_record_domain_dkim?(head_link.name) &&
Dns.Core.is_record_payload_dkim?(List.last(tail_links).data) ->
Enum.map(record_chain, fn link ->
Map.merge(link, %{specialised_type: :dkim, weight: 30})
end)
Dns.Core.is_record_domain_dmarc?(head_link.name) &&
Dns.Core.is_record_payload_dmarc?(List.last(tail_links).data) ->
Enum.map(record_chain, fn link ->
Map.merge(link, %{specialised_type: :dmarc, weight: 30})
end)
true ->
record_chain
end
end
end
def handle_query_result_chain(record_chain, options \\ [])
when is_list(record_chain) do
record_chain
|> Enum.map(fn rec -> construct_dns_record_dispatcher(rec, options) end)
|> classify_record_chain_type()
|> convert_record_chain_to_tree()
end
def construct_dns_record_dispatcher(record, options \\ [])
when is_map(record) do
record
|> case do
%{type: :soa} = record ->
construct_soa_record(record, options)
%{type: :ns} = record ->
construct_ns_record(record, options)
%{type: :mx} = record ->
construct_mx_record(record, options)
%{type: :a} = record ->
construct_a_record(record, options)
%{type: :aaaa} = record ->
construct_a_record(record, options)
%{type: :cname} = record ->
construct_cname_record(record, options)
%{type: :txt} = record ->
construct_txt_record(record, options)
%{type: :spf} = record ->
construct_spf_record(record, options)
%{type: :caa} = record ->
construct_caa_record(record, options)
%{type: :srv} = record ->
construct_srv_record(record, options)
_ ->
[]
end
end
def index_record_dispatcher(record, index) do
record
|> Map.get(:type)
|> case do
:ns -> index_ns_record(record, index)
:mx -> index_mx_record(record, index)
:a -> index_a_record(record, index)
:aaaa -> index_a_record(record, index)
:caa -> index_caa_record(record, index)
_ -> record
end
end
def convert_record_chain_to_tree(record_chain)
when is_struct(record_chain, DnsRecord) do
record_chain
end
def convert_record_chain_to_tree(record_chain) when is_list(record_chain) do
case record_chain do
[
%DnsRecord{name: _name_first, data: _data_first} = first_record,
%DnsRecord{name: _name_second} = second_record | _tail
] ->
Dns.ZoneTree.RecordChains.resolve_chain(record_chain, [])
[%DnsRecord{name: _name} = record | []] ->
record
[[]] ->
[]
[] ->
[]
end
end
@doc """
Expects to receive a map, shaped as in the following example:
%{data: {~c"ns-586.awsdns-09.net", ~c"awsdns-hostmaster.amazon.com", 1, 7200,
900, 1209600, 86400},
id: 2, type: :soa, ttl: 855, domain: ~c"domain.org",
class: :in, query_tool: "...", answer_source: "..."}
"""
defp construct_soa_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
data = Dns.Core.handle_soa_record_data(record.data)
DnsRecord.construct_dns_record(%{
id: "SOA-#{record.domain}",
type: record.type,
weight: 0,
name: fqdn,
display_name: "#{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data:
"Master name: #{data.mname}, Responsible contact: #{data.rname}, Serial number: #{data.serial}, Refresh time: #{data.refresh}, Retry time: #{data.retry}, Expire time: #{data.expire}, Minimum TTL: #{data.minimum}",
structured_data: data,
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
@doc """
Expects to receive a map, shaped as in the following example:
%{data: ~c"dns4.p04.nsone.net", id: 4, type: :ns, ttl: 3600,
domain: ~c"domain.com", class: :in, query_tool: "...",
answer_source: "..."}
"""
def construct_ns_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
DnsRecord.construct_dns_record(%{
id: "NS-#{record.data}",
type: record.type,
weight: 5,
name: fqdn,
display_name: "NS #{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: "#{record.data}",
structured_data: %{fqdn: fqdn},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
def index_ns_record(record, index) when is_struct(record, DnsRecord) and is_number(index) do
Map.merge(record, %{
id: Enum.join(["NS", index, record.name], "-"),
display_name: "NS #{index}"
})
end
@doc """
Expects to receive a map, shaped as in the following example:
%{data: {0, ~c"mx.mail.com"}, id: 1, type: :mx, ttl: 3600,
domain: ~c"domain.com", class: :in,
query_tool: "...", answer_source: "..."}
"""
def construct_mx_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
parsed_data = Dns.Core.handle_mx_record_data(record.data)
DnsRecord.construct_dns_record(%{
id: "MX-#{parsed_data.data}",
type: record.type,
weight: 20,
name: fqdn,
display_name: "MX #{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: to_string(parsed_data.data),
structured_data: %{priority: parsed_data.priority, fqdn: DomainName.new!(parsed_data.data)},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
def index_mx_record(record, index) when is_struct(record, DnsRecord) and is_number(index) do
Map.merge(record, %{
id: Enum.join(["MX", index, record.name], "-"),
display_name: Enum.join(["MX", index, record.name], " ")
})
end
@doc """
Expects to receive a map, shaped as in the following AAAA query example:
%{data: {9734, 18176, 16, 0, 0, 0, 26644, 63309}, id: 5, type: :aaaa,
ttl: 300, domain: ~c"subdomain.domain.com", class: :in,
query_tool: "...", answer_source: "..."}
where the only difference for an A query is the shape of the `data` tuple:
data: {104, 20, 246, 77}
"""
def construct_a_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
data =
case record.type do
:a -> handle_a_record_data(record.data)
:aaaa -> handle_aaaa_record_data(record.data)
_ -> nil
end
id = Enum.join([String.upcase(to_string(record.type)), fqdn], "-")
DnsRecord.construct_dns_record(%{
id: id,
type: record.type,
weight: 10,
name: fqdn,
display_name: "#{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: data,
structured_data: %{raw_data: record.data},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
def index_a_record(record, index) when is_struct(record, DnsRecord) and is_number(index) do
Map.merge(record, %{
id: Enum.join([String.upcase(to_string(record.type)), index, record.name], "-")
})
end
def construct_subdomain_record(record, options \\ []) do
children = Keyword.get(options, :children, [])
DnsRecord.construct_dns_record(%{
id: "SUBDOMAIN-#{record.name}",
type: :virtual,
specialised_type: :subdomain,
weight: 12,
name: record.name,
display_name: "#{record.name}",
inside_zone: is_subdomain_of_zone?(record.name, options[:zone_root_domain]),
depth: Keyword.get(options, :depth, 0),
ttl: nil,
data: "",
structured_data: %{},
child_records: children,
query_tool: record.tool,
answer_source: record.source
})
end
@doc """
Expects to receive a map, shaped as in the following example:
%{data: ~c"domain.org", id: 1, type: :cname, ttl: 2322,
domain: ~c"subdomain.domain.com", class: :in,
query_tool: "...", answer_source: "..."
}
"""
defp construct_cname_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
children = Keyword.get(options, :children, [])
DnsRecord.construct_dns_record(%{
id: "CNAME-#{record.domain}",
type: record.type,
weight: 18,
name: fqdn,
display_name: "#{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: handle_record_data(record.data),
structured_data: %{fqdn: DomainName.new!(record.data)},
child_records: children,
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
@doc """
Expects to receive a map, shaped as in the following example:
%{
data: [~c"google-site-verification=..."], type: :txt, ttl: 900,
domain: ~c"domain.org", class: :in,
query_tool: "...", answer_source: "..."}
"""
defp construct_txt_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
label = Map.get(record, :label, "")
id_prefix = Keyword.get(options, :id_prefix, "")
specialised_type =
Keyword.get(
options,
:id_prefix,
Map.get(record, :specialised_type, classify_txt_record_type(record, options))
)
id_construction =
["TXT", id_prefix, specialised_type, record.domain]
|> Enum.reject(fn i -> i == "" or is_nil(i) end)
|> Enum.join("-")
DnsRecord.construct_dns_record(%{
id: id_construction,
type: record.type,
specialised_type: specialised_type,
weight: 30,
name: fqdn,
display_name: record.domain,
label: label,
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: Enum.join(record.data, " "),
structured_data: %{raw_data: record.data},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
defp construct_spf_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
DnsRecord.construct_dns_record(%{
id: "SPF-legacy-#{record.domain}",
type: record.type,
specialised_type: :spf,
weight: 35,
name: fqdn,
display_name: "[LEGACY SPF] #{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: Enum.join(record.data, " || "),
structured_data: %{raw_data: record.data},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
@doc """
Expects to receive a map, shaped as in the following example:
%{
data: {0, ~c"issuewild", ~c"digicert.com"}, type: :caa, ttl: 83,
domain: ~c"domain.org", class: :in,
query_tool: "...", answer_source: "..."
}
"""
defp construct_caa_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
parsed_data = Dns.Core.handle_caa_record_data(record.data)
DnsRecord.construct_dns_record(%{
id: Enum.join(["CAA", record.domain], "-"),
type: record.type,
weight: 40,
name: fqdn,
display_name: "#{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data: "CAA Flag: #{parsed_data.flag} Tag: #{parsed_data.tag} Value: #{parsed_data.value}",
structured_data: %{flag: parsed_data.flag, tag: parsed_data.tag, value: parsed_data.value},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
def index_caa_record(record, index) when is_struct(record, DnsRecord) and is_number(index) do
Map.merge(record, %{
id: Enum.join(["CAA", index, record.name], "-")
})
end
defp construct_srv_record(record, options \\ []) do
fqdn = DomainName.new!(record.domain)
parsed_data = Dns.Core.handle_srv_record_data(record.data)
DnsRecord.construct_dns_record(%{
id: "SRV-#{record.domain}",
type: record.type,
weight: 45,
name: fqdn,
display_name: "#{record.domain}",
inside_zone: is_subdomain_of_zone?(fqdn, options[:zone_root_domain]),
ttl: record.ttl,
data:
"SRV Priority: #{parsed_data.priority} Weight: #{parsed_data.weight} Port: #{parsed_data.port} Target: #{parsed_data.target}",
structured_data: %{
priority: parsed_data.priority,
weight: parsed_data.weight,
port: parsed_data.port,
target: parsed_data.target
},
query_tool: record.query_tool,
answer_source: record.answer_source
})
end
def handle_record_payload(payload, options \\ []) do
case payload do
{:ok, []} ->
[]
{:ok, payload} ->
handle_query_result_chain(payload, options)
{:error, reason} ->
[]
[ok: []] ->
[]
[ok: payload] ->
handle_query_result_chain(payload, options)
[error: _] ->
[]
[head | _tail] = record_list when is_list(record_list) and is_list(head) ->
Enum.map(record_list, fn item -> handle_record_payload(item, options) end)
[head | _tail] = record_list when is_list(record_list) and is_map(head) ->
handle_query_result_chain(record_list, options)
[] ->
[]
end
end
def handle_soa_record_data({mname, rname, serial, refresh, retry, expire, minimum} = data)
when is_tuple(data) do
%{
mname: mname,
rname: rname,
serial: serial,
refresh: refresh,
retry: retry,
expire: expire,
minimum: minimum
}
end
def handle_mx_record_data({priority, remaining_data} = data) when is_tuple(data) do
%{priority: priority, data: remaining_data}
end
def handle_aaaa_record_data(data) when is_tuple(data) do
data
|> Tuple.to_list()
|> Enum.map(fn int -> Integer.to_string(int, 16) end)
|> Enum.join(".")
end
def handle_a_record_data(data) when is_tuple(data) do
data
|> Tuple.to_list()
|> Enum.join(".")
end
def handle_record_data(data), do: Kernel.to_string(data)
def handle_caa_record_data({flag, tag, value} = data) when is_tuple(data) do
%{flag: flag, tag: tag, value: value}
end
def handle_srv_record_data({priority, weight, port, target} = data) when is_tuple(data) do
%{priority: priority, weight: weight, port: port, target: target}
end
end
Structuring records in a subdomain tree
The domain name is not a standard DNS record (from our point of view), so we will give it a unique structure, holding the most important information we wish to report about it. The DomainName.Information
structure tas care of organising this information for later retrieval.
To make reading and traversal of our zone easier, a DnsRecord
structure will store all the relevant information for each record.
defmodule Dns.ZoneTree do
def define_root_of_tree(parent_domain) do
%Whois.Record{
domain: domain,
nameservers: nameservers,
registrar: registrar,
created_at: created_at,
updated_at: updated_at,
expires_at: expires_at
} =
DomainName.Querying.get_critical_domain_info(parent_domain)
%ZoneRootDomain{
name: DomainName.new!(domain),
display_name: DomainName.name(DomainName.new!(domain)),
registrar: registrar,
created_at: created_at,
updated_at: updated_at,
expires_at: expires_at,
nameservers: nameservers,
query_tool: "whois: Elixir WHOIS client and parser",
zone_tree: []
}
end
def construct_dns_zone_tree(fqdn, options \\ []) do
[
get_soa_zone_record(fqdn, options),
get_ns_zone_records(fqdn, options),
get_a_zone_records(fqdn, options),
get_aaaa_zone_records(fqdn, options),
get_mx_zone_records(fqdn, options),
get_txt_zone_records(fqdn, options),
get_dmarc_record(fqdn, options),
get_dkim_records(fqdn, options),
get_spf_zone_records(fqdn, options),
get_caa_zone_records(fqdn, options),
get_srv_zone_records(fqdn, options)
]
|> Enum.reject(fn sublist -> sublist == [] end)
end
def get_soa_zone_record(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_soa_records()
|> Dns.Core.handle_record_payload(options)
end
def get_ns_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_ns_records()
|> Dns.Core.handle_record_payload(options)
end
def get_mx_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_mx_records()
|> Dns.Core.handle_record_payload(options)
end
def get_a_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_a_records()
|> Dns.Core.handle_record_payload(options)
end
def get_aaaa_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_aaaa_records()
|> Dns.Core.handle_record_payload(options)
end
def get_cname_zone_records(subdomain_list, options \\ []) when is_list(subdomain_list) do
subdomain_list
|> Enum.map(fn %{host: host} = rec ->
Map.merge(rec, %{
domain: host,
type: :subdomain,
specialised_type: nil,
weight: 15,
host_labels:
Dns.Core.get_host_name_without_domain(DomainName.new!(host), options[:zone_root_domain])
|> Dns.Core.get_subdomain_labels()
})
end)
|> Enum.map(fn rec ->
construct_cname_tree(rec, rec.host_labels, [], "", options)
end)
|> Enum.reject(fn rec -> rec == [] end)
end
def get_txt_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_txt_records()
|> Dns.Core.handle_record_payload(options)
end
def get_dmarc_record(fqdn, options \\ []) do
options = Keyword.put(options, :query_intent, :dmarc)
fqdn
|> Dns.Query.get_dmarc_record(options[:dmarc_subdomain_label])
|> Dns.Core.handle_record_payload(options)
end
def get_dkim_records(fqdn, options \\ []) do
options = Keyword.put(options, :query_intent, :dkim)
fqdn
|> Dns.Query.get_dkim_records(
options[:dkim_subdomain_label],
options[:known_dkim_selector_list]
)
|> Dns.Core.handle_record_payload(options)
end
def get_spf_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_spf_records()
|> Dns.Core.handle_record_payload(options)
end
def get_caa_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_caa_records()
|> Dns.Core.handle_record_payload(options)
end
def get_srv_zone_records(fqdn, options \\ []) do
fqdn
|> Dns.Query.get_srv_records()
|> Dns.Core.handle_record_payload(options)
end
def construct_cname_tree(_rec, [], [], _acc, _options), do: []
def construct_cname_tree(rec, [head | []], child_records, acc, options) do
host_name = DomainName.join!([head, acc])
fqdn = DomainName.join!(DomainName.name(host_name), options[:zone_root_domain])
Map.merge(rec, %{
host_name: host_name,
name: fqdn,
data: "",
child_records:
Dns.Query.get_cname_records(fqdn)
|> Dns.Query.resolves_to_already_seen?([options[:zone_root_domain]])
|> case do
true ->
[
fqdn
|> Dns.Query.get_cname_records()
|> Dns.Core.handle_record_payload(options)
]
false ->
[
Dns.ZoneTree.construct_dns_zone_tree(fqdn, options),
construct_cname_tree(
rec,
child_records,
[],
DomainName.name(host_name),
options
)
]
|> Enum.reject(fn i -> i == [] end)
|> List.flatten()
end
})
end
def construct_cname_tree(rec, [head | tail], _child_records, acc, options),
do: construct_cname_tree(rec, [head], tail, acc, options)
end
defmodule Dns.ZoneTree.RecordChains do
# The next children list is empty, return current record and end
def resolve_chain(current_node, [] = _subsequent_nodes)
when is_struct(current_node, DnsRecord) do
current_node
end
# Parent and current child domains match
def resolve_chain(
[
%{structured_data: %{fqdn: next_domain}} = current_node,
%{name: next_domain} = next_node | tail
],
[]
) do
resolve_chain(current_node, [next_node | tail])
end
# The current and next child domains match, enter sibling mode
# Since the next two records are siblings, all remaining records are siblings
# Sort by the `data` field and apply an index function, providing them with a unique ID
def resolve_chain([%{name: name}, %{name: name} | _tail] = sibling_nodes, []) do
Enum.map(sibling_nodes, fn node -> node end)
|> List.flatten()
|> Dns.ZoneTree.MergeTrees.sort_dns_records()
|> Enum.with_index(fn rec, index ->
Dns.Core.index_record_dispatcher(rec, index + 1)
end)
end
def resolve_chain(
%{name: name} = current_node,
[%{name: name} = next_node | tail_nodes] = _sibling_nodes
) do
Enum.map([current_node, next_node | tail_nodes], fn node -> node end)
|> List.flatten()
|> Dns.ZoneTree.MergeTrees.sort_dns_records()
|> Enum.with_index(fn rec, index ->
Dns.Core.index_record_dispatcher(rec, index + 1)
end)
end
# Worker, placing the (head of) tail node into the current child's child_nodes
# recursively call on the new next children list
def resolve_chain(current_node, [next_node | subsequent_nodes])
when is_struct(current_node, DnsRecord) and is_struct(next_node, DnsRecord) do
cond do
current_node.type not in [:a, :aaaa, :soa, :txt] ->
current_node
|> Map.merge(%{
child_records:
[
resolve_chain(next_node, subsequent_nodes)
]
|> List.flatten()
})
true ->
current_node
end
end
end
Merging subtrees
As strange as it sounds, the way the Erlang :inet_res.resolve/3
query works is that it iteratively queries a domain for the requested type, returning a chain of results leading up to the final answer. The upstream function, :inet_res.lookup/3
builds on top of resolve/3
, filtering out the answer with the requested Class
and Type
.
This is useful in providing a traceroute
like functionality, showing all the resolution steps leading up to the answer, but has a strange side-effect: it can result in duplicate records. For example, on an FQDN that has both A
and AAAA
records, at the end of a long chain of CNAME
aliases (somewhat common when using external services), every step of the way to the answer will be a duplicate record for the two queries.
It isn’t useful having this duplication in the tree, so the best thing to do with it is to merge all duplicate nodes together, and only finally have the last-step answers fan out at the end of the tree. Given the above example, all the steps in a long CNAME
chain, will be merged, with both sets of answers sharing the same trunk of the tree, ending in multiple leaf nodes for the final A
and AAAA
type record resolutions. This provides us with a cleaner zone tree.
Functionally, the work is simple, taking a list of each subdomain’s (as well as the root domain’s) child trees, applying a sliding window comparing every pair of adjacent tree records for equality, merging them if they match. This is a faster approach than a combinatoric effort to match every tree with every other in the list, as the latter is arguably not needed. Realistically, most record types (encountered to date) will not use CNAME
aliasing. To do so could result in breaking older DNS resolvers, think putting an MX
record behind a CNAME
. It’s not impossible, but unlikely. The expectation is that this will mostly be [ab]used on A
, AAAA
and TXT
records. And, as the tree is built with these queries kept adjacent, the adopted approach is likely to work in all current situations. As with all things, there is much unexpected and exotic use of DNS in the wild, so these assumptions may not hold for long.
Once two records are selected, the structures aren’t compared in depth; only the most important fields—name
, type
, and structured_data
—are compared, as the others are deemed irrelevant (such as TTL
), or unnecessary, such as display_name
strings, favouring structured data. When a match is found, a Map.merge/2
is sufficient to update child_records
, at which point the match is retried recursively, for instances where there is a long list of CNAME
redirections before the answer. Long chains of redirections are now quite common, used by external services to create the illusion of a customer’s subdomain, pointing to an internal service type, pointing to a cloud region, pointing to an individual cloud server instance, all the way to the server registering the record with the answer.
defmodule Dns.ZoneTree.MergeTrees do
def merge_trees(record) when is_struct(record, DnsRecord), do: record
def merge_trees(record_list) when is_list(record_list) do
group_dns_records_by_domain_type_data(record_list)
|> Enum.map(fn
%DnsRecord{} = tree ->
case tree.child_records do
[] ->
tree
_ ->
Map.merge(tree, %{
child_records:
merge_trees(tree.child_records)
|> List.flatten()
|> sort_dns_records()
})
end
[_head | _] = tree_list ->
Enum.map(tree_list, fn tree ->
case tree.child_records do
[] ->
tree
_ ->
Map.merge(tree, %{
child_records:
merge_trees(tree.child_records)
|> List.flatten()
|> sort_dns_records()
})
end
end)
|> sort_dns_records
end)
end
def sort_dns_records(record_list) when is_list(record_list) do
Enum.sort_by(record_list, fn record ->
{record.weight, record.type, record.specialised_type, record.name, record.data}
end)
end
def group_dns_records_by_domain_type_data(record_list) when is_list(record_list) do
record_list
|> Enum.group_by(fn
%DnsRecord{name: name, type: type, structured_data: structured_data} = _record ->
{name, type, structured_data}
%{type: :subdomain} = subdomain ->
{subdomain.name, subdomain.type}
end)
|> Enum.map(fn
{{_name, _type, _structured_data}, grouped_record_list} ->
compare_list_items_in_pairs(grouped_record_list, [])
{{_name, _type}, subdomain_list} ->
Enum.map(subdomain_list, fn subdomain ->
Map.merge(subdomain, %{
child_records:
merge_trees(subdomain.child_records)
|> List.flatten()
})
end)
end)
end
def compare_list_items_in_pairs([] = _tree_list, acc), do: acc
def compare_list_items_in_pairs([%DnsRecord{} = tree1 | []] = _tree_list, []), do: tree1
def compare_list_items_in_pairs([%DnsRecord{} = tree1 | []] = _tree_list, acc),
do: compare_child_trees(acc, tree1)
def compare_list_items_in_pairs(
[%DnsRecord{} = tree1, %DnsRecord{} = tree2 | tail] = _tree_list,
[]
) do
compare_list_items_in_pairs(tail, compare_child_trees(tree1, tree2))
end
def compare_list_items_in_pairs([%DnsRecord{} = tree1 | tail] = _tree_list, acc) do
compare_list_items_in_pairs(tail, compare_child_trees(acc, tree1))
end
def compare_list_items_in_pairs(
[%{type: :subdomain} = tree1 | tail] = _tree_list,
acc
) do
compare_list_items_in_pairs(tail, [tree1 | acc])
end
def compare_child_trees(%DnsRecord{} = tree1, record_list) when is_list(record_list),
do: [tree1 | record_list]
def compare_child_trees(record_list, %DnsRecord{} = tree2) when is_list(record_list),
do: [tree2 | record_list]
def compare_child_trees(
%DnsRecord{
name: name,
type: type,
structured_data: structured_data
} = tree1,
%DnsRecord{
name: name,
type: type,
structured_data: structured_data
} = tree2
) do
merge_records(tree1, tree2)
end
def compare_child_trees(%DnsRecord{} = tree1, %DnsRecord{} = tree2) do
[tree1, tree2]
end
def merge_records(
%DnsRecord{
answer_source: answer1_source,
child_records: child1_records
} = tree1,
%DnsRecord{
answer_source: answer2_source,
child_records: child2_records
}
) do
Map.merge(tree1, %{
answer_source: Enum.join([answer1_source, answer2_source], ", "),
child_records: [child1_records, child2_records] |> List.flatten()
})
end
end
Structuring subdomain records
defmodule Dns.ZoneTree.Subdomains do
def find_subdomains(record_list, options \\ []) when is_list(record_list) do
Enum.map(record_list, fn
%{type: :subdomain} = subdomain ->
process_subdomain(subdomain, options)
%DnsRecord{} = record ->
record
|> Map.put(:depth, options[:depth] + 1)
|> Map.put(
:child_records,
find_subdomains(
record.child_records,
Keyword.put(options, :depth, options[:depth] + 1)
)
)
end)
end
def process_subdomain(subdomain, options \\ []) do
subdomain
|> find_matching_cname_record?()
|> construct_structured_subdomain_record(subdomain, options)
end
def find_matching_cname_record?(subdomain) do
Enum.find_index(subdomain.child_records, fn record ->
record.type == :cname and DomainName.equal?(record.name, subdomain.name)
end)
end
def construct_structured_subdomain_record(matching_record_index, subdomain, options)
when is_integer(matching_record_index) do
{matching_record, remaining_child_records} =
List.pop_at(subdomain.child_records, matching_record_index)
new_depth = options[:depth] + 1
matching_record
|> Map.merge(%{
specialised_type: :subdomain,
depth: new_depth,
child_records:
[remaining_child_records | matching_record.child_records]
|> List.flatten()
|> find_subdomains(Keyword.put(options, :depth, new_depth))
})
end
def construct_structured_subdomain_record(_, subdomain, options),
do:
Dns.Core.construct_subdomain_record(
subdomain,
options
|> Keyword.put(:depth, options[:depth] + 1)
|> Keyword.put(
:children,
find_subdomains(
subdomain.child_records,
Keyword.put(options, :depth, options[:depth] + 1)
)
)
)
end
Constructing the zone tree
defmodule Dns.ZoneTree.Construct do
def construct_zone_tree(subdomain_list, options \\ []) when is_list(subdomain_list) do
starting_depth = Keyword.get(options, :depth, 0)
Dns.ZoneTree.define_root_of_tree(options[:zone_root_domain])
|> Map.merge(%{
depth: starting_depth,
zone_tree:
[
Dns.ZoneTree.construct_dns_zone_tree(options[:zone_root_domain], options),
Dns.ZoneTree.get_cname_zone_records(subdomain_list, options)
]
|> List.flatten()
|> Dns.ZoneTree.MergeTrees.merge_trees()
|> List.flatten()
|> Dns.ZoneTree.MergeTrees.sort_dns_records()
|> Dns.ZoneTree.Subdomains.find_subdomains(Keyword.put(options, :depth, starting_depth))
})
end
end
zone_tree =
Dns.ZoneTree.Construct.construct_zone_tree(enumerated_subdomains,
zone_root_domain: Dns.ZoneExplorer.UI.read_user_input(domain),
parent_domain: Dns.ZoneExplorer.UI.read_user_input(domain),
dmarc_subdomain_label: dmarc_subdomain_label,
dkim_subdomain_label: dkim_subdomain_label,
known_dkim_selector_list: known_dkim_selector_list,
depth: 0
)
defmodule Dns.ZoneTree.Height do
def calculate_tree_height(record_list, height \\ 0) when is_list(record_list) do
Enum.reduce(record_list, height, fn record, acc ->
get_height(record, acc)
end)
end
defp get_height(%DnsRecord{depth: depth, child_records: []}, acc),
do: max(depth, acc)
defp get_height(%DnsRecord{depth: depth, child_records: child_records}, acc) do
calculate_tree_height(child_records, max(depth, acc))
end
end
Traversing the DNS zone tree
defmodule Dns.ZoneTree.RecordList do
def get_flattened_zone_record_list(zone_tree, options \\ %{include_virtual: false})
when is_struct(zone_tree, ZoneRootDomain) do
traverse_zone_tree(zone_tree, options)
|> List.flatten()
|> Dns.ZoneTree.MergeTrees.sort_dns_records()
end
def traverse_zone_tree(tree, options) do
iterate_record_list(tree.zone_tree, [], options)
end
def iterate_record_list([], acc, _options), do: acc
def iterate_record_list(record_list, acc, options) when is_list(record_list) do
record_list
|> Enum.map(fn record ->
handle_record(record, acc, options)
end)
end
def handle_record(%DnsRecord{inside_zone: true, child_records: []} = record, acc, _options),
do: [record | acc]
def handle_record(
%DnsRecord{type: :virtual, inside_zone: true, child_records: [_ | _] = children} =
_record,
acc,
%{include_virtual: false} = options
) do
[acc | iterate_record_list(children, [], options)]
end
def handle_record(
%DnsRecord{inside_zone: true, child_records: [_ | _] = children} = record,
acc,
options
) do
[record |> Map.put(:child_records, []) | [acc | iterate_record_list(children, [], options)]]
end
def handle_record(%DnsRecord{inside_zone: false} = _record, acc, _options), do: acc
end
defmodule Dns.ZoneTraversal do
def traverse_zone_tree(zone_tree, options)
when is_struct(zone_tree, ZoneRootDomain) do
traverse_zone_tree(zone_tree, options)
end
def traverse_zone_tree(tree, options) do
iterate_record_list(tree.zone_tree, [], options)
end
def iterate_record_list([], acc, _options), do: acc
def iterate_record_list(record_list, _acc, _options) when is_list(record_list) do
record_list
|> Enum.map(fn record ->
handle_record(record, [], [])
end)
end
def handle_record(%DnsRecord{label: "", inside_zone: true, child_records: []}, acc, _), do: acc
def handle_record(%DnsRecord{label: nil, inside_zone: true, child_records: []}, acc, _), do: acc
def handle_record(
%DnsRecord{specialised_type: :dkim, label: label, inside_zone: true, child_records: []},
acc,
_
),
do: [label | acc]
def handle_record(
%DnsRecord{specialised_type: _, inside_zone: true, child_records: []},
acc,
_
),
do: acc
def handle_record(
%DnsRecord{type: :virtual, inside_zone: true, child_records: [_ | _] = children} =
_record,
acc,
%{include_virtual: false} = options
) do
[acc | iterate_record_list(children, [], options)]
end
def handle_record(
%DnsRecord{label: "", inside_zone: true, child_records: [_ | _] = children},
acc,
options
) do
[acc | iterate_record_list(children, [], options)]
end
def handle_record(
%DnsRecord{label: nil, inside_zone: true, child_records: [_ | _] = children},
acc,
options
) do
[acc | iterate_record_list(children, [], options)]
end
def handle_record(
%DnsRecord{
specialised_type: :dkim,
label: label,
inside_zone: true,
child_records: [_ | _] = children
},
acc,
options
) do
[[label | acc] | iterate_record_list(children, [], options)]
end
def handle_record(
%DnsRecord{
specialised_type: _,
inside_zone: true,
child_records: [_ | _] = children
},
acc,
options
) do
[acc | iterate_record_list(children, [], options)]
end
def handle_record(%DnsRecord{inside_zone: false} = _record, acc, _options), do: acc
end
dkim_selectors_discovered_in_zone =
zone_tree.zone_tree
|> Dns.ZoneTraversal.iterate_record_list([],[])
|> List.flatten()
|> Enum.dedup()
Get DKIM, DMARC and SPF records
defmodule DomainSecurityScan do
def scan(domain, dkim_selectors) do
dss_binary = System.get_env("LB_DSS_BIN")
System.cmd(
"bash",
[
"-lc",
"#{dss_binary} scan #{DomainName.name(Dns.ZoneExplorer.UI.read_user_input(domain))} --advise --dkimSelector '#{selector_list_to_string(dkim_selectors)}' --checkTLS --format json"
],
stderr_to_stdout: true
)
end
def selector_list_to_string(dkim_selector_list) when is_list(dkim_selector_list) do
dkim_selector_list
|> Enum.join(",")
end
def selector_list_to_string(_dkim_selector_list), do: ""
end
{dss_domain_scan, 0} = DomainSecurityScan.scan(domain, dkim_selectors_discovered_in_zone)
parsed_dss_domain_scan =
dss_domain_scan
|> Jason.decode!()
%{
"advice" => parsed_dss_domain_scan_advice,
"scanResult" => parsed_dss_domain_scan_result
} = parsed_dss_domain_scan
DNS record list
defmodule Dns.Visualisation.CSS do
def styles(_options \\ []) do
"""
:root {
--primary-colour: #FFBE98; /* Pantone 13-1023, Peach Fuzz */
--colour-0: #fff2eb;
--colour-1: #ffe5d6;
--colour-2: #ffd8c2;
--colour-3: #ffcbad;
--colour-4: #ffbe98;
--colour-5: #ffb185;
--colour-6: #ffa570;
--colour-7: #ff985c;
--colour-8: #ff8b47;
--colour-9: #ff7e33;
--colour-10: #ff711f;
--indentation-increment: 1rem;
--record-ttl-width: 8ch;
--record-class-width: 5ch;
--record-type-width: 6ch;
--heading-text-opacity: 0.95;
--out-of-zone-bg-colour: color-mix(in srgb, var(--primary-colour) 7%, white);
}
figure {
margin-left: 0;
margin-right: 0;
}
.figure-caption {
color: rgb(97 117 138 / var(--heading-text-opacity));
margin: 1.5rem 0;
}
.record-list-display {
border: 1px solid color-mix(in srgb, var(--primary-colour) 70%, white 0%);
border-radius: 0.5rem;
box-shadow: 0 0 25px -5px #0000001a, 0 0 10px -5px #0000000a;
padding-left: 2.5rem;
padding-right: 2.5rem;
}
.record-list-display,
.figure-caption {
font-family: Inter, system-ui, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji";
}
.domain-name-display {
color: color-mix(in srgb, var(--primary-colour), black 15%);
letter-spacing: 0.05rem;
font-weight: 500;
}
.record-list-display ul { list-style-type: none; }
.record-list-display__list { border-radius: 0.5rem; }
.record-list-display table {
border-collapse: collapse;
table-layout: auto;
}
.record-list-display--flat-list table thead tr {
border-bottom: 1px solid color-mix(in srgb, var(--primary-colour) 40%, white);
}
.record-list-display--flat-list table tbody tr {
border-bottom: 1px solid color-mix(in srgb, var(--primary-colour) 20%, white);
}
.record-list-display--flat-list table tbody tr:last-child { border-bottom-width: 0; }
.record-list-display table th,
.record-list-display table td {
color: rgb(97 117 138 / var(--heading-text-opacity));
padding: 0.75rem 1rem;
}
.domain-name-infomation__headers { vertical-align: bottom; }
.iso-date { width: 10ch; }
.out-of-zone { background: var(--out-of-zone-bg-colour);}
.record-name {
}
.record-ttl { width: var(--record-ttl-width); } /* 31,536,000 is longest expected length, one year in seconds */
.record-class { width: var(--record-class-width); } /* CHAOS is longest class name, 5 characters */
.record-type { width: var(--record-type-width); } /* UNSPEC is longest type name, 6 characters */
.record-specialisation,
.out-of-zone__label {
color: color-mix(in srgb, var(--primary-colour), black 30%);
font-size: 0.55rem;
font-weight: 500;
text-transform: uppercase;
}
.record-data {
max-width: 50ex;
}
.record-name, .record-data { overflow-wrap: anywhere; }
.record-depth-0, .record-depth-1, .record-depth-2,
.record-depth-3, .record-depth-4, .record-depth-5,
.record-depth-6, .record-depth-7, .record-depth-8,
.record-depth-9, .record-depth-10 { border-top: 1px solid white; }
.record-depth-0 { padding-left: 0; }
.record-depth-1 { border-top: 1px solid var(--colour-1); }
.record-depth-2 { border-top: 1px solid var(--colour-2); }
.record-depth-3 { border-top: 1px dotted var(--colour-3); }
.record-depth-4 { border-top: 1px dotted var(--colour-4); }
.record-depth-5 { border-top: 1px dotted var(--colour-5); }
.record-depth-6 { border-top: 1px dotted var(--colour-6); }
.record-depth-7 { border-top: 1px dotted var(--colour-7); }
.record-depth-8 { border-top: 1px dotted var(--colour-8); }
.record-depth-9 { border-top: 1px dotted var(--colour-9); }
.record-depth-10 { border-top: 1px dotted var(--colour-10); }
"""
end
end
defmodule Dns.Visualisation.ZoneRecordTable do
def construct_html_zone_table(zone_tree) when is_struct(zone_tree, ZoneRootDomain) do
"""
Name
TTL
Class
Type
Value
""" <>
format_record_list(zone_tree.zone_tree) <>
"""
Figure 1. Flat record table of #{zone_tree.display_name} zone records, queried on #{Calendar.strftime(zone_tree.queried_at, "%d") |> String.replace("0", "")} #{Calendar.strftime(zone_tree.queried_at, "%B %Y")}
""" <>
Dns.Visualisation.CSS.styles()
end
def format_record_list(tree) when is_list(tree) do
tree
|> Enum.map(fn %DnsRecord{} = record ->
"""
#{record.name}
#{record.ttl}
#{record.class |> to_string() |> String.upcase()}
#{record.type |> to_string() |> String.upcase()}
#{if record.specialised_type != nil do
"" <> String.upcase(to_string(record.specialised_type)) <> ""
end}
#{record.data}
"""
end)
|> Enum.join("\n")
end
end
html_zone_tree =
Dns.Visualisation.ZoneRecordTable.construct_html_zone_table(zone_tree)
Kino.render(
Kino.Download.new(fn -> html_zone_tree end,
label: "Download DNS zone record table",
filename: "#{domain}-DNS_zone_record_table.html"
)
)
Kino.HTML.new(html_zone_tree)
Displaying a text-based tree of the zone
defmodule Dns.Visualisation.ZoneRecordTree do
@iso_date_format_string "%Y-%m-%d"
@domain_name_length zone_tree.name
|> DomainName.name()
|> String.length()
|> Kernel.*(1.8)
|> ceil()
# @max_depth Dns.ZoneTree.Height.calculate_tree_height(zone_tree.zone_tree)
def construct_html_zone_tree(zone_tree) when is_struct(zone_tree, ZoneRootDomain) do
"""
-
Domain name
Domain registrar
Initial registration date
Last update date
Expiry date
#{zone_tree.display_name}
#{zone_tree.registrar}
#{zone_tree.created_at |> Calendar.strftime(@iso_date_format_string)}
#{zone_tree.updated_at |> Calendar.strftime(@iso_date_format_string)}
#{zone_tree.expires_at |> Calendar.strftime(@iso_date_format_string)}
-
"""
<>
format_record_list(zone_tree.zone_tree) <>
"""
Figure 2. Hierarchical table of #{zone_tree.display_name} zone records, queried on #{Calendar.strftime(zone_tree.queried_at, "%d") |> String.replace("0", "")} #{Calendar.strftime(zone_tree.queried_at, "%B %Y")}
""" <>
Dns.Visualisation.CSS.styles()
end
def format_record_list(tree) when is_list(tree) do
tree
|> Enum.map(fn %DnsRecord{} = record ->
"""
#{record.name}
#{if not record.inside_zone, do: "External record"}
#{record.ttl}
#{record.class |> to_string() |> String.upcase()}
#{record.type |> to_string() |> String.upcase()}
#{if record.specialised_type != nil do
"" <> String.upcase(to_string(record.specialised_type)) <> ""
end}
#{record.data}
"""
<>
create_nested_record_level(record.child_records) <>
"""
"""
end)
|> Enum.join("\n")
end
def create_nested_record_level([]), do: ""
def create_nested_record_level(subtree) when is_list(subtree) do
format_record_list(subtree)
end
end
hierarchical_dns_zone_tree =
Dns.Visualisation.ZoneRecordTree.construct_html_zone_tree(zone_tree)
Kino.render(
Kino.Download.new(fn -> hierarchical_dns_zone_tree end,
label: "Download hierarchical DNS zone record table",
filename: "#{domain}-DNS_zone_record_table_with_hierarchy.html"
)
)
Kino.HTML.new(hierarchical_dns_zone_tree)
Visualising the DNS zone
With the information now gathered, we can create a usable flowchart diagram, presenting the zone visually. We will use Mermaid for this, as it is an easy to use tool for programmatic diagram creation.
To begin with, let’s define invariant sections of the diagram definition: initialisation configuration, diagram type and title, and styling. By doing this up front, we’re developing the diagram definition in easily managable segments, and defining the diagram in a literate programming style.
dns_zone_diagram_configuration =
"""
%%{ init: {
'theme': 'base',
'themeVariables': {
'clusterBkg': '#60C8B3',
'lineColor': '#FFBE98',
'primaryColor': '#6699dd',
'primaryTextColor': '#ddddff',
'textColor': '#ff9999',
'nodeTextColor': '#99ff99'
},
'fontFamily': 'Calluna Sans, Helvetica, Trebuchet MS, Verdana, Arial, Sans-Serif',
'flowchart': {
'curve': 'bumpX',
}
} }%%
"""
diagram_type = "flowchart LR"
In the following step, we insert the list of edges by order of definition. The double-spacing is inserted to prevent Markdown-interpreted single lines, which can cause problems when Mermaid builds the final diagram.
diagram_styles =
"""
classDef parent-zone fill:#eff5fa,color:#222,stroke:#FFA74F,stroke-width:2px;
classDef parent-node fill:#FFBE98FF,stroke:#FFA74F,stroke-width:5px;
classDef soa-record fill:#FFA74F,color:white,stroke:#FFBE98,stroke-width:5px;
classDef ns-record fill:#CE3375,color:white,stroke:#E881A6,stroke-width:5px;
classDef mx-record fill:#E881A6,color:white,stroke:#CE3375,stroke-width:4px;
classDef a-record fill:#279D9F,color:white,stroke:#60C8B3,stroke-width:5px;
classDef aaaa-record fill:#3acdcf,color:white,stroke:#60C8B3,stroke-width:5px;
classDef cname-record fill:#1B5091,color:white,stroke:#6EA1D4,stroke-width:5px;
classDef subdomain-record fill:#6EA1D4,color:white,stroke:#1B5091,stroke-width:3px;
classDef txt-record fill:#60C8B3,color:white,stroke:#279D9F,stroke-width:5px;
classDef spf-record fill:#60C8B3,color:white,stroke:#279D9F,stroke-width:3px;
classDef srv-record fill:#a4e0d4,color:#222,stroke:#279D9F,stroke-width:2px;
classDef caa-record fill:#c2eae2,color:#222,stroke:#279D9F,stroke-width:2px;
linkStyle default fill:none,stroke-width:3px,stroke:#6EA1D4
"""
First, we need a module, ZoneTree
to convert this data into Mermaid flowchart definition code.
defmodule Dns.Visualisation.ZoneTreeDiagram do
@iso_date_format_string "%Y-%m-%d"
def define_parent_node(zone_tree) when is_struct(zone_tree, ZoneRootDomain) do
date_created = zone_tree.created_at |> Calendar.strftime(@iso_date_format_string)
date_updated = zone_tree.updated_at |> Calendar.strftime(@iso_date_format_string)
date_expires = zone_tree.expires_at |> Calendar.strftime(@iso_date_format_string)
"#{zone_tree.display_name}([\"#{DomainName.name(zone_tree.name)}
Created #{date_created}
Updated #{date_updated}
Expiry #{date_expires}\"]):::parent-node;"
end
def define_dns_zone(parent_node_id, record_list)
when is_bitstring(parent_node_id) and is_list(record_list) do
for rec <- record_list do
construct_zone_records(parent_node_id, rec)
end
end
def construct_parent_zone_subgraph(zone_tree) when is_struct(zone_tree, ZoneRootDomain) do
"""
subgraph #{zone_tree.display_name}-zone [#{DomainName.name(zone_tree.name)} zone];
direction LR;
#{zone_tree.display_name};
#{Dns.ZoneTree.RecordList.get_flattened_zone_record_list(zone_tree, %{include_virtual: true}) |> Enum.map(fn record -> record.id <> ";" end)}
end;
#{zone_tree.display_name}-zone:::parent-zone;
"""
end
defp construct_zone_records(parent_node_id, record)
when is_bitstring(parent_node_id) and is_struct(record, DnsRecord) do
"#{parent_node_id} #{define_link_type(record.type)} #{record.id}([\"#{record.display_name}
TTL #{record.ttl}
#{Enum.join([record.type |> to_string() |> String.upcase(), record.specialised_type |> to_string() |> String.upcase()], " ")}
#{if record.type === :mx, do: record.structured_data.priority} #{if (record.specialised_type == :dkim and record.type == :txt) or record.type == :soa, do: String.slice(record.data, 0..25) <> "...", else: record.data}\"])#{attach_style_class(record.type)};
#{if Map.get(record, :child_records, []) != [], do: define_dns_zone(record.id, record.child_records)}
#{construct_circular_reference(parent_node_id, record)}"
end
defp construct_circular_reference(parent_node_id, %DnsRecord{data: parent_node_id} = record) do
"#{record.id} #{define_link_type(record.type)} #{parent_node_id};"
end
defp construct_circular_reference(_parent_node_id, %DnsRecord{} = _record), do: ""
def define_link_type(type) when is_atom(type) do
case type do
:a -> "---->"
:ns -> "-...->"
:mx -> "---->"
:txt -> "---->"
:cname -> "---->"
_ -> "---->"
end
end
def attach_style_class(type) when is_atom(type) do
case type do
:soa -> ":::soa-record"
:ns -> ":::ns-record"
:mx -> ":::mx-record"
:a -> ":::a-record"
:aaaa -> ":::aaaa-record"
:cname -> ":::cname-record"
:virtual -> ":::subdomain-record"
:txt -> ":::txt-record"
:spf -> ":::spf-record"
:srv -> ":::srv-record"
:caa -> ":::caa-record"
_ -> ""
end
end
def enumerate_relationships(zone_tree) do
edges =
for i <- 0..(length(zone_tree) - 1) do
"#{i}"
end
Enum.join(edges, ",")
end
end
The construct_tree/1
function takes the parent domain and the list of subdomains, converting them to a node-and-relationship definition string.
parent_node = Dns.Visualisation.ZoneTreeDiagram.define_parent_node(zone_tree)
nodes_with_relationships =
Dns.Visualisation.ZoneTreeDiagram.define_dns_zone(zone_tree.display_name, zone_tree.zone_tree)
parent_zone = Dns.Visualisation.ZoneTreeDiagram.construct_parent_zone_subgraph(zone_tree)
Feeding this definition string into the scaffolded diagram definition provides us with the complete diagram definition.
dns_zone_diagram =
"""
#{dns_zone_diagram_configuration}
#{diagram_type};
#{parent_node}
#{nodes_with_relationships}
#{parent_zone}
#{diagram_styles}
"""
Kino.Markdown.new(dns_zone_diagram)
The diagram definition, stored in the dns_zone_diagram
variable, can now be used to draw the DNS zone diagram.
Kino.Mermaid.new(dns_zone_diagram)
Kino.HTML.new(
"""
Figure 3. Full zone tree graph of #{zone_tree.display_name} zone, queried on #{Calendar.strftime(zone_tree.queried_at, "%d") |> String.replace("0", "")} #{Calendar.strftime(zone_tree.queried_at, "%B %Y")}
""" <>
Dns.Visualisation.CSS.styles()
)
Audit summary and advice
Kino.Markdown.new("""
### BIMI
#{Map.get(parsed_dss_domain_scan_advice, "bimi", "") |> Enum.join("\n\n")}
### DKIM
#{Map.get(parsed_dss_domain_scan_advice, "dkim", "") |> Enum.join("\n\n")}
**DKIM record** _`#{Map.get(parsed_dss_domain_scan_result, "dkim", "") |> String.slice(0..60)}...`_
### DMARC
#{Map.get(parsed_dss_domain_scan_advice, "dmarc", "") |> Enum.join("\n\n")}
**DMARC record** _`#{Map.get(parsed_dss_domain_scan_result, "dmarc", "")}`_
### Domain
#{Map.get(parsed_dss_domain_scan_advice, "domain", "") |> Enum.join("\n\n")}
### MX
#{Map.get(parsed_dss_domain_scan_advice, "mx", "") |> Enum.join("\n\n")}
### SPF
#{Map.get(parsed_dss_domain_scan_advice, "spf", "") |> Enum.join("\n\n")}
**SPF record** _`#{Map.get(parsed_dss_domain_scan_result, "spf", "")}`_
""")
Kino.HTML.new(
"""
Figure 4. DNS record audit and automated advice for #{zone_tree.display_name} zone, queried on #{Calendar.strftime(zone_tree.queried_at, "%d") |> String.replace("0", "")} #{Calendar.strftime(zone_tree.queried_at, "%B %Y")}
""" <>
Dns.Visualisation.CSS.styles()
)