XFTFSecurity For the Folks

Lesson 4.5

Infrastructure Recon

Learning Objectives

  • After this, students can look up domain registration data with WHOIS/RDAP.
  • After this, students can query every major DNS record type with dig and read what each reveals.
  • After this, students can enumerate subdomains passively using certificate transparency logs and dedicated tools.
  • After this, students can use internet-scan search engines (Shodan, Censys) and the Wayback Machine’s CDX API to map an organization’s footprint without touching its servers.

Passive by Design

Everything in this file is passive or semi-passive: you query third parties (registries, public logs, scan databases, archives) that already hold information about the target, rather than probing the target’s own systems. That keeps you quiet and, importantly, on the right side of the authorization line — passive recon of public infrastructure data is legal; active scanning (port scans, brute-forcing) is not, without permission (see Module 2 and the passive-vs-active discussion in the intro script). The one gray area, noted below, is connecting to the target’s own web server, which puts your IP in their logs.

WHOIS and RDAP

When someone registers a domain, that registration is recorded and (partly) published. Two protocols expose it:

  • WHOIS — the old text-based protocol.
  • RDAP (Registration Data Access Protocol) — the modern JSON-based replacement. As of 28 January 2025, RDAP became the authoritative source for generic TLD (.com, .net, .org…) registration data, and ICANN began sunsetting the old WHOIS obligation. Most people still say “WHOIS” for the activity.

What you learn: registrar, creation/expiration/last-changed dates, name servers, and status flags. Since GDPR (2018), registrant names and addresses are usually redacted for individuals, so don’t expect a person’s contact details on a modern record — the dates and name servers are the durable value.

Command-line WHOIS:

whois companyx.com

Query RDAP directly and get clean JSON (rdap.org bootstraps to the authoritative server):

curl -s -L https://rdap.org/domain/example.com | jq .

Pull just the registration events:
curl -s -L https://rdap.org/domain/example.com | jq -r '.events[] | "\(.eventAction): \(.eventDate)"'

RDAP also answers for IPs and ASNs:
curl -s -L https://rdap.org/ip/93.184.216.34
curl -s -L https://rdap.org/autnum/15169

Free official web lookup: ICANN Lookup at lookup.icann.org. Historical WHOIS (pre-redaction records) is sold by services like DomainTools and WhoisXML — occasionally useful, mostly paid.

DNS with dig

DNS is a goldmine because it describes how an organization is wired without you touching their servers. dig is the standard query tool; +short trims the output to just the answer.

dig companyx.com A +short          # IPv4 address(es)
dig companyx.com AAAA +short       # IPv6 address(es)
dig companyx.com MX +short         # mail servers → email provider
dig companyx.com NS +short         # authoritative name servers → DNS host
dig companyx.com TXT +short        # SPF, DMARC, and verification tokens
dig companyx.com SOA +short        # zone admin + serial
dig _dmarc.companyx.com TXT +short # DMARC policy specifically

What each record type tells you:

Record Reveals
A / AAAA The server IPs — pivot into Shodan/Censys and IP-ownership lookups
MX Mail provider (e.g. Google Workspace vs. Microsoft 365 vs. self-hosted)
NS Who runs their DNS (their own, or Cloudflare/Route 53/etc.)
TXT SPF (v=spf1…), DMARC (v=DMARC1…), and domain-verification tokens
CNAME Third-party services a hostname points to (CDNs, help desks, marketing tools)
SOA Primary name server and zone serial number

Reading TXT verification records

TXT records are unusually revealing because organizations paste verification tokens there to prove domain ownership to SaaS providers. Each token names a service the org uses:

  • google-site-verification=… → Google Workspace / Search Console
  • MS=… or v=verifydomain MS=… → Microsoft 365
  • atlassian-domain-verification=… → Atlassian (Jira/Confluence)
  • facebook-domain-verification=…, docusign=…, stripe-verification=…, and so on

Reading the TXT records is like reading a partial list of an org’s vendors. Combined with MX (mail provider) and CNAMEs (pointed services), you can sketch the SaaS estate before ever looking at their website.

Note: DNS zone transfers (dig AXFR) would dump an entire zone at once, but a zone transfer request is sent to the target’s name server — that’s active, occasionally misconfigured to allow it, and out of scope unless you’re authorized. Awareness only for this class.

Certificate Transparency Logs

Every publicly trusted TLS certificate is published to append-only Certificate Transparency (CT) logs. Because organizations get certificates for their internal-ish hostnames (vpn., dev., staging., jira.), CT logs are one of the best passive ways to discover subdomains — you’re reading a public log, not scanning the target.

crt.sh is the easiest interface. In a browser, search %.companyx.com (the % is a SQL wildcard). For scripting, request JSON and extract the hostnames:

curl -s "https://crt.sh/?q=%25.companyx.com&output=json" | jq -r '.[].name_value' | sort -u

(%25 is the URL-encoding of %.) The name_value field holds the certificate’s subject and Subject Alternative Names — i.e. the hostnames. Sorting and de-duplicating gives you a clean subdomain list. Censys (below) also searches certificates and is a good cross-check.

Passive Subdomain Enumeration Tools

These tools aggregate CT logs, passive DNS, and many other public sources into one subdomain list — no packets to the target required.

subfinder (passive by design):

go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
subfinder -d companyx.com -silent
subfinder -d companyx.com -all -o subs.txt      # use all sources
subfinder -dL domains.txt -oJ -o subs.jsonl     # many domains, JSON lines
subfinder -ls                                    # list available sources

API keys for extra sources go in $HOME/.config/subfinder/provider-config.yaml.

OWASP Amass (v5, passive mode). Amass changed significantly in v5 — enumeration and result-viewing are separate steps, and results are stored in a graph database:

amass enum --passive -d companyx.com

Verify the exact v5 result-retrieval subcommand in class — v5’s output handling differs from older guides, and several flags were reworked. For a quick, reliable passive list, subfinder is the smoother choice right now; treat Amass as the deeper (and more finicky) option.

Internet-Scan Search Engines

Services like Shodan and Censys continuously scan the whole internet and let you search the results. Because they did the scanning, querying their database is passive for you — you learn what services a target exposes without sending it a single packet.

Shodan

Install the CLI and add your API key:

pip install -U --user shodan
shodan init YOUR_API_KEY

Search with filters (no space after the colon; quote multi-word values):
shodan search hostname:companyx.com
shodan search 'org:"Company X" port:443'
shodan search 'ssl.cert.subject.cn:companyx.com'
shodan host 93.184.216.34
shodan domain companyx.com

Useful filters: hostname:, org:, net: (CIDR), port:, product:, http.title:, ssl:, country:, city:. By default Shodan only matches the banner’s data field, so filters are how you target specific properties. Some filters and bulk features require a paid membership.

Censys

Censys uses the Censys Query Language (CenQL). It matches fields with : (tokenized/case-insensitive) or = (exact), and supports and/or/not:

host.services: (port = "443" and protocol = "HTTP")
cert.names: "companyx.com"
host.services.tls.certificates.leaf_data.subject.common_name: "companyx.com"

Searching cert.names: "companyx.com" is an excellent certificate-based way to discover hosts and subdomains — a good cross-check against crt.sh. (CenQL is the current syntax; it evolved from the older Censys Search 2.0 query language, so ignore pre-2024 syntax guides.)

Related engines worth knowing (details in Module 5): FOFA and ZoomEye (strong Asia coverage), and GreyNoise (who is scanning the internet — mostly defensive).

The Wayback Machine (CDX API)

The Internet Archive’s Wayback Machine archives historical snapshots of sites — old staff pages, removed documents, previous configurations. Its CDX API lists every archived URL for a domain, which surfaces forgotten endpoints and files. The base endpoint is http://web.archive.org/cdx/search/cdx.

Every archived URL under a domain, de-duplicated, as JSON:

curl -s "http://web.archive.org/cdx/search/cdx?url=companyx.com*&output=json&collapse=urlkey&fl=original"

Only successful pages, within a date range:
curl -s "http://web.archive.org/cdx/search/cdx?url=companyx.com*&output=json&filter=statuscode:200&from=2023&to=2025&collapse=urlkey&fl=original,timestamp"

Key parameters: url= (add * or matchType=domain/prefix for scope), output=json, fl= (fields), collapse= (de-dupe), filter=statuscode:200, from=/to= (YYYYMMDDhhmmss).

Two tools wrap this for you:

go install github.com/tomnomnom/waybackurls@latest
echo companyx.com | waybackurls

go install github.com/lc/gau/v2/cmd/gau@latest
gau --subs companyx.com          # pulls from Wayback, Common Crawl, OTX, URLScan

gau is broader (it also queries Common Crawl, AlienVault OTX, and URLScan), while waybackurls is Wayback-only and dead simple. Both are great for finding old URLs and file paths a site no longer links to.

Putting the Chain Together

A typical passive infrastructure sweep of an authorized companyx.com:
1. whois/RDAP → registrar, dates, name servers.
2. dig A/MX/NS/TXT → IPs, mail provider, DNS host, SaaS vendors from TXT tokens.
3. crt.sh + subfinder → subdomain list (vpn., dev., staging.…).
4. Shodan/Censys on the discovered IPs/certs → exposed services and software versions.
5. Wayback CDX / gau → forgotten pages and files.

You now have the org’s external footprint — hosts, services, vendors, and history — without having sent a single packet to their servers. That’s the whole point of passive recon.

Key Terms

  • WHOIS / RDAP: protocols for looking up domain, IP, and ASN registration data (RDAP is the modern JSON successor).
  • DNS record types: A/AAAA (addresses), MX (mail), NS (name servers), TXT (SPF/DMARC/verification), CNAME (aliases).
  • Certificate Transparency (CT) log: a public, append-only log of every trusted TLS certificate — a passive subdomain source.
  • Passive DNS: historical/aggregated DNS data collected by third parties, queried instead of the target.
  • Internet-scan search engine: a service (Shodan, Censys) that scans the internet so you can search the results passively.
  • CDX API: the Wayback Machine’s endpoint for listing all archived URLs of a domain.

Key Takeaways

  • All of this is passive: you query registries, public logs, and scan databases, not the target’s servers.
  • RDAP replaced WHOIS as authoritative for gTLDs in January 2025; registrant identities are usually GDPR-redacted.
  • DNS TXT verification tokens quietly list an organization’s SaaS vendors.
  • Certificate Transparency (crt.sh, Censys) is a top-tier passive way to enumerate subdomains.
  • Shodan and Censys let you see exposed services because they did the scanning — no packets from you.
  • The Wayback CDX API resurrects forgotten pages, endpoints, and files.

References

Free to use, copy, change and teach. Security For the Folks dedicates this course to the public domain under CC0 1.0 Universal: no permission or credit needed. Quoted material and the sources it links to belong to their own authors.