When we say "Amazon SES is the #1 ESP" or "self-hosted email fell to 22.53%", those claims are only as good as the pipeline that produced them. Measurement studies that hide their methodology invite exactly the skepticism they deserve. So here is ours, end to end — what we scan, how we classify, and the specific places where the numbers can mislead. If you only take one thing away: every figure we publish can be checked against the machine-readable snapshot at api/latest.json.
The raw material: OpenINTEL and Tranco
We do not run our own million-domain DNS crawler. The heavy lifting is done by OpenINTEL, the long-running active DNS measurement project at the University of Twente, which resolves large domain lists daily and archives the answers. Our population is the Tranco top-1M — a research-oriented domain ranking designed to resist the manipulation and day-to-day noise of raw traffic lists.
Replaying that archive gives us 192 snapshots from January 2016 through the 2026-07-05 snapshot. In the latest cut, 664,715 domains publish MX records, 623,370 publish SPF, and 459,124 publish DMARC. Those three record types are the substrate for everything else.
Step 1: who receives the mail (MX classification)
Each domain's MX targets are matched against a dictionary of 310+ regex patterns mapping hostnames to providers — aspmx.l.google.com to Google Workspace, *.mail.protection.outlook.com to Microsoft 365, *.pphosted.com to Proofpoint, and so on. Two rules keep the results honest:
- Classification follows the primary MX — the record with the lowest preference value. A domain with a security gateway first and Google as backup is counted as gateway-fronted, because that is where the mail actually goes.
- One domain, one class. Mailbox shares are exclusive and sum to 100%, unlike ESP shares (below).
Whatever the dictionary cannot match falls into the long tail: 36,377 unique unmatched MX hostnames covering 81,514 domains — 12.45% of the MX-publishing population. We publish that tail rather than sweeping it into "other", because a residual that size deserves scrutiny.
Step 2: who sends the mail (SPF attribution)
Sending infrastructure is read from SPF. We parse the TXT record at the apex of each domain and attribute include: mechanisms to ESPs — include:amazonses.com, include:sendgrid.net, and so on. This attribution currently covers 81.5% of SPF includes. Because one domain routinely lists several providers (marketing, transactional, support), ESP shares deliberately sum past 100% — that is a feature of the measurement, not a bug.
On top of SPF we detect SaaS verification tokens in TXT records (google-site-verification, MS=, Atlassian, Stripe, and about 70 more patterns), which turn DNS into a rough map of a company's software stack.
Where it breaks: the honest list
SPF flattening makes ESPs invisible
Some operators replace include: chains with raw IP ranges to stay under SPF's 10-lookup limit. A flattened record still authorizes SendGrid's servers — but no longer names SendGrid, so we cannot attribute it. Consequence: every ESP share we publish is a floor, not a ceiling, and the bias grows with sender sophistication, since flattening correlates with mature email programs. This affects every DNS-based survey, not just ours.
CNAME chains on MX are not expanded
If an MX target is a CNAME pointing at a known provider, we classify the name we see, not the name it resolves to. Some genuinely-Google or genuinely-Microsoft domains therefore land in "Unknown". This slightly inflates the long tail and deflates major-provider shares.
Dictionary growth can masquerade as market growth
When we add a new pattern, every domain it matches appears in the series at once — a step function that looks like explosive adoption but is actually us getting smarter. The canonical example is HubSpot: it entered our SPF dictionary in April 2026 and immediately shows 3.18% of SPF domains. That number is a legitimate level; treating its appearance as a trend would be nonsense, which is why we mark late-start series with their dictionary entry date wherever they are charted.
Apex-only SPF misses subdomain senders
We read SPF at the domain apex. Organizations that send exclusively from subdomains (mail.example.com) with their own SPF records are undercounted on the sending side, even though their mailbox classification remains correct.
Directional trends in long-running series are robust — the self-hosted decline or the SES rise dwarf any classification noise. Absolute ESP shares are conservative floors. Sudden jumps in a single series deserve suspicion: check whether the series has a dictionary start date before citing it as adoption.
Check our work
The pipeline runs nightly and publishes everything it produces:
- The daily email infrastructure report — all series, charts, and a 17-section methodology covering every rule described here in more detail.
api/latest.json— the current snapshot as machine-readable JSON, suitable for reproduction, diffing, and citation.llms.txt— a structured summary for AI agents and automated pipelines that want the numbers without scraping HTML.- The long-tail CSVs — the full list of unmatched MX hostnames, which is simultaneously our error bar and an open invitation: if you can classify what we could not, we want the patterns.
Methodology sections that hide limitations are marketing. We would rather you know exactly where this dataset bends — and then use it anyway, because ten years of consistent measurement with documented bias beats a perfect measurement that does not exist.