Testing Outbound Email in CI: Catch the Bug in the Bytes with Mailpit
Transport mocks prove your code called sendMail — not that the message leaving your box is correct. Point your app at a real SMTP sink, capture the RFC 5322 bytes, and assert on them. Here is the full CI recipe with Mailpit.
EvilMail TeamJuly 31, 202611 min read
A welcome email shipped to 40,000 new signups with a literal {{unsubscribe_url}} in the footer. The template engine had been handed a variable named unsubscribeUrl (camelCase), never found the snake_case key the template asked for, and — per its default config — silently emitted the raw placeholder instead of throwing. Nobody caught it, because every test in the suite mocked transporter.sendMail. The specs asserted that sendMail was called once, with the right recipient and a subject. All green. The bug lived one layer below the mock, in the bytes a real SMTP transport would have serialized.
That is the whole problem with how most teams test email. Mocking the transport tests your intent, not your output. You prove that your code *decided* to send a message. You prove nothing about the message itself — the MIME structure, the encoding, the header set, the rendered HTML. The only honest test captures the actual RFC 5322 message that leaves your application and asserts against it.
The pattern is simple: stand up a throwaway SMTP listener that accepts every message, stores it, and exposes it over HTTP so your tests can read it back and assert on it. The tool for that in 2026 is Mailpit.
Why transport mocks lie
A mock intercepts the call and inspects the arguments you passed in. But the interesting failures happen *after* that call, in the serialization step the mock replaces. Every one of these is invisible to a
jest.mock
on nodemailer or a
MagicMock
on
smtplib
:
Unrendered template variables — the {{unsubscribe_url}} from the intro. Your code passed a context object; whether the template consumed it correctly is a separate question the mock never asks.
Envelope sender vs. From header mismatch — you set from: "[email protected]" in code, but a misconfigured transport rewrites the envelope MAIL FROM to the SMTP auth user. DKIM aligns to one, your From shows the other, and Gmail quietly downgrades you.
Missing or blank Subject — a conditional that evaluates to undefined produces Subject: on the wire. The mock got undefined and shrugged.
Encoding corruption — emoji or non-ASCII in a body that should be quoted-printable or base64 but got mangled because a header claimed us-ascii.
Attachment Content-Type errors — a PDF sent as application/octet-stream, or an inline image with a broken Content-ID.
Missing bulk headers — no List-Unsubscribe, no List-Unsubscribe-Post. Under the Gmail/Yahoo bulk-sender rules (in force since February 2024 and unchanged in 2026), that gets your marketing mail throttled or binned.
Header injection — an unsanitized display name containing \r\n lets an attacker inject arbitrary headers. You only see it in the serialized output.
Draw the line clearly: a mock asserts on the *arguments* going into sendMail. A sink asserts on the *bytes* coming out of it. Everything worth testing lives in that gap.
Mailpit, not MailHog, in 2026
For years MailHog was the default. Use Mailpit now. MailHog has been effectively unmaintained since 2020 — its last meaningful release, v1.0.1, shipped in August 2020 — with open CVEs unpatched and Go module rot that makes it painful to build from source. It still runs, but you would be pinning your CI to abandonware.
Mailpit (github.com/axllent/mailpit) is the drop-in successor: a single ~15 MB static Go binary, actively released, with the same muscle-memory defaults — SMTP on 1025, web UI and API on 8025. What it adds is exactly what CI needs:
A real JSON API with message listing, full-message retrieval, and search.
HTML source checks, link checks, and a SpamAssassin hook — so "did it send" becomes "would it deliver."
SMTP AUTH and STARTTLS/TLS support, so you can exercise the credentialed sending path instead of pretending it doesn't exist.
A --max ring buffer so a runaway loop can't OOM the runner.
One caveat worth naming: Mailpit persists to SQLite by default. In CI you want speed and a clean slate, so leave MP_DATABASE unset (in-memory) or point it at a tmpfs path.
For CI, run it as a sidecar so it mirrors production wiring — your app talks to a real network socket, not an in-process fake. A docker-compose.ci.yml at repo root:
The /mailpit readyz subcommand is built into the image and exits 0 only once the SMTP and HTTP listeners are up, so Compose won't report the container healthy too early. The same wiring as a GitHub Actions services: block in .github/workflows/test.yml:
Django is EMAIL_HOST=localhost, EMAIL_PORT=1025, EMAIL_USE_TLS=False. When you specifically want to test credentialed sending, keep MP_SMTP_AUTH_ACCEPT_ANY=1 and MP_SMTP_AUTH_ALLOW_INSECURE=1 set and give the transport any username/password — Mailpit accepts them, and you exercise the AUTH handshake instead of skipping it.
Writing the assertion
SMTP delivery is asynchronous relative to your test. The moment your code returns from sendMail, the message may not have hit Mailpit yet. A bare assertion right after the trigger races and flakes. Poll the API with a budget; never `sleep`.
The loop: clear the mailbox, trigger the action, poll GET /api/v1/messages until the total climbs, then fetch the full message and assert. A Vitest helper:
js
const API = "http://localhost:8025/api/v1";
beforeEach(async () => {
await fetch(`${API}/messages`, { method: "DELETE" });
});
async function waitForMessage(timeout = 5000) {
const start = Date.now();
while (Date.now() - start < timeout) {
const { total, messages } = await (await fetch(`${API}/messages`)).json();
if (total > 0) {
return (await fetch(`${API}/message/${messages[0].ID}`)).json();
}
await new Promise((r) => setTimeout(r, 100));
}
throw new Error("no message captured within timeout");
}
test("welcome email renders a real unsubscribe link", async () => {
await registerUser({ email: "[email protected]" });
const msg = await waitForMessage();
expect(msg.To[0].Address).toBe("[email protected]");
expect(msg.From.Address).toBe("[email protected]");
expect(msg.Subject).not.toBe("");
expect(msg.HTML).not.toContain("{{"); // the intro bug, caught
expect(msg.HTML).toContain("https://evilmail.pro/unsubscribe/");
});
That single negative assertion — expect(msg.HTML).not.toContain("{{") — is the one the mocked suite could never make, and it catches the exact class of bug that shipped the broken footer. The DELETE runs in beforeEach, not afterEach, deliberately: if a test crashes mid-run, the *next* test still starts from a clean mailbox instead of inheriting garbage.
Keeping specs isolated and fast
State bleed is the number-one source of email-test flakes: a message from test A gets picked up by test B's poll, and the wrong assertion fires. The rules that kill it:
Clear in `beforeEach`, never `afterEach`. A crashed test leaves a clean slate for the next one.
Run in-memory in CI. No SQLite file to sync-flush; leave MP_DATABASE unset.
Cap the buffer with MP_MAX_MESSAGES=500 so a runaway loop can't exhaust the runner's memory.
Do not share one Mailpit across parallel workers. Either serialize the email specs (fileParallelism: false in Vitest, --runInBand in Jest for that project), or give each worker a tagged recipient and filter: worker 3 sends to worker3@test and queries GET /api/v1/search?query=to:worker3@test.
Poll with an explicit budget (5000 ms, 100 ms interval) and throw loudly on timeout. A broken send should surface as a red test, not a 10-minute hang that eventually times out the whole job.
Beyond capture: content and deliverability lint
Once you own the real message, email becomes just another tested output. Mailpit's extras turn "it sent" into "it would deliver":
bash
# flags client-compatibility issues + warns near Gmail's clip threshold
curl -s http://localhost:8025/api/v1/message/$ID/html-check
# catches dead links before customers do
curl -s http://localhost:8025/api/v1/message/$ID/link-check
The HTML check reports total message size. Gmail clips messages larger than ~102 KB — everything past the cut, including your unsubscribe footer, gets hidden behind a "View entire message" link. Gate on it: fail the build if a template crosses 102 KB. The SpamAssassin hook returns a score; a reasonable CI gate fails anything above 5.0. And assert the header set directly from the captured Headers — require List-Unsubscribe plus List-Unsubscribe-Post: List-Unsubscribe=One-Click on bulk mail, and confirm the envelope sender aligns with your DKIM d= domain. If a template regresses on any of these, the build goes red before a customer ever sees it.
Pre-merge checklist
Mailpit sidecar running with a readyz healthcheck.
App SMTP config points at localhost:1025, no TLS for the base case.
Mailbox cleared in beforeEach (not afterEach).
Poll the API with a timeout budget — never sleep.
Assert To / From / Subject + rendered HTML, and not.toContain("{{").
Email specs serialized, or per-worker recipient isolation.
The sink pattern generalizes — any RFC 5322-compliant capture server will do the job, and the assertions you write against Mailpit's API port cleanly. Mailpit is simply the one that won't fight you in 2026.
