Capacity Planning a Mail Platform: Sizing CPU, IOPS, and Storage Against Real Volume
Mail platforms fall over on IOPS and fsync latency long before they run out of disk or CPU. Here is how to size all three budgets against a concrete workload model — and why the burst, not the average, is what pages you at 3am.
EvilMail TeamJuly 27, 202611 min read
A mail queue backs up. You SSH in expecting CPU pinned at 100% or a full disk. Neither. Load average is fine, df -h shows 40% used, and yet postqueue -p scrolls for pages. The active queue won't drain. The number you weren't watching is fsync latency on the mail spool, which just crossed 20ms — and every local delivery agent is blocked waiting on fdatasync() to return.
Nobody sizes for this failure mode. Capacity planning for mail gets treated as a storage question — "how many terabytes do we need?" — when the thing that actually takes the platform down is IOPS and synchronous write latency. Storage is the easy budget: it's cheap, it grows predictably, and running out of it is a solved problem. The budgets that bite are the ones that never show up on a capacity graph.
Start from the workload, not the hardware
Every sizing decision derives from four inputs. Nail these before you look at a single spec sheet:
Peak inbound msg/s — not the daily average, the p99 burst.
Mail Capacity Planning: Sizing CPU, IOPS & Storage | evilmail.pro — EvilMail Blog
Average message size
— with a fat tail for attachments.
Total mailbox count — current, plus its trajectory.
Growth rate — new mailboxes/month and per-mailbox message accrual.
I'll carry one baseline through the whole article so the numbers stay concrete: 200,000 mailboxes, 2 million messages/day, average message 75KB. Two million messages a day is about 23 msg/s averaged flat — a deceptively gentle number. Mail doesn't arrive flat. A p99 burst at 4x the daily mean puts you at roughly 92 msg/s, and that burst is what every component has to survive. Provision for 23 msg/s and you've built the queue you'll be staring at come 3am.
The temp-email profile behind evilmail.pro sits at an extreme end of this space: write-heavy, extremely high churn, short TTL. Messages land, get read once (or scraped by an API client), and expire within minutes to hours. Storage stays nearly flat because aggressive expiry deletes as fast as delivery writes — but IOPS stays pinned the entire time. It's the mirror image of a classic archival ISP mailbox that fills disk slowly and barely writes. Same software, opposite sizing.
The IOPS budget — this is the one that bites
Mail is a random, small, synchronous write workload, which is close to the worst thing you can hand a storage device. A delivered message is never one write. In Maildir it's a create in tmp/, a rename() into new/, an fsync(), plus directory metadata updates. In dbox/mdbox it's an append to a mail file plus a cascade of Dovecot index updates. Count the sync operations per delivered message for mdbox and you land around 3 to 6: the message append, dovecot.index.log, dovecot-uidlist, the cache write, and often the index itself.
At 92 msg/s and 5 writes/message, delivery alone is ~460 write IOPS. Then read amplification piles on: IMAP clients polling, mobile push checking for new mail every few seconds, SEARCH across mailboxes, thread reconstruction. On a real consumer platform that multiplier is easily 2-4x. Now you're budgeting 1,500-2,000 IOPS at peak, all of it small and random, much of it synchronous.
The device choice follows directly. Spinning rust for mail indexes is malpractice in 2026 — a 7,200 RPM disk gives you ~150 random IOPS and 8-12ms seeks, and your index writes alone would saturate it. SATA SSDs are fine for bulk mail bodies, but their sync-write latency under queue depth is inconsistent. Put indexes and tmp on NVMe, separate from bulk storage. NVMe sustains tens of thousands of random IOPS with sub-millisecond fsync — and that's the metric that keeps the active queue draining.
Never guess this from a spreadsheet. Measure the device you actually have:
bash
# sustained random-write IOPS, await (ms), and %util per device
iostat -x 1
# per-process disk I/O — find which daemon is actually writing
pidstat -d 1
# fsync-ish latency against the mail volume — watch the p99, not the mean
ioping -c 20 /var/vmail
If ioping shows sync latency creeping past 15ms on the mail spool, you're already living on borrowed time. Alert on that number directly — not on %util, which happily reads 100% on a device that's keeping up fine.
The CPU budget: TLS, spam, DKIM
Mail CPU doesn't go where people assume. The actual local delivery — writing bytes to a mailbox — costs sub-millisecond CPU. The expensive work all happens before delivery:
Opportunistic TLS handshakes. Every SMTP connection does STARTTLS; every IMAP session negotiates TLS. On a high-connection-count platform this is a real slice of CPU, which is why session resumption and modern ciphers matter.
Content scanning. rspamd (or SpamAssassin, if you enjoy suffering) is the dominant consumer — tens of milliseconds of CPU per message, more with heavy rule sets, Bayes, and network checks. This single number dwarfs everything else.
DKIM signing and verification — cheap per message, but non-zero at volume.
Sieve filtering on delivery.
Size cores against scan concurrency, not delivery rate. At 92 msg/s peak and, say, 40ms of CPU per scan, you need roughly 92 × 0.04 = 3.7 cores continuously busy just scanning — before TLS, before anything else. Provision 6-8 cores for that stage with headroom, and treat rspamd worker count and Postfix process limits as one system. If default_process_limit lets Postfix hand rspamd more concurrent messages than it has workers to scan, you get context-switch thrash and latency spikes that look like a CPU shortage but are really a coordination bug:
ini
# postfix main.cf — match delivery concurrency to what rspamd can absorb
default_process_limit = 100
smtpd_client_connection_count_limit = 50
lmtp_destination_concurrency_limit = 20
One 2026 note worth acting on: for TLS- and scan-bound mail, modern ARM (Graviton and equivalents) is genuinely competitive per-core and cheaper per msg/s. Mail is exactly the many-small-tasks workload where ARM's economics show up. Benchmark your rspamd throughput on both before committing to a fleet.
Fill factor is the honest part — nobody uses their full quota, so 20-40% is realistic for consumer mail. The overhead for indexes, logs, and the active queue is small but real; budget 2-5% on top. Then keep a separate reserve for snapshots and backups, because that reserve is what saves you during an index rebuild.
Two traps hide inside the easy budget.
Inode exhaustion. Maildir stores one file per message. At millions of ~75KB files you can run out of inodes long before you run out of bytes — and the failure looks bizarre: writes fail with "No space left on device" while df -h shows plenty free. Check both:
bash
df -h /var/vmail # bytes
df -i /var/vmail # inodes — the one that surprises people
The fix is dbox/mdbox, which packs many messages into fewer, larger files and largely eliminates the inode problem. For any write-heavy or high-churn platform, mdbox isn't optional.
Compression and dedup change the math. Dovecot's zlib plugin (zstd or gz) typically buys 30-50% on text-heavy mail for modest CPU, and single-instance storage dedups identical attachments across recipients. A config that does both:
For temp-email the storage lever isn't compression — it's TTL. Aggressive expiry keeps the byte count nearly flat regardless of ingest rate. Retention policy, not disk size, is the storage plan.
Modeling growth and headroom
Turn the four inputs into a forward projection. A serviceable model is linear:
The headroom rule is non-negotiable: target 60-70% steady-state utilization on IOPS and storage. The remaining 30-40% isn't slack you're wasting — it's the room a p99 burst consumes, and the room a reindex-after-corruption needs when Dovecot rebuilds indexes for a large mailbox while normal traffic keeps flowing. Run at 90% steady state and the first corruption event becomes an outage.
The scale-up-vs-scale-out trigger is measurable. When a single node's peak IOPS crosses ~70% of the measured device capacity, or fsync p99 starts climbing under load, stop buying bigger boxes. Move to a Dovecot director with per-user sharding so mailboxes distribute across nodes with session affinity. Scaling up — bigger NVMe, more cores — buys you time; scaling out buys you a platform.
Instrumentation: measure before you provision
Capacity planning without a two-week measured baseline is astrology. Establish the real numbers first:
bash
# storage IOPS, await, utilization
iostat -x 1
# queue health — deferred is the canary
postqueue -p | tail
qshape deferred
# per-mailbox size and message counts
doveadm quota get -A
doveadm mailbox status -u [email protected] 'messages' 'vsize' '*'
# spam scanner throughput and scan timing
rspamc stat
qshape deferred shows you the shape of a problem before it becomes one: a growing deferred queue clustered on one destination is a different problem than a broad backup, and you size for them differently.
Sizing checklist
Run this down before you sign off on a config or a hardware order:
Modeled the p99 burst, not the average — sized every budget at 3-4x daily mean.
Indexes and tmp on NVMe, physically separate from bulk mail storage.
Measured real fsync latency with ioping; set an alert at ~15ms on the mail spool.
Counted writes-per-message for your storage format (mdbox ~3-6; Maildir worse).
Sized IMAP read amplification from real client behavior, not a guess.
CPU sized against scan concurrency, with rspamd workers matched to Postfix process limits.
Checked `df -i`, not just `df -h` — and used mdbox to dodge the inode trap.
Enabled zlib/zstd and confirmed the CPU-vs-storage tradeoff on your hardware.
Set headroom at 60-70% steady-state on IOPS and storage.
Defined the scale-out trigger (~70% device IOPS or rising fsync p99) as a real alert, not a vibe.
Get the IOPS and fsync budgets right and the platform stays boring. Get them wrong and no amount of spare disk or idle CPU will save you when the burst arrives.
