Splitting the Postfix Queue Across Per-Destination Transports
One shared smtp transport is why a single tarpitting receiver drags your whole outbound queue down. Shard the queue by destination and each lane gets its own concurrency, rate budget, and scheduler accounting — so the slow domains crawl without touching the fast ones.
EvilMail TeamJuly 28, 202612 min read
Your newsletter to 40,000 Gmail recipients should be done in ninety seconds. It isn't, because 300 messages to one corporate Exchange that greylists everything are sitting at the front of the same active queue, grabbing delivery slots on every retry. That is head-of-line blocking, and no amount of default_process_limit tuning fixes it — the problem is not process count. It is that Postfix ships with onesmtp transport and one shared active queue, and that single lane is the whole bottleneck.
The fix is architectural: shard the queue by destination. Give the fast mailbox providers an aggressive, connection-cached lane, and quarantine the slow tarpitters into a throttled lane where a rate delay collapses them to concurrency 1. They crawl. Everyone else flies. Below is exactly how, including the two-layer config model that copy-paste tuning guides silently get wrong.
The single-transport trap
When qmgr schedules delivery it builds an in-memory active queue — the working set of messages it is trying to hand to delivery agents right now. That set is bounded by qmgr_message_active_limit (default 20000), and the number of concurrent delivery processes is capped by default_process_limit
(default 100). Per destination, concurrency starts small and ramps:
default_destination_concurrency_limit
is 20, and each transport begins at
initial_destination_concurrency
(5), then grows.
Here is the mechanism that bites you. qmgr runs a concurrency feedback loop per destination: every delivery that succeeds nudges the window up by default_destination_concurrency_positive_feedback (1), and a connection refusal or 4xx deferral nudges it down, with default_destination_concurrency_failed_cohort_limit (1) triggering the back-off. So qmgr *does* notice a misbehaving domain and shrinks its window. What it does not do is isolate that domain. All destinations share the same active queue and the same pool of delivery slots. When Yahoo starts answering 421 4.7.0 [TSS04] ... temporarily deferred, those messages defer, land in the deferred queue, and their scheduled retries keep re-entering the active queue and eating slots your Gmail traffic wanted.
One transport means one shared scheduler. Split the transport and you get isolation almost for free: separate scheduler accounting, a separate concurrency window, and a separate rate budget per lane.
Diagnose before you shard
Do not tune blind. Find out which domains actually own your deferred queue before you build a lane for them.
bash
# Which domains dominate the deferred queue, across age buckets?
qshape deferred
# Same, but sorted by total so the worst offender is on top
qshape -s deferred
qshape prints a matrix: rows are domains, columns are age buckets in minutes (T 5 10 20 40 80 160 320 640 1280 1280+). A domain with a fat row far to the right is one whose mail is aging out — stuck for over an hour. That is your isolation candidate. A domain heavy only in the T/5 columns is transient and not worth a dedicated lane.
The second confirmation lives in the logs. Every delivery logs a delays= field with four numbers:
delays=0.1/1200/0.2/0.5
Read them as before-qmgr / time-in-active-queue / connection-setup / transmission. The second number is the smoking gun. 1200 means the message sat in the active queue for twenty minutes waiting for a delivery slot — classic HOL blocking. Then name the offenders by their defer codes:
The signatures worth a dedicated slow lane, as of 2026: Yahoo 421 4.7.0 [TSS04] ... temporarily deferred, Microsoft 451 4.7.500 Server busy, and generic greylisters 450 4.2.0 ... try again later. Threshold rule of thumb: if a domain holds more than ~200 messages past the 320-minute bucket, or shows delays= second-field values consistently above 300 seconds, it earns its own throttled lane.
Carve transports in master.cf
Clone the smtp delivery agent into named services. In /etc/postfix/master.cf:
Everything here is a legitimate smtp-agent override. syslog_name gives each lane its own log prefix so you can watch them independently. smtp_connection_cache_on_demand and smtp_connection_reuse_time_limit control connection reuse — on for the fast lane so you pipeline many messages down one held connection to Gmail, off for the slow lane where holding a connection open to a tarpitting server buys nothing. These belong in master.cf because they are properties of the delivery agent — the process that opens the SMTP connection.
What must not go here: concurrency and rate limits. That is the mistake that turns half the tuning blogs on the internet into no-ops.
Route with transport_maps
Build /etc/postfix/transport mapping domains to lanes:
The detail that separates a working config from a broken one: transport_maps supports exactly three left-hand key forms — user@domain, domain, and .domain (the leading dot matches the parent domain and all subdomains). **There is no * wildcard.** The catch-all is not a table entry at all; it is default_transport in main.cf. The trailing colon with an empty nexthop means "use this transport, resolve the destination normally via MX."
postfix reload # reload, never restart, after a map or config change
Budget each lane
Here the two-layer model becomes concrete, and here most guides go silently wrong. Concurrency, rate delay, and recipient batching are not set with -o in master.cf. qmgr reads them from main.cf, keyed by the master.cf service name. The parameter pattern is <service>_destination_<knob>:
The critical side effect: a `_destination_rate_delay` greater than zero forces that transport's concurrency to 1, regardless of the concurrency limit you set. Postfix delivers one message at a time when a rate delay is active, then waits the delay between the end of one and the start of the next. Setting smtp-slow_destination_concurrency_limit = 2 alongside rate_delay = 3s is belt-and-suspenders — the rate delay wins, and you get serialized, one-every-three-seconds delivery. That is exactly right for a server that returns 451 Server busy the moment you open a second connection.
The fast lane keeps the default feedback loop, so if Gmail suddenly starts deferring, qmgr still backs its window off automatically — you get throttle-on-trouble without giving up the ceiling.
The penalty box
Combine three tiers and you have a self-defending outbound path:
`default_transport = smtp-fast` — the sane default. Anything not named in the map gets the aggressive lane. Most of the internet delivers fine at high concurrency.
`smtp-slow` for the known tarpitters you found with qshape — Yahoo, AOL, that one corporate Exchange.
Promote and demote domains by editing /etc/postfix/transport, running postmap lmdb:/etc/postfix/transport, then postfix reload. No restart, no dropped connections. When a domain that lived in the fast lane starts throwing 421 for a day, drop it into smtp-slow for the duration and pull it back out once the reputation event clears.
The rate-delay-forces-concurrency-1 behavior is the whole point of the penalty box: a freshly-detected offender is guaranteed to send one message at a time, spaced out, so it can never again grab a fistful of slots from your good traffic.
Verify it worked
Confirm both config layers actually resolved. postconf -P reads the effective per-service overrides from master.cf; postconf <param> reads main.cf:
bash
# master.cf agent overrides resolved for the slow lane?
postconf -P | grep smtp-slow
# main.cf concurrency keying landed on the service name?
postconf smtp-slow_destination_concurrency_limit
# => smtp-slow_destination_concurrency_limit = 2
# read the deferred queue, worst offenders on top
qshape -s deferred | head
# watch each lane's real concurrency + delay independently
grep 'postfix/fast/smtp' /var/log/mail.log | grep -Eo 'delays=[^,]+' | tail
grep 'postfix/slow/smtp' /var/log/mail.log | grep -Eo 'delays=[^,]+' | tail
The split syslog_name prefixes (postfix/fast/smtp, postfix/slow/smtp) are what make this observable. Watch the fast stream: its delays= second field should stay near zero as concurrency climbs toward 40. Watch the slow stream: deliveries appear roughly every three seconds and the queue drains slowly but steadily — without ever touching the fast lane's numbers.
Checklist before you call it done:
[ ] Identified the offending domains via qshape and the delays= second field
[ ] Cloned smtp into named services in master.cf, each with a distinct syslog_name
[ ] Put connection caching / MX-session limits on the agent in master.cf
[ ] Put concurrency / rate / recipient limits in main.cf, keyed by the service name
[ ] Built the transport map with real key forms (domain, .domain) and set default_transport — no * wildcard
[ ] postfix reload (not restart); confirmed with postconf -P and postconf smtp-slow_destination_concurrency_limit
[ ] Watched the split syslog streams: fast lane concurrency climbed, slow lane held at 1
One operational reminder: the map is a lever you pull during incidents, not a set-and-forget file. Re-run qshape -s deferred weekly and move domains between lanes as sender reputation and receiver behavior drift.
