SMTP at Full Throttle: PIPELINING, CHUNKING, and BDAT
SMTP throughput is bound by round trips, not bandwidth. A classic transaction spends four round trips per message just waiting for replies — three of them before you may send a single body byte. Here is how PIPELINING and BDAT collapse that to one flight per message, without desyncing the stream.
EvilMail TeamAugust 1, 202611 min read
You have a warm TCP connection to an MX, TLS is already negotiated, and ten thousand messages are queued behind it. Your link is 10 Gbit. And you are still only pushing two or three messages a second. Nothing is saturated — not CPU, not the NIC, not the pipe. What throttles you is the speed of light and a handful of blocking replies.
Classic SMTP is a stop-and-wait protocol. Every MAIL FROM, every RCPT TO, and the DATA command each require the client to send, then sit and wait a full round trip for the server's reply before it may say the next thing. At 80 ms RTT you burn roughly 320 ms per message just waiting — 240 ms of it before you are even allowed to send the body. Fatten the pipe all you want; the ceiling does not move, because the cost is measured in round trips, not bytes.
The fix is two extensions that have been in the RFCs for over two decades and are still underused: PIPELINING (RFC 2920) lets you fire the command handshake in one flight instead of ping-ponging for each reply, and CHUNKING / BDAT (RFC 3030) removes the last synchronization barrier — the 354 "go ahead" — by framing the payload with an explicit byte count. Together they take a well-behaved sender from ~4 RTT per message down to 1. Here is exactly how, and where it still breaks in 2026.
Count the round trips before you optimize anything
SMTP PIPELINING, CHUNKING & BDAT: Cut Round Trips for Throughput — EvilMail Blog
Walk one message across a persistent, already-authenticated connection. Every arrow that goes right-then-left is a round trip you pay for:
text
C: MAIL FROM:<[email protected]> SIZE=20480
S: 250 2.1.0 Ok
C: RCPT TO:<[email protected]>
S: 250 2.1.5 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: ... (body)
C: .
S: 250 2.0.0 Ok: queued as 4F2A1
That is four synchronous waits after EHLO: the two 250s for the envelope, the 354 to unlock DATA, and the final 250 after the terminating dot. The 354 is the one that hurts most conceptually. DATA is a two-step command by design — the server must acknowledge before the client is allowed to transmit the body — so in classic SMTP that barrier is unavoidable. You cannot pipeline your way past it.
The math is brutal and bandwidth-independent. At 80 ms RTT:
4 RTT (stop-and-wait) = 320 ms/message ≈ 3 msg/s
2 RTT (pipelined handshake, still blocked at 354) = 160 ms ≈ 6 msg/s
1 RTT (BDAT single flight) = 80 ms ≈ 12.5 msg/s
And this is the *good* case, where the connection is reused. If your sender opens a fresh connection per message — a shockingly common mistake in homegrown code and some queue workers — you also pay TCP's 1 RTT handshake plus TLS's ~2 RTT on top, before you have even said EHLO. That is why per-message-connection senders are not merely slow, they are catastrophically slow, and why the single most important optimization is boring: reuse one warm connection for the whole queue.
PIPELINING: stop waiting for every 250
RFC 2920 is deceptively simple: the client may send a group of commands in one write() without waiting for the individual replies, and the server buffers its input and returns the responses in order. No new verbs, no negotiation beyond seeing PIPELINING in the EHLO response.
The rules that actually matter are about *grouping*. Some commands may be batched freely — MAIL, RCPT, and RSET. Others must end a group, meaning the client has to read the reply before it continues, because the outcome changes what happens next: EHLO, DATA, VRFY, EXPN, TURN, and QUIT. So you can legitimately send MAIL FROM + all your RCPT TOs + DATA in a single flight — but DATA's 354 still terminates the group, so you wait once before streaming the body. That is the ~2 RTT case, not 1.
Three wire-level details separate a working pipelined sender from a mysteriously slow one:
One `write()` per flight. If you emit each command with a separate syscall, Nagle's algorithm on your side plus delayed-ACK on theirs can conspire to add ~40 ms of dead air between packets. Set TCP_NODELAY on the socket and coalesce the whole group into a single buffer before writing.
Still read every reply. Pipelining defers reading, it does not eliminate it. A RCPT can be rejected 550 while MAIL succeeded, and that changes which recipients you actually committed. Parse all responses, in order, and reconcile them against what you sent.
The server may reply mid-flight. A conforming server can send its MAIL250 before it has finished reading your RCPT. Don't assume replies only arrive after your whole flight lands.
BDAT and CHUNKING: remove the 354 barrier
CHUNKING (RFC 3030) replaces DATA entirely with a new command: BDAT <octet-count> [LAST]. The integer is the exact number of octets that follow the CRLF ending the BDAT line — the server reads precisely that many bytes as the chunk and then expects the next SMTP command. LAST marks the final chunk and closes the message.
Because the length is explicit, three things vanish at once: there is no 354 handshake, no <CRLF>.<CRLF> terminator to detect, and no dot-stuffing. That last point is a real CPU win, not just elegance — classic DATA requires the server to scan every single line looking for a leading . to strip (transparency), while BDAT is a raw read of a known length, effectively a memcpy. On large messages at volume, that scan is measurable.
And critically: with no barrier left, MAIL FROM + RCPT TO + BDAT <n> LAST + the n body bytes all go out in one flight with nothing to wait for. One RTT per message. Here is the whole thing over the wire, against Google's inbound MX:
EHLO mta1.evilmail.pro
250-mx.google.com at your service
250-SIZE 157286400
250-8BITMIME
250-PIPELINING
250-CHUNKING
250-SMTPUTF8
250 ENHANCEDSTATUSCODES
# one single-flight write() below:
MAIL FROM:<[email protected]> SIZE=1024
RCPT TO:<[email protected]>
BDAT 1024 LAST
...exactly 1024 octets of RFC 5322 message, sent raw...
# server replies, in order, batched back:
250 2.1.0 OK
250 2.1.5 OK
250 2.0.0 OK 1701... - gsmtp
You wrote the envelope, the framing, and the entire body in one shot and read three 250s back. That is the whole optimization.
Chunk sizing, BINARYMIME, and failing early
For small messages, a single BDAT <n> LAST is the whole story. For large ones you don't want to buffer the entire body in memory before writing the count — stream it as several chunks, e.g. BDAT 65536, BDAT 65536, … then a final BDAT <remainder> LAST. Chunk boundaries are pure byte offsets; they are allowed to split mid-line, and the receiver reassembles them transparently. That lets you start writing bytes as soon as the first 64 KB is ready.
BDAT 0 LAST is legal and useful — it closes a message after you have already sent the content in prior chunks.
BINARYMIME (MAIL FROM:<...> BODY=BINARYMIME) rides on CHUNKING and lets you ship raw 8-bit or binary content with no base64 inflation — a ~33% size saving on attachments. The catch is fatal if you ignore it: a BINARYMIME message cannot be gatewayed to a hop that doesn't support it, because there is no defined downgrade. If the next relay speaks plain 7-bit, the message is stuck. Unless you control both ends, prefer 8BITMIME, which every modern receiver announces and which downgrades cleanly.
Finally, always declare SIZE=<n> on MAIL FROM. A server that advertises SIZE 157286400 will reject an oversized message with 552 at the envelope stage — before you stream a 40 MB chunk you would otherwise have to abort mid-flight. Failing early is free; failing late costs you the whole upload.
What actually supports this in 2026
Gate everything on EHLO. Parse the capability lines every connection — PIPELINING, CHUNKING, BINARYMIME, SIZE, 8BITMIME — and branch. Never assume; the same MX can change behind a load balancer.
The uncomfortable reality: Postfix, still one of the two most common inbound MTAs on the internet, implements PIPELINING on both sides but has never supported CHUNKING/BDAT as a receiver. So a large share of the mail you send will never see a CHUNKING advertisement, and you fall back to pipelined DATA (~2 RTT). Where it *is* offered you get the full win: Exim has advertised CHUNKING since 4.88 (2017, chunking_advertise_hosts = * by default), Sendmail supports it, Google's inbound MX (gmail-smtp-in) advertises CHUNKING, and Microsoft/Exchange advertises both CHUNKING and BINARYMIME. The rule writes itself: use BDAT when offered, fall back to pipelined DATA otherwise.
Two subtleties earn their own paragraph:
**BDAT makes error handling *cleaner*, not just faster.** If your MAIL FROM is rejected 5xx inside a pipelined BDAT flight, the server still consumes exactly the declared byte count, so the stream stays framed and in sync — you can read the errors and move on. A botched pipelined-DATA fallback, by contrast, can desync: miscount where the 354 was expected and body bytes get interpreted as commands, poisoning the connection. Counting is more robust than terminator-hunting.
Never pipeline across a STARTTLS boundary. RFC 3207 requires the server to discard any buffered plaintext received before STARTTLS — this is the entire defense against the STARTTLS command-injection class disclosed in 2021, where an attacker prepends plaintext commands that get "replayed" inside the encrypted session. Send STARTTLS, wait for 220, complete the handshake, then send a fresh EHLO and only begin pipelining inside TLS. If you are the receiver, verify your implementation actually flushes the pre-TLS buffer.
Checklist
Reuse one warm TCP+TLS connection for the entire queue; never one connection per message.
Parse EHLO capabilities every connection and branch on CHUNKING / BINARYMIME / PIPELINING / SIZE.
Batch MAIL + RCPT + BDAT <n> LAST (or MAIL + RCPT + DATA on fallback) into a single write().
Set TCP_NODELAY to kill Nagle + delayed-ACK stalls between flighted packets.
Declare SIZE=<n> on every MAIL FROM so oversized messages die at the envelope with 552.
Stream large bodies as 64 KB BDAT chunks; close with BDAT 0 LAST if already fully sent.
Prefer 8BITMIME; only use BINARYMIME when you control both ends of the path.
Keep a pipelined-DATA fallback with correct dot-stuffing for the Postfix-shaped majority.
Read and match every reply in order, even the ones you didn't block on — a rejected RCPT changes what you committed.
Never carry buffered commands across STARTTLS; re-EHLO inside the encrypted session.
Do this and a single connection at 80 ms RTT goes from ~3 msg/s to ~12.5 — roughly 4x with zero extra bandwidth, zero extra hardware, and nothing but correct use of extensions that shipped before most of your queue workers were written.
