The 10-minute window
03:14. A login lands on submission port 587 for [email protected]. The password is correct — it was dumped in a combolist eight months ago and nobody rotated it. The source IP geolocates to Lagos; the account's owner is asleep in Munich. Nothing about this login looks malformed. Dovecot authenticates it, Postfix accepts the SASL handshake, and the session sits quiet for ninety seconds while the attacker's tooling warms up.
03:16. The first burst goes out — 40 messages to 40 unique domains, then another 40. By 03:24 the account has relayed roughly 900 messages through your egress IP. Every one carries a valid DKIM signature, because they left through *your* authenticated submission path. At the SMTP layer they are indistinguishable from legitimate mail.
03:41. Spamhaus CSS lists your outbound IP. Now every other tenant sharing that egress — dozens of innocent mailboxes — starts getting deferred by Gmail and rejected outright by a few enterprise gateways. Your abuse desk finds out when a customer opens a ticket asking why their invoices bounced. The delisting request you file at 09:00 takes most of the day, and it only sticks because you can prove the source was contained.
The whole event that mattered fit inside ten minutes. No on-call rotation responds in ten minutes to a signal nobody was watching. But two signals sat in your logs the entire time: where the login came from, and how fast the account started sending. Neither needs ML, a SIEM licence, or anything you aren't already collecting. Here is how you wire them to an automatic, reversible lock that fires before the RBLs do.
Two signals worth trusting
You could score a dozen features. Most cost more to compute than they're worth and throw false positives that train your operators to ignore alerts. Two signals carry almost all the value.
Impossible travel (geo-velocity). Take two consecutive *successful* logins for one account. Compute the great-circle (haversine) distance between their IPs and divide by the time delta. If the implied speed exceeds a commercial-jet ceiling of ~900 km/h, the same human did not physically produce both logins. Istanbul at 03:02 and Lagos at 03:11 implies ~35,000 km/h — that's not a traveller, that's a stolen credential.
Send-rate breakout. Track a rolling per-account send count against the account's *own* baseline, not a global limit. A newsletter mailbox that legitimately pushes 200/hour shouldn't share a threshold with a personal inbox that sends 15 a day. The spike that matters is relative: >5× the account's 14-day median, or a hard floor like >30 distinct recipients in 60 seconds.
The traps are real, and you design around them or you drown:
- Mobile CGNAT rotates IPs constantly inside one carrier. Distance jumps, but the ASN stays constant — ignore it.
- Corporate VPNs, iCloud Private Relay, shared office egress all move a user's apparent location without anything being wrong.
- Real travellers exist. A single new country is *not* a compromise signal on its own.
The discriminator that cuts through nearly all of this: country AND ASN changing together, at an implied speed over the ceiling. Same ASN across two logins → almost always benign churn, drop it. New ASN in a new country at impossible speed → hard flag. And never auto-lock on geography *alone*: require the send-rate signal to co-fire. One signal alerts; two signals lock.
Wiring the log sources
The raw events are already being written. On Dovecot, successful authentications land in the auth log with the remote IP:
dovecot: imap-login: Login: user=<[email protected]>, method=PLAIN, rip=102.89.34.12, lip=..., TLS
dovecot: submission-login: Login: user=<[email protected]>, method=PLAIN, rip=102.89.34.12Postfix records the SASL username and client on smtpd, and submission (587) is where hijacked accounts almost always relay from:
postfix/submission/smtpd: NOQUEUE: client=unknown[102.89.34.12], sasl_method=PLAIN, [email protected]
postfix/submission/smtpd: A1B2C3: client=unknown[102.89.34.12], [email protected]You have three ways to consume these in real time. Cron-polling the log every minute is the wrong answer — a minute is six full attack-bursts of latency. rsyslog `omprog` pipes each matching line to a long-running program, which is solid. The lowest-latency option, and the one I run, is a small daemon reading a named pipe that rsyslog writes to, so events arrive the instant they're logged. Dovecot also exposes an auth-policy webhook and a push-notification plugin if you'd rather receive structured JSON than parse text — worth it on a greenfield build, but log-tailing works against any existing stack without touching Dovecot config.
Parse each line into a normalised event: {account, ip, port, event_type, ts}. Feed successful logins to the geo scorer and accepted submissions to the rate scorer.
Scoring geography: GeoIP + ASN
Use MaxMind's free GeoLite2-City and GeoLite2-ASN databases, refreshed weekly with geoipupdate — a stale GeoIP DB is a false-positive factory. Look them up with the Python geoip2 reader and compute velocity:
import geoip2.database, math
city = geoip2.database.Reader("/var/lib/GeoIP/GeoLite2-City.mmdb")
asn = geoip2.database.Reader("/var/lib/GeoIP/GeoLite2-ASN.mmdb")
def haversine_km(a, b):
r = 6371.0
dlat = math.radians(b[0]-a[0]); dlon = math.radians(b[1]-a[1])
x = (math.sin(dlat/2)**2
+ math.cos(math.radians(a[0]))*math.cos(math.radians(b[0]))
* math.sin(dlon/2)**2)
return 2*r*math.asin(math.sqrt(x))
def geo_score(prev, curr): # each: {ip, lat, lon, asn, country, ts}
if prev["asn"] == curr["asn"]:
return 0 # CGNAT / mobile churn — ignore
dist = haversine_km((prev["lat"], prev["lon"]), (curr["lat"], curr["lon"]))
hours = max((curr["ts"] - prev["ts"]) / 3600.0, 1/3600.0)
speed = dist / hours # km/h
if curr["country"] != prev["country"] and speed > 900:
return 100 # hard flag: physically impossible
if curr["country"] != prev["country"]:
return 40 # soft: new country, plausible speed
return 10 # same country, new ASNNote the shape of the rules. Same ASN short-circuits to zero — that one line kills the overwhelming majority of mobile false positives. A country and ASN change at >900 km/h is the only thing that scores 100. Everything else is a *soft* score that feeds the combiner but never locks on its own. You want a number, not a boolean, so you can tune the lock threshold without rewriting logic.
Scoring send-rate: rolling counters in Redis
Track each account's outbound in a Redis sorted set keyed on send timestamp, then count the recent window. A sorted set gives you exact sliding windows for free:
# On each accepted submission for the account:
redis-cli ZADD acct:1042:sends 1751600000 "msgid-abc"
# Recipients in the last 60 seconds:
redis-cli ZCOUNT acct:1042:sends 1751599940 +inf
# Trim old entries so the set doesn't grow unbounded:
redis-cli ZREMRANGEBYSCORE acct:1042:sends -inf 1751596400The threshold is where naive rate limits go wrong. Don't use one global number. Maintain an EWMA of each account's own history — its typical hourly volume — and flag on a multiple of that. A practical starting rule set:
- Hard floor: >30 distinct recipients in 60 seconds, regardless of baseline.
- Relative: >5× the account's 14-day median hourly volume.
- Distinct-domain count matters more than raw recipient count. 200 messages to one mailing list is a newsletter; 200 messages to 190 different domains is a spam run. Weight unique recipient domains heavily.
Emit a rate_score on the same 0–100 scale as the geo scorer, then combine. My combiner locks when geo_score >= 100 AND rate_score >= 60, and alerts when either crosses on its own. Requiring both signals is what makes automatic action safe: a phished account that logs in from Lagos *and* immediately fans out 400 messages is not a false positive, and demanding both is what lets you flip the switch with no human in the loop.
The lock: reversible and immediate
Two things must happen together, and one of them is easy to forget.
Postfix — refuse new sends. Add the account to a sender-access map and rebuild it:
echo "[email protected] REJECT account temporarily locked (security)" \
>> /etc/postfix/sender_lock
postmap /etc/postfix/sender_lockReference the map with check_sender_access hash:/etc/postfix/sender_lock in smtpd_sender_restrictions, so any authenticated attempt to send *from* that address is rejected — at RCPT time, with the default smtpd_delay_reject = yes. No reload needed: postmap regenerates the .db and Postfix picks it up on the next lookup.
Dovecot — refuse new auth AND kill the live session. This is the trap: flipping the passdb password field stops the *next* login but does nothing to a session already established. The attacker's IMAP IDLE connection stays open and keeps relaying. You must actively terminate it:
doveadm who [email protected] # list active sessions/PIDs
doveadm kick [email protected] # terminate every live sessionSet your disabled state (a passdb nologin extra field, an account attribute, or moving the row to a locked state in your userdb) *and* run doveadm kick in the same operation. Then drop the authenticated smtpd connection so the current submission session dies too.
Persist the lock in your database. If lock state lives only in a Postfix hash and a Dovecot flag, a service restart or a config-management run can silently reset it and the account quietly reopens. Write an audit row — account, trigger reason, both scores, the geo/ASN detail, timestamp, and later the unlock method — and treat the DB as the source of truth that reconciles the Postfix map and Dovecot flag on boot.
Self-service unlock without a ticket
Locking a real user out with no path back turns a security win into a support-queue disaster. The moment the lock fires, send a templated notice to the account's recovery address (and optional SMS): *"We locked [email protected] after a login from Nigeria (AS29465) that didn't match your usual pattern, plus an unusual send burst."* Be specific about the country and ASN — it reads as competence, not noise.
The unlock path is a password reset plus a fresh 2FA challenge. Once the user completes both, automatically clear the Postfix map entry (postmap again), lift the Dovecot flag, and write the unlock to the audit table. A genuinely compromised user resets and moves on in two minutes; an attacker who controls neither the recovery address nor the 2FA device is simply stuck. Every lock and unlock is logged, so you can watch the ratio and prove containment if a delisting desk asks.
The payoff is pure reputation math: one locked mailbox versus a Spamhaus CSS listing on shared egress that degrades delivery for every co-tenant and costs you a manual, hours-long delisting. Locking pre-emptively is always the cheaper trade.
Tuning without alert fatigue
- Baseline for two weeks before enforcing. Collect scores, lock nothing.
- Start alert-only. Watch what *would* have locked; fix the false positives first.
- Allowlist known ASNs — corporate VPNs, iCloud Private Relay, your office egress.
- Exempt verified bulk senders from the rate rule, or give them their own EWMA baseline.
- Require BOTH signals to auto-lock; a single signal alerts. No exceptions.
- Circuit breaker: cap auto-locks per hour. A sudden flood of locks means a bad GeoIP update or your own outage, not 50 simultaneous compromises — fail safe to alert-only.
- Monitor the lock-to-unlock ratio. A high unlock rate means thresholds are too tight and you're annoying real users.
- Refresh GeoIP weekly and test the `doveadm kick` path monthly — an unlock path that silently stopped working is worse than none.
Get this running and the 03:14 incident that opened this article ends at 03:17 instead of 09:00: two signals cross, both fire, the account locks, the live session dies, the recovery mail goes out — and your egress IP never makes it onto a blocklist in the first place.