How IMAP Works: Server-Side Email Access and Synchronization

IMAP (Internet Message Access Protocol) is the standard protocol for accessing and managing email messages stored on a remote mail server. Unlike POP3, which downloads messages to a single device and optionally deletes them from the server, IMAP treats the server as the authoritative store: messages remain on the server, organized into mailboxes, and can be accessed from multiple devices simultaneously. Defined in RFC 9051 (IMAP4rev2, published 2021, superseding RFC 3501's IMAP4rev1 from 2003), IMAP provides a rich command set for searching, filtering, flagging, and organizing email without downloading entire mailboxes. It is the protocol that makes modern multi-device email work.

IMAP vs POP3: Fundamental Differences

POP3 (Post Office Protocol, RFC 1939) was designed for an era when users checked email from a single computer on a dial-up connection. The model is simple: connect, download new messages, optionally delete them from the server, disconnect. This works for one device but breaks completely with multiple devices — a message downloaded to your phone is no longer available on your laptop.

IMAP inverts this model. The server is the source of truth. Clients synchronize with the server rather than downloading from it. This distinction has cascading implications:

The IMAP Connection Model

An IMAP session begins with a TCP connection to port 143 (cleartext) or port 993 (IMAPS, IMAP over TLS). In practice, all modern IMAP connections use TLS — either IMAPS on port 993 or STARTTLS upgrade on port 143. The session progresses through four states:

Not Authenticated — the initial state after TCP connection. The server sends a greeting (* OK for cleartext, * PREAUTH if already authenticated via TLS client certificate). The client typically issues CAPABILITY to discover server features, then authenticates via LOGIN (plaintext username/password, only safe over TLS) or AUTHENTICATE (SASL mechanisms like PLAIN, XOAUTH2, SCRAM-SHA-256).

Authenticated — the client can manage mailboxes (LIST, CREATE, DELETE, RENAME, SUBSCRIBE) but cannot access messages. The client must SELECT a mailbox to enter the Selected state.

Selected — a specific mailbox is open. The client can now access messages: FETCH, STORE (modify flags), SEARCH, COPY, MOVE, EXPUNGE (permanently delete flagged messages). Selecting a different mailbox implicitly closes the current one.

Logout — the session is terminating. The server sends a BYE response and closes the connection.

Every IMAP command is prefixed with a client-generated tag (e.g., a001, a002), and the server's completion response uses the same tag. This allows multiple commands to be in flight simultaneously (command pipelining), with the server responding out of order.

The Mailbox Model

IMAP organizes messages into mailboxes (commonly called folders in email clients). Every IMAP server must support at least the INBOX mailbox. Beyond that, the mailbox namespace is hierarchical, with a server-defined delimiter (usually / or .):

a001 LIST "" "*"
* LIST (\HasChildren) "/" "INBOX"
* LIST (\HasNoChildren) "/" "INBOX/Work"
* LIST (\HasNoChildren) "/" "INBOX/Personal"
* LIST (\HasNoChildren \Drafts) "/" "Drafts"
* LIST (\HasNoChildren \Sent) "/" "Sent"
* LIST (\HasNoChildren \Trash) "/" "Trash"
* LIST (\HasNoChildren \Junk) "/" "Junk"
* LIST (\HasNoChildren \Archive) "/" "Archive"
a001 OK LIST completed

The special-use mailbox attributes (\Drafts, \Sent, \Trash, \Junk, \Archive) are defined in RFC 6154. They allow email clients to discover the purpose of mailboxes without relying on name conventions (which vary by language and server).

When a client SELECTs a mailbox, the server returns metadata about the mailbox:

a002 SELECT INBOX
* 172 EXISTS         -- 172 messages in the mailbox
* 1 RECENT           -- 1 message since last SELECT
* OK [UNSEEN 168]    -- first unseen message is sequence number 168
* OK [UIDVALIDITY 3857529045]  -- UID validity value
* OK [UIDNEXT 4392]  -- next UID to be assigned
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded \*)]
a002 OK [READ-WRITE] SELECT completed

Two critical concepts here are sequence numbers and UIDs. Sequence numbers are positional (1 through N for N messages) and change when messages are expunged. UIDs are persistent identifiers assigned by the server that never change for a given message within a UIDVALIDITY epoch. Clients should use UIDs for bookmarking messages because sequence numbers are unstable. If UIDVALIDITY changes (due to mailbox reconstruction), all cached UIDs are invalid and the client must re-synchronize from scratch.

FETCH: Retrieving Message Data

The FETCH command is IMAP's workhorse. It retrieves specified data items for specified messages. IMAP's power lies in its ability to fetch selective parts of a message without downloading the entire thing:

# Fetch envelope (From, To, Subject, Date) for messages 1-10
a003 FETCH 1:10 (ENVELOPE)

# Fetch headers only (no body)
a004 FETCH 42 (BODY[HEADER])

# Fetch specific headers
a005 FETCH 42 (BODY[HEADER.FIELDS (From To Subject Date)])

# Fetch the plain text body part of a MIME message
a006 FETCH 42 (BODY[1.1])

# Fetch entire message (RFC822 format)
a007 FETCH 42 (BODY[])

# Fetch flags and internal date
a008 FETCH 42 (FLAGS INTERNALDATE)

# Fetch body structure (MIME tree) without downloading content
a009 FETCH 42 (BODYSTRUCTURE)

The BODYSTRUCTURE response describes the MIME structure of a message as a nested tree. For a multipart message with attachments, it might look like:

* 42 FETCH (BODYSTRUCTURE (
  (("text" "plain" ("charset" "utf-8") NIL NIL "quoted-printable" 1234 42 NIL NIL NIL NIL)
   ("text" "html" ("charset" "utf-8") NIL NIL "quoted-printable" 5678 89 NIL NIL NIL NIL)
   "alternative" ("boundary" "----=_Part_123") NIL NIL NIL)
  ("application" "pdf" ("name" "report.pdf") NIL NIL "base64" 98765 NIL ("attachment" ("filename" "report.pdf")) NIL NIL)
  "mixed" ("boundary" "----=_Part_456") NIL NIL NIL))

This tells the client: the message has two parts. Part 1 is a multipart/alternative with a plain text version (part 1.1) and an HTML version (part 1.2). Part 2 is a PDF attachment (98765 bytes, base64-encoded). The client can display the HTML body by fetching only BODY[1.2] — no need to download the 98KB PDF attachment.

The BODY.PEEK[] variant fetches data without setting the \Seen flag. This is important for "preview" functionality where the client shows a snippet without marking the message as read.

STORE and Flags

IMAP messages have flags that track their state. The standard system flags are:

• \Seen — the message has been read
• \Answered — the message has been replied to
• \Flagged — the message is flagged/starred for attention
• \Deleted — the message is marked for deletion (not yet permanently removed)
• \Draft — the message is a draft

Servers can also support custom keyword flags (like $Forwarded, $Junk, $NotJunk, or any custom label). The STORE command modifies flags:

# Mark message 42 as read
a010 STORE 42 +FLAGS (\Seen)

# Mark messages 1-10 as flagged
a011 STORE 1:10 +FLAGS (\Flagged)

# Remove the \Answered flag
a012 STORE 42 -FLAGS (\Answered)

# Set flags to exactly these values (replacing all existing flags)
a013 STORE 42 FLAGS (\Seen \Answered)

IMAP's deletion model is two-phase: first, mark messages with \Deleted using STORE, then permanently remove them with EXPUNGE. This allows undoing deletions (by removing the \Deleted flag) before EXPUNGE is called. Modern servers often support the MOVE extension (RFC 6851), which atomically moves messages to another mailbox — typically the Trash folder.

SEARCH: Server-Side Filtering

The SEARCH command finds messages matching specified criteria without downloading them. This is critical for performance — a mailbox with 50,000 messages cannot be searched by downloading all of them:

# Search for unread messages from a specific sender
a014 SEARCH UNSEEN FROM "[email protected]"

# Search by date range
a015 SEARCH SINCE "01-Apr-2026" BEFORE "25-Apr-2026"

# Complex boolean search
a016 SEARCH OR (FROM "noc@" SUBJECT "BGP") (FLAGGED UNSEEN)

# Full-text body search
a017 SEARCH BODY "route hijack"

# Search by message size
a018 SEARCH LARGER 1000000

# Search using UIDs
a019 UID SEARCH UNSEEN SINCE "20-Apr-2026"

The server returns a list of sequence numbers (or UIDs for UID SEARCH) matching the criteria:

* SEARCH 3 5 12 42 168 170 171
a014 OK SEARCH completed

SEARCH criteria can be combined (implicit AND) or joined with OR. Available criteria include: ALL, ANSWERED, BCC, BEFORE, BODY, CC, DELETED, DRAFT, FLAGGED, FROM, HEADER, KEYWORD, LARGER, NEW, NOT, OLD, ON, OR, RECENT, SEEN, SENTBEFORE, SENTON, SENTSINCE, SINCE, SMALLER, SUBJECT, TEXT, TO, UID, UNANSWERED, UNDELETED, UNDRAFT, UNFLAGGED, UNKEYWORD, UNSEEN.

The ESEARCH extension (RFC 4731) provides more efficient search results by returning only the minimum information needed: MIN (smallest matching UID), MAX (largest), COUNT (total matches), or ALL (complete list in a compressed format).

IDLE: Push Notifications

Without the IDLE extension (RFC 2177), an IMAP client must poll the server periodically to check for new messages. IDLE eliminates polling by allowing the server to push unsolicited notifications to the client when new messages arrive, flags change, or messages are expunged:

a020 IDLE
+ idling
* 173 EXISTS         -- new message arrived
* 42 FETCH (FLAGS (\Seen \Answered))  -- flags changed on message 42
DONE                  -- client sends DONE to stop IDLE
a020 OK IDLE terminated

IDLE is essential for real-time email delivery. Without it, email clients must poll every 1-5 minutes, introducing delays of up to 5 minutes between message arrival and notification. With IDLE, notification is effectively instantaneous.

However, IDLE only works for the currently selected mailbox. If the user has Inbox, Sent, and three other folders, the client only receives push notifications for whichever one is currently SELECTed. Monitoring multiple mailboxes requires either multiple IMAP connections (one per mailbox, each in IDLE state) or periodic polling of non-selected mailboxes. The NOTIFY extension (RFC 5465) addresses this by allowing clients to register interest in events across multiple mailboxes on a single connection, but NOTIFY support is not universal.

IDLE connections have a practical limitation: network intermediaries (NATs, firewalls, load balancers) often close idle TCP connections after 5-30 minutes. IMAP clients must periodically break out of IDLE (send DONE), issue a NOOP command (which elicits an OK response, keeping the connection alive), and re-enter IDLE. The recommended interval is every 29 minutes (RFC 2177).

CONDSTORE and QRESYNC: Efficient Synchronization

The hardest problem in IMAP client development is efficient synchronization: how does a client with a local cache of 50,000 messages determine what changed since the last sync without re-downloading everything?

The CONDSTORE extension (RFC 7162) introduces modification sequences (mod-sequences): monotonically increasing 64-bit integers that the server assigns to every mailbox modification. Each message has a mod-sequence, and the mailbox has a HIGHESTMODSEQ value. On reconnection, the client says "give me everything that changed since mod-sequence X":

# Fetch flags for messages changed since mod-sequence 12345
a021 FETCH 1:* (FLAGS) (CHANGEDSINCE 12345)
* 42 FETCH (FLAGS (\Seen \Answered) MODSEQ (12350))
* 170 FETCH (FLAGS (\Seen) MODSEQ (12348))
a021 OK FETCH completed

CONDSTORE only tracks flag changes. It does not handle message expunges — the client cannot determine which messages were deleted since the last sync. The QRESYNC extension (RFC 7162, same document) fills this gap. QRESYNC allows the client to select a mailbox with a known UIDVALIDITY and HIGHESTMODSEQ, and the server responds with both the changed messages and the expunged UIDs:

# Select with QRESYNC: "I last synced at modseq 12345, UIDs were valid"
a022 SELECT INBOX (QRESYNC (3857529045 12345))
* OK [HIGHESTMODSEQ 12360]
* VANISHED (EARLIER) 56,78:80     -- UIDs 56, 78, 79, 80 were expunged
* 42 FETCH (FLAGS (\Seen \Answered) MODSEQ (12350))
* 170 FETCH (FLAGS (\Seen) MODSEQ (12348))
a022 OK [READ-WRITE] SELECT completed

The VANISHED (EARLIER) response reports UIDs that were expunged since the client's last known state. Without QRESYNC, the client must compare its entire local UID list against the server's — a process that requires fetching all UIDs and diffing, which is slow for large mailboxes.

IMAP AUTHENTICATE and OAuth

The LOGIN command sends username and password in cleartext (which is safe over TLS but disallowed on plaintext connections). The AUTHENTICATE command supports SASL (Simple Authentication and Security Layer, RFC 4422) mechanisms that provide more sophisticated authentication:

• PLAIN — base64-encoded username and password (essentially the same as LOGIN but wrapped in SASL)
• XOAUTH2 — Google's OAuth 2.0 mechanism for IMAP. The client sends a base64-encoded token containing the user's email address and an OAuth access token. This allows apps to access email without handling user passwords. Microsoft uses a similar mechanism (XOAUTH2) for Office 365 IMAP.
• OAUTHBEARER — the standardized OAuth mechanism (RFC 7628), replacing vendor-specific XOAUTH2. Not yet universally supported.
• SCRAM-SHA-256 — challenge-response authentication that avoids sending the password (even encrypted) over the wire. Resistant to replay attacks and server impersonation.

OAuth is increasingly important as major providers (Google, Microsoft) deprecate "less secure apps" (password-based IMAP authentication) in favor of OAuth-based access. This shift requires IMAP clients to implement a full OAuth flow: redirect the user to the provider's consent screen, receive an authorization code, exchange it for access and refresh tokens, and periodically refresh the access token.

IMAP Extensions

IMAP's extensibility is one of its strengths. Extensions are negotiated via the CAPABILITY command, and the server advertises supported extensions. Key extensions beyond CONDSTORE and QRESYNC include:

• MOVE (RFC 6851) — atomic MOVE command instead of COPY + STORE \Deleted + EXPUNGE
• SPECIAL-USE (RFC 6154) — standardized mailbox roles (\Sent, \Trash, \Drafts, etc.)
• COMPRESS=DEFLATE (RFC 4978) — zlib compression on the IMAP connection, reducing bandwidth by 60-80% for text-heavy email
• LITERAL+ (RFC 7888) — non-synchronizing literals, eliminating a round-trip when uploading messages (APPEND)
• NAMESPACE (RFC 2342) — discovery of personal, shared, and public mailbox namespaces
• ID (RFC 2971) — client and server exchange software identification (name, version), useful for debugging
• OBJECTID (RFC 8474) — unique, persistent identifiers for mailboxes, messages, and threads that survive renames and moves
• SORT and THREAD (RFC 5256) — server-side sorting and threading of messages, avoiding the need to download all messages for client-side sorting
• SNIPPET (RFC 8970) — server-generated message preview text, eliminating the need to download body content for previews in message lists

IMAPS and Transport Security

IMAP traffic includes authentication credentials and email content — both highly sensitive. Two TLS deployment models exist:

IMAPS (port 993) wraps the entire IMAP connection in TLS from the first byte. This is analogous to HTTPS: the TLS handshake occurs before any IMAP protocol data is exchanged. This is the recommended approach and the default for all modern email clients.

STARTTLS (port 143) starts as a plaintext IMAP connection, then upgrades to TLS via the STARTTLS command. The client connects on port 143, discovers STARTTLS support via CAPABILITY, issues STARTTLS, and the connection switches to TLS. The problem: a man-in-the-middle can strip the STARTTLS capability from the CAPABILITY response, tricking the client into continuing in plaintext. This is called a STARTTLS stripping attack. To prevent it, clients should refuse to authenticate on plaintext connections.

Email security extends beyond the IMAP connection itself. Messages may have been delivered to the server via SMTP with or without TLS (opportunistic TLS). The message content may include DKIM signatures that the client can verify, though few IMAP clients do so. End-to-end encryption (S/MIME, PGP) operates at the message layer, independent of IMAP transport security.

JMAP: The Modern Alternative

JMAP (JSON Meta Application Protocol, RFC 8620/8621) is a modern protocol designed as a successor to IMAP. Where IMAP was designed for interactive terminal sessions over slow links (its command syntax dates to 1986's original ISTRSTRSTRSTRSTR RFC 1langford protocol), JMAP is built for modern client applications:

• JSON over HTTP — JMAP uses standard HTTP POST with JSON request and response bodies. No custom binary protocol, no stateful TCP sessions. Works through any HTTP proxy and CDN.
• Batched operations — a single JMAP request can contain multiple method calls with explicit data dependencies between them, eliminating the round-trip problem of serial IMAP commands.
• Efficient sync — JMAP has built-in change tracking (similar to CONDSTORE/QRESYNC) as a core protocol feature, not an optional extension.
• Push via EventSource — JMAP uses Server-Sent Events for real-time notifications instead of IMAP's IDLE command.

Despite its technical advantages, JMAP adoption has been slow. Gmail and Outlook do not support it. Fastmail (a key contributor to the JMAP specification) is the most prominent JMAP provider. For the foreseeable future, IMAP remains the universal protocol for email access.

IMAP at Scale

Running an IMAP server at scale presents significant challenges. Each user's mailbox may contain hundreds of thousands of messages spanning decades. IMAP's stateful connection model means the server must maintain per-connection state (selected mailbox, search results, pending IDLE notifications) for every connected client.

Large-scale IMAP deployments (Gmail, Outlook, Yahoo) typically use:

• Connection proxies (like NGINX's mail module or Perdition) that terminate TLS, authenticate users, and route connections to the correct backend server. The proxy multiplexes thousands of client connections onto fewer backend connections.
• Distributed storage — messages stored in object stores or distributed filesystems, with metadata and indexes in databases. Cyrus IMAPd supports murder (partitioned IMAP namespace across servers). Dovecot supports director (user-level proxying with consistent hashing).
• Full-text search indexes — IMAP SEARCH with BODY/TEXT criteria requires full-text search. At scale, this demands dedicated search infrastructure (Solr, Elasticsearch, Xapian) integrated with the IMAP server.

Summary

IMAP is a remarkably capable protocol that has evolved significantly from its origins. Its client-server synchronization model, with messages stored authoritatively on the server, enables seamless multi-device access. The FETCH command's ability to retrieve selective message parts minimizes bandwidth. IDLE provides push notifications for the selected mailbox. CONDSTORE and QRESYNC enable efficient incremental synchronization that transforms reconnection from an O(n) operation to an O(changes) operation.

The protocol's complexity is a direct reflection of the email problem's complexity: MIME message structure, mailbox hierarchies, concurrent access from multiple clients, and the need for both real-time notification and efficient batch synchronization. Modern alternatives like JMAP offer cleaner semantics, but IMAP's universal support makes it the protocol that email clients must implement for broad compatibility.

Email delivery via SMTP, authentication via DKIM, SPF, and DMARC, and transport security via TLS are all interconnected pieces of the email ecosystem. To explore the network infrastructure that carries this traffic, try the god.ad BGP Looking Glass to look up the routes and autonomous systems behind any mail server's IP address.