SMTP and the Evolution of Email

We worked with our friends at Windows IT Pro to take an in-depth look at the history of Simple Mail Transfer Protocol (SMTP). The following is a guest post from their team that covers the origins of SMTP, the evolution of email, and what SMTP does, and doesn’t do.

SMTP Defined

The Simple Mail Transfer Protocol (SMTP) has a name that perfectly describes its origin and intended purpose. SMTP was built to be a lightweight, easy-to-implement means to transfer plain-text messages between devices. The original SMTP standard distinguished between a mail transport agent (MTA)—what modern administrators would call a server—and mail user agents (MUA), which most people refer to as clients.

SMTP’s goal was to provide a way for two MTAs to pass message traffic back and forth. The protocol doesn’t define any standards for storing messages, giving clients access to them, or carrying complex content types such as audio or image data. It’s just a set of rules for moving a properly structured message from point A to point B.

ARPANET

The first variant of SMTP was designed in a very different environment than what we face today. The original Internet grew from ARPANET, a closed research network that was designed for academic and government use. ARPANET and the early implementation of the Internet were designed around the assumption that only a limited number of trusted users would be able to connect to the network, and that commercial traffic wouldn’t be allowed. The competing X.400 protocol was intended to be a more robust message transfer protocol that would be suitable for commercial use; X.400 included features for guaranteed reliable delivery, message tracking, and other security and authentication features—but it was complex to implement, and it ultimately lost out to the simpler, less complex, less expensive SMTP implementations from early vendors.

The version of SMTP that we use today still shows a number of design influences that trace back to its origins:

• The protocol itself didn’t provide for authentication or encryption, which weren’t especially important on a trusted network with trustworthy users. (Since SMTP’s appearance, a series of extensions to the standard have added these features.)
• The ability to get return receipts or delivery notifications is dependent on which client recipients use; servers are required to send delivery status notifications (DSNs) only for messages that are permanently undeliverable.
• There’s no enforced mechanism for classes of service; for example, FedEx and UPS both let you choose from multiple delivery speeds according to how much you’re willing to pay.
• Email filtering often seems capricious, and users have learned through experience that important messages may be filtered arbitrarily by the spam filters in their client, in their server, or somewhere in the cloud. (And now of course, recipients can filter messages within their own inboxes through features like Gmail’s “tabs”.)
• SMTP (or email in general) often falls back to using web-based technologies (i.e. img and anchor tags) to archive click and open tracking metrics.

The Evolution of Email

As time passed and the Internet spread, businesses were allowed to connect to it but were informally banned from using it for commercial purposes. A few large companies (notably IBM, DEC, and Control Data) built corporate email systems for large-company use, and as time passed, smaller companies—including Novell, Lotus (now IBM Lotus), and Microsoft—saw the business value in email systems and began building and delivering intra-company systems that broadened the email market. None of these systems provided much in the way of interoperability, although after the SMTP standard was defined a few forward-looking vendors began adding SMTP support through gateways or connectors that linked MTAs from different vendors.

The difficulty of getting interoperability to reliably work across multiple MTA vendors and their individual connectors put a lot of market pressure on vendors to come up with a simpler, more robust solution, so native SMTP began to gather real momentum in the mid-1990s. Now, it is the de facto standard for transferring mail between servers; even products that once used proprietary protocols, such as Microsoft Exchange, rely on SMTP both for intra- and inter-organization message transport.

What SMTP Does, and Doesn’t Do

As the “T” in its name indicates, SMTP is a transport protocol: All it does is move messages from point A to point B. Like many other Internet protocols, SMTP is intended to be used on top of the Transmission Control Protocol (TCP), which guarantees reliable delivery of the individual packets that make up a conversation.

The use of IP means that we can depend on SMTP to eventually get the message contents to a server, but what happens to it after that is up to the server. Imagine what happens when a delivery driver leaves a package at your office on a Saturday. What happens to it after delivery isn’t his problem. Deliverability, monitoring, tracking, authentication, and encryption are all examples of services that SMTP doesn’t necessarily provide itself but that are still very valuable.

For more information on the delivery side of things, stay tuned to the blog for further posts!

---

This post is courtesy of our friends at Windows IT Pro.