TRENDS IN ELECTRONIC MAIL SYSTEMS AND THEIR LIKELY EFFECTS ON THE QUALITY OF NETWORKED-BASED COMMUNICATIONS

TRENDS IN ELECTRONIC MAIL SYSTEMS AND THEIR LIKELY EFFECTS ON THE QUALITY OF NETWORKED-BASED COMMUNICATIONS

Christinger Tomer

School of Library and Information Science
University of Pittsburgh
Pittsburgh, PA 15260, USA

E-mail: ct@coltrane.lis.pitt.edu

Electronic mail is growing at a remarkable rate. Today, there are probably about thirty million electronic mail users worldwide. The Electronic Mail Association (EMA) predicts that the number of electronic mail users will grow at an annual rate of twenty to thirty percent, with the number of messages sent by the average user increasing at an annual rate of twenty-five percent, and the EMA foresees at least fifty million electronic mail users worldwide by 1995. Moreover, it is growing so rapidly that electronic mail is now viewed by many observers as an application of strategic impor-tance in education, research and development, and commerce.

As access to and the use of electronic mail increases, so, too, do the demands of an increas-ingly sophisticated cohort of users. At the heart of these demands are two main objectives. First, users want more flexibility in terms of the message formats to which they have ready access. They want "multimedia" mail. Besides being able to send ASCII text, they want to be able to transmit and receive documents generated by word processing in their native formats, as well as messages that include binary graphics, digitized facsimiles, and audio clips. (It has been suggested that as the evolution of the electronic work group will depend largely on developing more subtle, more sophis-ticated forms of electronic communication, and ones that make access to information more "human-like." According to at least one commentator, the relationship of the computer "to the user [is changing] from that of an isolated productivity tool to that of an active collaborator in the acquisition, use and creation of information, as well as a facilitator of human interaction" (Tesler, 1991). Elec-tronic messaging systems capable of transporting complex documents are often cited as necessary condition for such developments.)

In addition to being able to send and receive complex documents, users want the ability to store and forward complex messages on a global basis, i.e., sending messages among the greatest number of different electronic mail systems. It is no longer sufficient to be to send messages to or receive messages from other users on the same LAN. Users want to send and receive mail over wide-area networks; more specifically, they want transparent gateway services that make it simple to exchange messages with remote systems, even if the remote system's messaging capabilities are otherwise incompatible.

However, today's electronic mail systems are, as an editorial in PC Week noted, "somewhat dysfunctional" (E-mail standards..., 1993). The principle reason for this statement is that the universal communications infra-structure necessary to make electronic messaging easy and painless is absent. Yet, there is now the distinct possibility that such an infrastructure will soon be in place, and that users will have access to implementation-independent, multimedia messaging on a global scale. But the process of creating such a system will be neither simple nor untroubled. The communities of interest that are working to broaden the scope and quality of electronic mail services are divided in many ways. Some divisions arise amid matters of philosophy and technique, other differences are born of the need to make money. It will be some time before any or all of these differences is resolved satisfactorily.

Perhaps the best means of understanding the efforts to enhance the quality and functionality of electronic mail is to examine the competing standards that are intended to furnish the conceptual framework for so-called "multimedia" mail and produce levels of interoperability approaching universality. Today, there are two competing "standards" for the transmission of electronic mail over wide-area networks. One is the Consultative Committee for International and Telephone and Telegraph (CCITT)'s X.400, which is a series of recommendations for the Open System Inter-connection (OSI) Message Handling System (MHS). The other "standard" is a suite of protocols and extensions that provide the operational basis for the exchange of electronic mail over the Inter-net. The most recent addition to this suite is the Multipurpose Internet Mail Extensions (MIME), which extends the capabilities of Internet mail into the realm of complex documents. Of the two sets of standards, the protocols for Internet mail are probably more important at this point, if only because it is the basis for the vast majority of electronic mail exchanges transacted over wide-area networks. The long-term importance of X.400 will probably rely to a significant degree on the success of OSI, but also on the need for meta-level messaging services, where meta-level services refer in particular to X.400's double-enveloping capabilities, which allows a message generated by one system at the local to be readdressed for the purposes of routing and/or delivery, and its comparatively more extensible addressing scheme.

This paper examines these competing standards and consider how the changes mapped out in these standards will affect the functionality and quality of electronic mail.

Any general discussion of electronic mail must focus to a significant degree on the standards for electronic mail on the Internet, inasmuch as Internet currently supports as many as twenty-five million users, the vast majority of whom use electronic mail regularly. To wit, Internet mail is currently based on two standards established in 1982, RFC 821, the Simple Mail Transfer Protocol and RFC 822, the Standard for the Format of ARPA Internet Text Messages (Crocker, 1982). These standards limited the transmission of electronic mail to messages that:

Under SMTP a mail request entails the establishment of a two-way communication channel between the sender and the receiver, where the receiver may be the ultimate destination or an intermediate recipient. An SMTP server listens on TCP port 25 for a client connection. When a client and server have exchanged "greetings," essentially an exchange of information concerning the server's mail transfer agent, the client initiates one or more SMTP transactions. When the client is finished delivering its messages, the client and server exchange another set of signals, in this instance releasing the transfer agent from service and closing the TCP connection.

The Multi-purpose Internet Mail Extensions are a freely available set of specifications offering a means of interchanging text in languages with different character sets and multi-media mail among the many different computer systems that support the SMTP standard. To be even more specific, MIME represents a way of encoding, in ASCII, a multi-part message structure. A MIME body part is a sequence of bytes with an associated content-type and content-transfer-encoding. The parts may be nested and may be of seven different types -- Text, Audio, Image, Video, Message, Application and Multi-part (nested). The default MIME content-type is text/plain, i.e. a sequence of lines of text.

MIME has a number of other features, inlcuding provisions for encoding binary data in ASCII in a base 64 format similar to uuencode or btoa. In addition, it includes support for international character sets, tagging each part of a message with the character set in which it has been written and providing seven bit encoding of eight bit character sets, and it provides a simple "rich text" format for marking text. Finally, there is a mechanism for splitting messages into multiple parts and reassembling them at the receiving end1 (Borenstein & Freed, 1992).

MIME builds on the established standards for Internet mail by adding new fields for message headers that describe types of content and organization for messages. As a result, Internet mail supported by compliant applications may now contain: (1) multiple objects in a single message; (2) text having unlimited line length or overall length; (3) character sets other than ASCII; (4) multi-font messages; (5) binary or application specific files; (6) images, audio, video and multi-media messages; and (7) calls to external message body parts.

MIME is designed to be compatible with the older Internet mail standards. Nathaniel Boren-stein, the principal architect of MIME, has written:

MIME is an extensible mechanism. As a result, it is likely that the set of content-type/ subtype pairs and associated parameters will grow with time. Several other MIME fields, such as character set names, are also likely to have new values defined. To guarantee that the set of defined values develops in an orderly manner, MIME defines a registration process that uses the Internet Assigned Numbers Authority (IANA) as a central registry for such values.

Of the properties defined by the first version of MIME, the most important is the Content-Type header field, which specifies the type and subtype of data in the body of a message and the encoding of such data, where encoding has been used. The MIME standard defines seven content-types. Although the authors of the MIME standard say that the set of seven content types is "substantially complete," they expect additional supported types to be accommodated by creation of new subtypes of the seven initial top-level types.

MIME's content types include: (1) a Text Content-Type, which can be used to represent textual information in a number of character sets and formatted text description languages; (2) a Multi-part Content-Type, which can be used to combine several body parts, including parts consisting of different types of data, into a single message; (3) an Application Content-Type, which can be used to transmit application data or binary data; (4) a Message Content-Type, for encapsulating a mail message; (5) an Image Content-Type, for transmitting still image data; (6) an Audio Content-Type, for transmitting audio data; and (7) a Video Content-Type, for transmitting video or moving image data, possibly with audio as part of the composite video data format.

The Content-Type field describes the data contained in the body of the message in detail suffi-cient to enable the mail reader to select an appropriate mechanism for presentation of the data to the user, or to deal with the data in another, appropriate manner.2 Under MIME, parameters are modi-fiers of the content-subtype. Although most of MIME's defined parameters function only with certain content-types, some of them are "global" in the sense that they might apply to any subtype. For example, the boundary parameter, which is used to show how body parts are separated from one other, is valid only in a multi-part content-type, whereas the character set parameter is valid for several content-types. The syntax for the content type header field is:

For example, MIME defines four subtypes of the Multi-part Content-Type. The four subtypes are: (1) mixed (which indicates sequential processing); (2) parallel; (3) digest (which shows that each part is an encapsulated mail message); and (4) alternative (which indicates that although multi-ple body parts are present, each part has identical content, with the part selected for display by the recipient's user agent being a function of that applications interpretative capabilities).3

In addition, MIME defines a Content Type Message/External-Body, which is intended to indicate that the actual body of the message, the data itself, has not been included, but merely referenced. When a body or body part is external, the reference consists of a header, a blank line, and the message header for the encapsulated message. If another blank line appears, this ends the message header for the encapsulated message. Such a reference would look like this:

MIME is significant in at least three ways. First, it builds upon the single largest electronic mail system, Internet mail, in a simple, almost elegant manner, facilitating the transmission of complex documents without disrupting service to the installed base of users or requiring any major modifications at the systems level. Under MIME, many users of SMTP-based mail services will be able to send and receive complex documents while continuing to use familiar user agents such as Elm, Pine, or GNU Emacs' rmail facility. Second, MIME is capable of functional co-existence with both X.400 and at least some proprietary electronic mail systems, e.g., Microsoft Mail and Lotus's cc:Mail, which means that there should configurations under which clients effectively have the advantages of not one but a series of standards and system. Finally, there is the fact that MIME is an open standard -- essentially free and public -- that should engender rapid evolution and compa-ratively widespread implementation, and that it can be elaborated by public domain implementation and interfaces.

MIME's shortcomings seem "political," rather than technical. Thus far, commercial vendors seem much more interested in X.400. This is largely because as an ISO standard, X.400 is thought to afford commercial developers a better documented and more stable base on which to construct both clients and servers. Although a few commercial developers have promised to bring compliant clients to the marketplace, MIME suffers comparatively because none of the major developers have thus far judged it to be in their interests to promote SMTP or its extensions as a mail transport technology. There is, too, the matter of MIME being rooted in the Unix environment, where many major vendors have little or no foothold. By comparison X.400 has the acceptance of the standards communities, a variety of features of potential benefit to users, including the capacity to transport complex documents, substantial and growing deployment in most parts of the world, and the increasing interest of commercial vendors.

As noted previously, X.400 is a series of recommendations that specify how an electronic mail message may be transported from one system to another and how such messages are addressed.4 X.400 and its parallel protocol, the Open Systems Interconnection (OSI) Message Handling System (MHS), are the so-called "flagship protocols" in the OSI stack. Many observers view X.400 as a key step toward universal interoperability, where everyone with electronic mail capabilities can send and receive electronic messages as part of a worldwide electronic community. On the other hand, its many critics see X.400 as a system at once unnecessarily complex and underdeveloped. They fear that if X.400 enjoys widespread acceptance, it will stifle innovation and impede the progress of the networking community.

It is true that current implementations of the X.400 MHS tend to be comparatively too compli-cated, as well as often too demanding of system resources. They also tend to be unreliable; readers who follow postings to USENET groups such as comp.mail and comp.mail.misc will recall that during 1992 users of MCI Mail claimed bitterly about lost messages, both incoming and outgoing. Moreover, critics such as Marshall Rose have pronounced X.400 addressing "incomprehensible." Even users less critical than Rose find X.400 addressing, with its many variations on the keyword/ value pairs, four different allowable configurations, and no official textual syntax, to be demanding. Usually, X.400 will require more than sixty characters to address a message (Combs, 1992).

A more serious problem of X.400, given the needs and expectations of many users, is its failure to support the multimedia capabilities that its proponents often cite as one of X.400's most important advantages. According to Marshall Rose:

Rose is also critical of the fact that X.400 is not suited in its native form to providing trans-parent gateway services. He is also highly critical of the bulk-mode transfer facility, because the lack of address verification capabilities at this level means that a message transfer agent may accept delivery of a message that it cannot deliver at the user level. He suggests that it is more appropriate "to perform verification optionally followed by transfer."

• There is a distinct possibility that it will be enhanced to support the more general exchange of complex documents;

• X.400 addressing and related facilities may be used to transcend the numerical limits of the Internet addressing scheme;

• it can be bound closely to X.500 directory services (which should tend in the long term to minimize, if not eliminate, most addressing problems); and

• despite the aforementioned criticism concerning the absence of an explicit provision for gateway services, X.400 provides a basis for integrating incompatible mail systems, e.g., through the facilities of a X.400 mail server, users of IBM PROFS mail, Microsoft Mail, CompuServe Mail, MCI Mail, and Internet mail could exchange messages without loss of information, owing to X.400's ability to re-address messages via its double enveloping capability.

In operational terms, X.400 is an applications protocol that uses an end-to-end transport protocol to deliver messages. While it is intended primarily for electronic mail (or, in OSI parlance, "interpersonal messaging"), the protocol is also general enough to support the transport of digital facsimiles or act as a vehicle for electronic fund transfer and/or electronic document interchange. Even more important, X.400 supports the interchange of multimedia messages, meaning that, like MIME-compliant systems, X.400 systems can transfer graphics and other non-textual information.

Under X.400, an electronic mail system consists primarily of user agents, message transfer agents, and a message store. The user agent is the mail processor, responsible for message prepara-tion, submission, and reception, also providing text editing, presentation, security, message priority, and delivery notification services. The message transfer agent establishes the store-and-forward path, ensures channel security, and relays messages. A collection of message transfer agents is usually called the MTS, or message transfer system. Essentially a "database of messages," the message store is a server providing facilities for message storage, submission, and retrieval (Schnaidt, 1992).

At its most basic level, an X.400 message is a stream of octets. Viewed at another, higher level, an X.400 message consists of the message envelope, also known as the transfer envelope, followed by the contents of the message. The transfer envelope contains information used by the MHS to deliver the message. Typical fields include a unique message identification number for tracing the message; the address of the originator and the recipient of the message; the priority of the message; the type of contents of the message; and so-called "trace" information, which tracks the message within the MHS. In terms of message content, the content-type header defines each body part, followed by the number of body parts specified in the header. For example, the contents of a message may contain ASCII text followed by a chart in a binary graphical format, with the header of the message showing where each of these body parts is stored within the body of the message and how each part should be interpreted.

A main reason for the great interest in X.400 is the sophistication of the delivery services that can be supported by elements standardized in the header of an X.400 message. These delivery services include multi-destination distribution, delivery to an alternate recipient, redirection of incoming messages, grade of delivery selection, deferred delivery and related queuing service, security services, including proof of submission and proof of delivery, physical delivery services, and conversion services. Another distinguishing feature of X.400 is that it defines a standard addressing format known as Origin/Destination (O/R) addressing. An O/R address is a series of attribute/value pairs that provide a unique identity for the electronic location of the origin or des-tination of a message. For example the following is a stylized representation of an O/R address:

The addressing scheme is complicated and correct addresses can be difficult to construct, but the fact that the scheme provides for naming many attributes ranging from physical delivery service name to country name affords X.400 systems degrees of flexibility not available through other systems.

There are other problems. For example, most X.400 systems use static routing, which is based "primarily on the hierarchical nature of the address structure" and can be cumbersome to maintain (Betanov, 1993). Within a PRMD a message router must decide which MTA within the PRMD will be the final destination and what is the best route to that MTA. A message destined for another PRMD or ADMD must be sent to the correct gateway. A partial solution to the routing issue is to use an X.500 directory. (Like X.400, X.500 is a CCITT protocol, in this instance designed to build a distributed, global directory. X.500 offers decentralized maintenance. Each site running X.500 is responsible for its local part of the directory, so updates and maintenance can be done instantly. Like the Domain Name Service, X.500 provides a single homogeneous namespace to users; how-ever, the X.500 namespace is more flexible and expandable than the DNS. In addition, X.500 defines the information framework used in the directory, allowing local extensions, and it provides searching facilities powerful enough to allow users to construct arbitrarily complex queries. Finally, because X.500 can be used to build a standards-based directory, applications requiring directory information, e.g., electronic mail, automated resources locators, special-purpose directory tools, can access information and information services in a uniform manner, no matter where they are based or currently running.)6

The future of X.400 as a significant factor in electronic messaging services is not in doubt. Although the standard has its share of critics, some of them harsh, there are many observers who believe that X.400 is a set of concepts superior to any other existing scheme for wide-area messaging, but they also point out that it is arguably the only available means by which to build a global communications infrastructure. However, X.400 is a complex and cumbersome system, and the attendant problems should not be underestimated.

The specifications themselves are difficult to read and at times hard to understand and deci-pher. Perhaps more important, the two main versions of X.400 are incompatible with one another. As noted above, the 1984 version defined a basic store-and-forward technology, with message creation, routing and delivery services at its core, whereas the decision to expand delivery services and add security features in the 1988 version required changes extensive enough to render the two versions incompatible.

It is also hindered by the weakness of key basic features; for example, the lack of adequate directory services has hindered X.400 product development. Eventually X.500 will provide a globally published database of electronic addresses, telephone numbers, fax numbers, physical addresses, and so on. However, creating and maintaining such a database will be a mammoth undertaking that is expected to require billions of dollars to complete. So, many experts do not expect full implementation of X.500 services for at least another decade. As a consequence, directory services remain a serious problem for X.400 users, and one which many experts believe will limit for some time the full implementation of X.400 on a global scale. And, as noted above, another key weakness of X.400 is the failure to define adequately the interchange of complex messages.

Despite its complexity and current shortcomings, it seems likely that X.400 will be widely deployed. Part of this expectation is because the Government Open Systems Interconnection Profile firmly established the X.400 standard as a federal government buying requirement. Another rele-vant aspect is that X.400 has been embraced by major telecommunications forms, such as MCI and Sprint, and is already being used to link users on an international basis (Ayre, 1991). Perhaps most important of all, the stability provided by CCITT and OSI's commitment to X.400 and X.500 offers a powerful encouragement to developers like Microsoft, which plans to implement both in the forthcoming Windows NT.

In the near term, an area in which rapid development and development of X.400 may be expected is that of gateways. Despite the advantages that a truly universal X.400 message exchange would presumably offer, many large-scale users are today reluctant to become involved in the massive retraining and support efforts entailed by switching from one electronic mail system to another. However, X.400 gateways connecting two or more disparate electronic mail systems are likely to become very popular, because the X.400 MHS provides three distinct means of re-addressing messages coming from and/or going to non- X.400 users; in fact, the International Data Corporation reports that the electronic mail gateway market is growing at an annual rate of thirty-four percent, with X.400 gateways expected to account for most of the $520 million market that is anticipated by 1995.

In the long term, however, the key factor seems to be one of critical mass: enough people must be affected by X.400 services for it to become "a truly compelling connection" (Ayre, 1991). Interestingly, such a critical mass may not be that far off. Looking ahead, dramatic growth is expected in terms of the business uses of electronic mail services; given the scale and scope of modern commercial enter-prise, it seems reasonable to imagine that a growing cadre of business users spread across the world will strengthen the need for an international messaging standard. Another factor that may contribute to X.400's more widespread implementation is the possibility that when the National Science Foun-dation and the mid-level networks engage in the restructuring that seems implicit in the legislation underlying the National Research and Education Network (NREN), basic services such as electronic mail may be "privatized" and after that provided by corporations such as AT&T, MCI, and Sprint, each of which already supports X.400.

What are the key issues as we look ahead to the next generation of electronic mail service? First, it seems certain that electronic mail applications are going to be more closely bound to direc-tory services, whether the means of generating the directory services is based upon an open standard like X.500 or proprietary scheme, because the growth required to make electronic mail a truly ubiquitous medium can be sustained only by the provision of services that simplify its more complex procedures, such as ascertaining an address. The key issue where directory services are concerned is X.500. If global directory services are a necessary condition for electronic mail to reach its full potential as a communications medium, then the future of electronic mail cannot be separated from X.500 and its implementation. Although many observers predict that general implementation of X.500 will be slow and expensive, there are a number of reasons to be optimistic, not the least of them being the experimental X.500 service that has been mounted on the Internet by AT&T and Microsoft's commitment to the X.500 standard in its development of Windows NT.

In terms of the interchange of complex documents, the issues are cloudier. In the absence of a significant improvement in the multimedia capabilities of the X.400 MHS, MIME may play a critical role not only on the Internet, but beyond the Internet as well. There are at least a couple of reasons why this may prove to be so. First, the technologies of the Internet are important by virtue of the Internet's size and continuing growth. Even if low-end Internet services are not "privatized," the scale on which the Internet supports messaging would seem to make it an essential part of any serious effort to standardize the exchange of complex documents at the operational level. Or, considered from another perspective, the solution may be in the hands of Microsoft, Novell, the members of COSE (Common Open Systems Environment), and other commercial developers -- if they make their messaging APIs MIME-compliant, a major obstacle is removed; if they don't, then complex document exchange may not be a pervasive element in this global communications infrastructure. In some ways, a single global standard may be less important than a rationally defined hierarchy of protocols and services. Already there are signs that this may prove to be the case. For example, there are many reports in the computing industry's publications about the use of X.400 as the medium for engine-to-engine communications along the backbone of a wide-area mail system and signs that developers will embrace MIME with equally great interest.

Ayre, Rick, "X.400 and X.500: In Pursuit of the Universal Mailbox," PC Magazine, 10: 310-311 (September 10, 1991).

Borenstein, Nathaniel. Internet Multimedia Mail with MIME: Emerging Standards for Inter-operability. Unpublished paper, 1992.

Borenstein, Nathaniel, and Freed, Ned. MIME (Multipurpose Internet Mail Extensions): Mechani-sms for Specifying and Describing the Format of Internet Message Bodies. Network Working Group, RFC 1341, 1992.

Combs, Harold B., "Future Prospects for X.400," Telecommunications, 26: 54-57 (July 1992).

Crocker, David H. Standard for the Format of ARPA Internet Text Message. RFC 822. University of Delaware, 1982.

"E-mail standards must support all data equally," PC Week, 10" 66 (April 26, 1993).

Rose, Marshall. The Internet Message: closing the book with electronic mail. The Internet Message: Closing the Book with Electronic Mail. Englewood Cliffs, N.J.: P T R Prentice Hall, c1993. p.325.

Tesler, L. "Networked computing in the 1990s," Scientific American, 265: 86-93 (September 1991).