Exchange’s primary purpose is email, although it supports other forms of messaging as well. Exchange is a server-based messaging system, which means the server receives email on behalf of users, and stores that email for the users. Users can connect to the server and view their email directly on the server, without having to download it to their client computers. Or, users can download the email to their client computers, which is useful for mobile users who can’t remain connected to the server while working with their email.

Messaging Databases

Exchange stores messages in an information store. The information store is simply a database, running on Microsoft’s Extensible Storage Engine (ESE). The ESE is the same database engine used by SharePoint Portal Server and Active Directory. The ESE is optimized for hierarchical storage, which is exactly what messaging systems need. Think about how an email Inbox is arranged into a hierarchy of folders and messages, much like the file system on a computer’s hard drive. The ESE is a transaction-based database, which means that changes submitted to the server are first written to a special transaction log. The appropriate data is loaded from the database into the server’s memory and changed. Every few seconds, the changed data in memory is written back out to the database, and the transaction log is marked to indicate which transactions are now safely in the database, or committed. In the event of a server failure, any changes still in memory would normally be lost. However, when Exchange restarts, it examines the transaction log for uncommitted transactions. If it finds any, it simply repeats those transactions, preventing a loss of data.

Note

The transaction process is very similar to the way SQL Server’s transaction-based database system works.

It’s ESE, Not Access

You may have heard that Exchange uses Microsoft’s Joint Engine Technology, or JET, database engine—supposedly the same database engine used in Microsoft Access. Many administrators who are new to Exchange find it alarming that an enterprise messaging system would be based on Microsoft Access. While there’s a grain of truth to the “Exchange uses Access” theory, it’s only a grain.

Here’s the deal: Microsoft actually maintains two versions of the JET database engine, named Jet Blue and Jet Red. One engine is the relatively simple database engine used in Microsoft Access. The other is the full-fledged, transaction-based, multi-threaded, server-based engine that’s also called the ESE. The engines are as different as different can be. The only reason they share the same name, in fact, is that they use similar internal indexing and query technologies, which are the heart of the JET concept.

Still, the ESE’s days are numbered. Maintaining two fully functional server-class database engines—the ESE and SQL Server—is expensive, even for a company the size of Microsoft. Microsoft has already committed to converting a future version of Exchange to use a future version of SQL Server as its database, and it’s likely that other ESE-based applications (SharePoint Portal Server and Active Directory) will eventually follow suit.

Exchange has a number of unique concepts around its messaging databases. A database, in Exchange terminology, consists of an EDB file that contains data formatted in either HTML or rich text, which includes most email messages. The database also includes an STM file, which includes other content—mainly file attachments such as audio or video formats—in their native format. The STM file is capable of streaming data, which means clients can begin using file attachments such as video and audio after receiving just the beginning of the file, and continue playing or using the file while the server continues to send it. The streaming technique provides a much better user experience than requiring users to wait until an entire file transfers to their client before allowing them to use that file.

Most Exchange Servers will contain at least two databases: a public store and a private store. The private store is where all personal email messages and their associated attachments are stored. A public store contains Exchange’s public folders, which can include discussion groups and other information designed to be accessible by groups of individuals within an organization (or even by Internet users). Within each store, Exchange uses single-instance storage. This means the store usually doesn’t contain more than one copy of a message. If the server receives a message addressed to 10 recipients, it keeps only one copy of the message, and includes pointers indicating which users received the message. When the last user deletes the message, the message is removed from the store.

Unlike prior versions of Exchange, each Exchange 2000 Server computer can contain multiple databases, enabling you to spread your users’ mailboxes across multiple files.

Note

The ability to configure multiple stores and databases is unique to Exchange’s Enterprise Edition. Currently, the Standard Edition is restricted to a single store with a single database.

Databases can be individually stopped, started, and restored, which helps minimize the number of users who are affected by a corrupted database file or other file-specific failure. Managing a large number of database files can be complex, however, so Exchange also has a concept called storage groups. A storage group is a set of private and/or public message stores that share a single set of transaction log files. Storage groups enable you to maintain large servers more easily, because you can administer the storage group as a single unit, rather than having to individually manage each store within the storage group. Exchange manages each storage group with a separate server process, which means you can also use storage groups to help evenly distribute a single Exchange server’s processing load. For example, if you have six private store databases on a server, configuring them into three storage groups will help ensure that each group receives approximately the same portion of the server’s processing power. Figure 10.1 shows the relationship between a server, databases, storage groups, and transaction logs.

Figure 10.1. By default, all of the databases on a server are contained within a single storage group.

Tip

You can improve Exchange’s performance and reliability by keeping transaction logs on separate physical drives, which are ideally accessed by a separate disk controller from the one that handles the actual databases.

IPM: Not Just for Email

Exchange works with messages in a format known as IPM, for interpersonal message. IPM is useful because not only can it store messages, it can also associate a display format with them. IPM enables a single Exchange database—the EDB file—to store not only email, but also contacts, calendar entries, journal entries, and much more. Internally, each of these IPM types is stored in a similar fashion, but the IPM type itself defines certain characteristics about the message content. For example, email messages don’t have to have an alarm time associated with them, although many calendar “messages” do have alarms.

The fact that all IPMs are treated alike within Exchange is important. All of Exchange’s features, such as single-instance storage, apply to all IPM types. For example, if you send a meeting request to five users within your organization, Exchange is able to track their responses because the meeting request itself exists only once within the Exchange message store. Each invitee’s calendar is also updated with their own personal IPM that represents the meeting.

The IPM format is especially of interest to developers, since the format enables them to work with the content of Exchange’s databases in a more consistent fashion than if each message type (email, calendar, and so forth) were stored in a unique format.

Because Exchange is completely integrated with Active Directory, it can take advantage of Active Directory’s own security features. In prior versions of Exchange, permissions on mailboxes and public folders were assigned to mailboxes, because they represented security principals within Exchange. You could assign permissions to distribution lists, too, which functioned as groups. Now, you simply assign permissions directly to Active Directory users and groups, just like you assign file permissions on a file server.

Exchange 2000 Server introduced a new add-on product called Exchange Conferencing Server. Conferencing Server builds upon the Exchange platform by adding videoconferencing and collaboration capabilities, which are completely integrated with the basic Exchange product. Users can use their Outlook calendars to schedule videoconferences, and join those conferences directly from Outlook when the time arrives. Conferences can be conducted through a number of industry standards, including T.120 and H.323 (used by Microsoft NetMeeting).

Server-based conferencing is more powerful than peer-to-peer conferencing (which is how NetMeeting is usually used), because servers can provide multiparty audio and video stream, enabling each conference participant to see and hear all of the other participants. Further, Exchange Conferencing Server relies on multicast technology to provide better network throughput.

In any TCP/IP network, there are basically two ways to send data: directly to another computer, or to a special multicast address. For videoconferencing, direct transmission is inefficient, because the conferencing server has to relay the conference data stream independently to each participant. With even just a few participants, the server can quickly become overwhelmed. Broadcasts enable the server to send only one copy of the data, but it’s received by all computers, even if they are not participating in the conference. Furthermore, broadcasts are limited to a single network segment, and cannot cross routers. Multicasts provide a combination of the direct transmission and broadcast methods. In a multicast, clients are given a special IP address when they join a conference. The server sends one copy of the conference’s data stream to that special IP address, and all clients who want to participate in the conference pick up the traffic sent to the special multicast IP address. Routers can be programmed to forward traffic destined to the multicast address, making the multicast work across large networks. Multicasting makes the most efficient use of network resources while delivering video, audio, and other conference data.

In addition to video and audio conferencing, Exchange Conferencing Server supports data conferencing, which includes multiuser collaboration tools, such as multiparty application sharing. This technology enables, for example, all conference participants to work in a shared copy of Microsoft Word on a single shared document, much as they might work on an easel pad during a physical conference.

Instant messaging has become one of the new “killer apps” for corporate environments, enabling users to quickly send messages to one another across the network. Unfortunately, many companies prohibit the use of instant messaging systems, because most of them—such as Yahoo! Messenger, AOL Instant Messenger, and MSN Messenger—also allow employees to easily communicate with individuals outside their organization, making the instant messaging concept seem like a productivity-killer.

There’s no reason, however, that you can’t implement your own internal-only instant messaging system. Such a system requires two basic components: Presence services act as a directory of users who are currently online, enabling other users to locate and contact them, while messaging services actually relay messages from one user to another. Exchange can provide both services to your network. In fact, a single Exchange server can easily support 15,000 or more instant messaging users, making it a relatively inexpensive investment. Exchange only works with the MSN Messenger client (called Windows Messenger in Windows XP), or with the shareware “universal” clients that interoperate with the MSN Messenger network.

Exchange implements the two components of instant messaging—presence and message routing—as two separate functions. You can designate home servers, which are Exchange servers that accept instant messaging connections from clients. Message routers are separate servers responsible for routing messages between users on different home servers. Smaller environments can make do with a single home server and forgo a dedicated router, but providing the router function as a separate entity enables you to create more scalable architectures for very large environments. Figure 10.2 shows a complex instant messaging architecture that utilizes multiple home servers and routers.

Figure 10.2. This instant messaging design could easily support tens of thousands of instant messaging users.

Exchange also provides chat services to users. The Exchange 2000 Chat Service is based on the Internet-standard Internet Relay Chat (IRC). Chat services are implemented as communities, which correspond to a single IP address and port on a server (a single server can host multiple communities). Communities exist on only one server, and can’t be spread across multiple servers; as a community grows in size you’ll either need to split it up into multiple communities which can be hosted on separate servers, or upgrade to more powerful server hardware. A single moderately powerful server can support more than 20,000 chat users.

Communities are divided into channels, with each channel representing a particular topic of discussion. Each channel has its own member list, and each channel can be configured with a variety of discussion options:

Microsoft provides chat client software called Comic Chat. Exchange also supports any IRC-compliant chat client, including popular freeware and shareware choices such as PIRCH, mIRC, and others.

Exchange also supports IRC extensions, which use the IRCX protocol. Developed by Microsoft, IRCX channels are visible only to clients that support the IRCX protocol. IRCX channels offer additional commands and features not found in regular IRC channels, such as enhanced user and channel management features.

Perhaps the biggest new feature in Exchange 2000 Server is its multi-tier architecture. Prior versions of Exchange combined all of the product’s capabilities onto each server in your environment. In other words, a single server could contain mailboxes and public folders, provide services to clients, and so forth. While Exchange 2000 can be used in the same fashion, you can also break Exchange into multiple tiers of functionality. For example, you might designate a set of back-end servers that do nothing but handle messaging databases, storing messages for users. A separate tier of servers might accept client connections via IMAP4, acting as a gateway for users to access the email stored on the back-end servers. Figure 10.3 shows a basic multi-tier architecture.

Figure 10.3. Designating servers to handle a single task, such as POP3 client connections, enables them to handle that task with maximum efficiency.

It’s a simple fact that servers waste time when they have to do more than one thing. Keep in mind that a server’s microprocessor can only do one thing at a time. Multitasking is really an illusion in which the processor jumps between tasks so rapidly that it seems to be doing them all at once. The switching process involves a certain amount of overhead, however. On the larger scale of an application server such as Exchange, focusing on a single task makes the server more efficient at that task. Microsoft, for example, was able to reduce the number of Exchange computers in its corporate environment when moving to Exchange 2000 Server, because it could designate each server to handle specific tasks.

Exchange supports a number of different clients:

Unfortunately, if you’re looking for a single client that does it all, you’re out of luck (although Outlook is perhaps the closest you’ll come). Most of the preceding clients are included with later Windows operating systems, such as Windows XP, but Microsoft Chat, for example, is a separate download (it’s also on the Exchange 2000 Server CD). NetMeeting doesn’t take advantage of all of Exchange Conferencing Server’s features, although you can use more powerful third-part T.120-compatible conferencing clients along with Conferencing Server.

As with many other .NET Enterprise Servers, Microsoft sells Exchange in a couple of different editions. The big version is Enterprise Edition, which includes clustering support and no restrictions on how Exchange can be deployed. The less-expensive Standard Edition lacks clustering support, and also includes a number of restrictions on how the product can be used:

Essentially, Standard Edition is appropriate if you need only one or two Exchange computers with fewer than 1,000 users apiece. For larger environments, the additional flexibility offered by Enterprise Edition is a must.

Exchange is one of the only .NET Enterprise Servers that is still licensed on a per-client basis, rather than per-processor, so it makes sense to use powerful, multi-processor servers that can support as many users as possible. You’re paying by the user, so supporting them with as little server hardware as possible will help save money.

Exchange Server’s reliance on Active Directory is absolute. While many of the other .NET Enterprise Servers can use Active Directory to store configuration information, or to assign permissions to domain users, Exchange Server simply won’t function without Active Directory. In prior versions of Exchange, Exchange itself maintained mailboxes, and linked those mailboxes to a specific NT domain account, thereby granting that account permission to use the mailbox. However, the domain account and the mailbox were always managed independently.

With Exchange 2000 Server, Exchange maintains the contents of mailboxes, but Active Directory itself maintains the definition of the mailbox. Any Active Directory user account can be mailbox-enabled, a process which links the account to a mailbox store in Exchange. Mailbox-enabling an account also extends the account’s attributes to include messaging-specific properties, such as the maximum amount of space the user’s mailbox may consume on the server. All management of the mailbox is performed from within Active Directory interfaces like Active Directory Users and Computers.

Active Directory also acts as Exchange’s Global Address List (GAL). Active Directory groups, including Universal Distribution Groups, can be used as mailing lists from within Exchange. Active Directory can also contain contacts, which represent the email addresses of external users, and enable Exchange users to easily store shared contact information within Active Directory. Whenever users type a new email message, Active Directory is used to ensure that the recipient addresses they provide are correct, a process called address resolution.

In fact, address resolution represents one of the most important interactions between Exchange and Active Directory. In prior versions of Exchange, Outlook clients would contact the Exchange Server directly to perform address resolution and GAL searches. In Exchange 2000 Server, however, Outlook clients contact the Exchange server to receive a list of Active Directory domain controllers configured as Global Catalog (GC) servers. Outlook then contacts the GC server directly to complete the address resolution. This behavior distributes the messaging load across a larger number of servers, but it places a heavy additional load on the domain controllers designated as GC servers. In fact, many organizations find that they need to double or even triple the number of GC servers in their environments to accommodate Exchange.

Exchange is aware of Active Directory domain and site configurations, although a single Exchange organization can span multiple sites domains, provided they are all contained within the same Active Directory forest. Additionally, each Active Directory forest can contain only one Exchange organization.

Although prior versions of Exchange have made ever-increasing use of Internet technologies, Exchange 2000 Server is built entirely around open Internet-standard protocols and technologies, including

Understanding the protocols that Exchange Server uses can help you configure your network to better support Exchange Server and Exchange clients. For example, if you’ve got Exchange Server computers spread out across a distributed network, you can program your routers to prioritize SMTP traffic, thereby making communication between Exchange Server computers more efficient. If you need users to be able to access their email from outside a firewall, then you’ll need to select protocols—POP3, IMAP4, or HTTP—to allow through the firewall and into the Exchange Server computer.

Exchange doesn’t directly integrate with Windows’s Certificate Services, but it can benefit from the digital certificates that Certificate Services creates. Exchange includes its own Key Manager application, which enables you to easily manage the certificates (keys) that your Exchange users need. Exchange uses certificates for two primary purposes:

Exchange is designed to use certificates from almost any source, including commercial certificate authorities like Thawte (www.thawte.com) or VeriSign (www.verisign.com). However, within an organization, it’s easier and much less expensive to create your own certificate authority using Windows Certificate Services (or another certificate system), and issue your own certificates to your users.

The simplest Exchange design is the same design Exchange administrators have been building since the product’s first version. Shown in Figure 10.4, this design includes all of Exchange’s services on a single computer. As shown, you might include a second server to host instant messaging and chat functions, if you need them in your organization. This design is suitable for small organizations with perhaps 1,000 mailboxes or fewer, and relatively moderate messaging volumes.

Figure 10.4. You can always start with this kind of all-in-one design, and later move certain services to additional Exchange computers if you need to increase the system’s capacity.

Suitable hardware for this kind of computer includes large RAID 5 hard drive arrays, multiple processors (2–4), and plenty of RAM—generally as much as you can install in the server hardware you’ve selected, or as much as your edition of Windows will support. Fast network connections are a must, especially to the supporting Active Directory domain controller and GC servers.

The disadvantage of single-server Exchange designs is that Exchange—like any other database engine—can only scale up so far, meaning that you can only upgrade server hardware to a certain point. Once you’re running on a eight-processor computer with 8GB of RAM, you’ve pretty much hit the ceiling of what hardware can do for you. If such a server still isn’t capable of supporting your business needs, then you need to scale out, by adding additional servers to handle the workload. Scaling out in Exchange means breaking the server into two tiers, as shown in Figure 10.5. Multiple front-end servers can accept protocol connections from users, and all talk to a single back-end Exchange computer which does nothing but handle the system’s message stores. Each front-end server can host one or more protocols, enabling you to provide the right number of front-end servers for each protocol, based on that protocol’s popularity. For example, you might provide the most POP3 servers if your client base mainly uses Outlook Express, or more HTTP servers if your clients make heavy use of Outlook Web Access.

Figure 10.5. Scaling out enables you to create a custom-sized front end for each Exchange protocol that you must support.

When the back-end storage server can’t keep up with the workload of the front-end, you can scale the back-end separately. While front-end servers can run on relatively inexpensive servers (thus creating a server farm not unlike a Web farm with many smaller-sized Web servers), back-end servers require fast drive arrays, significant amounts of memory, and other high-end resources. Scaling the back-end is therefore a more serious and costly decision. It also requires you to partition your back-end, splitting your user mailboxes between back-end servers, as shown in Figure 10.6.

Figure 10.6. You can divide your mailboxes any way you want, but try to create an even distribution of large mailboxes and high-volume users between the back-end servers.

One nice feature of Exchange’s architecture is that you don’t have to worry about moving mailboxes between back-end servers. Exchange’s front-end servers automatically determine where mailboxes are physically located. Users simply have to connect to a front-end server, and that server will locate the user’s mailbox on the appropriate back-end server. As a result, you can move mailboxes whenever you need to without having to reconfigure your clients.

Distributing Exchange across a number of different sites is more difficult than using Exchange in a single site, because of the relationship between Exchange, domain controllers, GC servers, and Active Directory itself. Figure 10.7 shows what is perhaps an ideal design from a management standpoint, with all Exchange back-end servers located in a central office, and front-end servers deployed to branch offices. Notice the relationship between domain controllers and GC servers in those branch offices.

Figure 10.7. Keeping the Exchange back-end servers in one place makes disaster recovery operations easier and more efficient.

Larger organizations with distributed IT departments may find multiple back-end Exchange Server computers easier to manage. In Figure 10.8, each major division of the company has a centralized Exchange back-end, with distributed front-end servers for that division’s employees. Some sites house front- and back-end servers for multiple divisions, but again notice the relationship between Exchange and the Active Directory servers.

Figure 10.8. Each division is responsible for disaster recovery operations on their own back-end servers.

When slow network links are a problem, you have two options. You could simply forgo local Exchange servers completely, and force users to access their email and other Exchange information through a low-bandwidth solution such as Outlook Web Access. That’s rarely an acceptable solution, since users prefer the richer, always-connected environment that regular Outlook offers. In those cases, you may need to include a complete Exchange Server computer in the branch office, perhaps using a computer that doubles as a DHCP or file server for the office as well.

Figure 10.9 shows what the network might look like. Notice again the strategic placement of Active Directory domain controllers and GC servers. Also notice that the rest of the organization—with faster wide-area links—continues to use a centralized back-end. The biggest potential problem in this case is a user from the “slow” office visiting a “fast” office, and having to access their email across the slow wide-area link.

Figure 10.9. Backup and restore of the remote Exchange storage will be more difficult unless the “slow” office has on-site IT staff.

Perhaps the most difficult Exchange configurations come into play in multi-domain environments. Exchange can easily span domains that exist within the same forest, but the risk of poor performance due to badly placed Active Directory domain controllers and GC servers is very real. The trick is to always remember the domain controller and GC server requirements of Exchange and Exchange clients, which I’ve already discussed. Figure 10.10 shows an example of a multi-domain environment, and how domain controllers and GC servers support Exchange.

Figure 10.10. Most sites have dual domain controllers and GC servers for redundancy.

IBM’s Lotus Notes product (the server component is named “Domino”) has been Exchange’s primary rival since Exchange’s introduction. Notes achieved a lot of early market share because it included application development capabilities, enabling organizations to build workflow and other collaborative applications right within Notes. Notes’s recent development has turned the Domino server component into a full-fledged Web server, making the product suitable for deploying certain types of Web-based applications, especially intranet applications. Notes also has the advantage of running on a wide range of operating systems, including Windows and even IBM’s AS/400 (getting Notes running on the AS/400 was a driving factor behind IBM’s acquisition of the Lotus company several years ago).

Internally, Notes’s storage engine is a bit less sophisticated than Exchange’s, using a store-and-forward technology that’s more than a decade old. Ironically, Exchange actually offers more robust application development capabilities, with its complete integration with both the Windows operating system and Microsoft’s COM software development architecture. Unfortunately, Exchange’s software development features are—and always have been—less readily accessible than Notes’s, making it more difficult for software developers to understand and leverage Exchange’s capabilities in application development. Even today, with significant workflow and data capabilities, few organizations build applications on Exchange.

Both Microsoft and IBM have presented strong cases for their products, each claiming that their platform offers a lower total cost of ownership, faster return on investment, and so forth—simply proving the point that you can make numbers say anything you want. Organizations with a heavy investment in Microsoft technologies tend to opt for Exchange; companies with a history of using IBM products—or Notes’s messaging predecessor, cc:Mail—tend to stick with the Notes/Domino platform. I’m obviously a strong fan of Exchange, but I heartily recommend that you perform a competitive analysis of both products within your organization.

The popularity of Internet protocols such as POP3, IMAP, and HTTP have resulted in an array of inexpensive mail server products—such as iMail—based on those standards. Many organizations that need nothing more than basic message storage and retrieval are taking advantage of these systems to provide inexpensive, easy-to-maintain solutions for their users.

These products invariably accept email via the SMTP protocol, send outgoing email via SMTP, and enable users to retrieve their mail by using a POP3 or IMPA4 client, or via a Web-based interface. These products rarely include any collaborative features, and rarely provide contacts, calendaring, notes, and the other features often associated with Exchange Server and Outlook. However, despite their lack of features, these solutions are usually only a few hundred dollars, can run on a computer costing less than $2,000, and often require little to no maintenance. Some companies provide “black box” solutions based on the SMTP/POP3/IMAP4/Web mixture, enabling you to simply plug the box into your network for an instant email server. These boxes can run from a few hundred dollars to less than $2,000, and provide a no-configuration, no-maintenance way to provide basic email to a small organization.