Deploying a Hybrid Messaging Infrastructure Using Office 365: Exchange Online

Transcription

1 Deploying a Hybrid Messaging Infrastructure Using Office 365: Exchange Online Enterprise Messaging Combining On-Premises and Cloud-Based Technologies Technical White Paper Published: June 2012 The following content may no longer reflect Microsoft s current position or infrastructure. This content should be viewed as reference documentation only, to inform IT business decisions within your own company or organization.

2 CONTENTS Executive Summary... 3 Hybrid Advantages... 4 Cost Savings Due to Cloud Efficiencies 4 Flexible Deployment and Management 4 Microsoft Messaging Infrastructure At-A-Glance... 6 On-Premises Messaging Architecture 6 Hybrid Messaging Architecture 6 Designing for Hybrid Messaging... 9 Identity Management 9 ADFS Architecture 10 Usage Patterns and Bandwidth 13 Client Performance 14 Service Dependencies 14 Mail Flow 15 Forefront Online Protection for Exchange (FOPE) 17 Migrating Mailboxes to Exchange Online Migration Approach and Process 19 Phases 20 Supporting Users Lessons Learned and Best Practices For More Information... 25

3 Situation Office 365: Exchange Online, offers Microsoft IT the opportunity to add flexibility to the messaging infrastructure and cut costs by deploying and operating a hybrid environment Solution As part of a long-term strategy, Microsoft IT onboarded 36,000 mailboxes to Exchange Online with the goal to migrate 80% of all mailboxes by This hybrid deployment offers the best features and benefits of both on-premises and cloud-based approaches. Benefits Seamless user experience using Outlook Web App and Outlook 2010 client Cost savings associated with using cloud-based messaging that Exchange Online provides Flexibility to accommodate business growth through Exchange Online without expanding the onpremises infrastructure SPAM and virus protection through Forefront Online Protection for Exchange Synchronized address book and single sign-on Products & Technologies Office 365: Exchange Online Exchange Server 2010 Active Directory Windows Server 2008 R2 EXECUTIVE SUMMARY Although hosted solutions for messaging have been available for many years, recent improvements have made it possible to deploy and operate a hybrid environment that makes the most of both on-premises and hosted services. Microsoft began offering Exchange Online as its multi-tenant enterprise messaging service in the cloud to customers starting at the end of 2008 based on Exchange Server 2007 technology with the goal of helping customers and its own workers realize the benefits of cloud computing. After onboarding millions of mailboxes from companies of all sizes, building out a scalable and highly available infrastructure, and upgrading Exchange Online to run Exchange Server 2010, Microsoft IT pursued an initiative to transition from operating its own on-premises Exchange environment to operating a hybrid environment. With a hybrid approach, Microsoft IT benefits from continuing to use previous investments in the existing on-premises infrastructure, with ability to accommodate business growth by using Exchange Online. To overcome the engineering and business challenges in transitioning to a hybrid environment, Microsoft IT focused on ensuring user satisfaction by engaging all teams involved in the deployment effort. One key objective was to provide users with a seamless transition and automatic Outlook profile update to Exchange Online yet retain the same features and functionality of the on-premises service. To ensure the best user experience, the hybrid architecture incorporates design elements that include the following: Single sign-on (SSO) using existing Active Directory credentials and Active Directory Federation Services (ADFS) Shared address book for a unified global address list (GAL) One microsoft.com domain namespace for both on-premises and Exchange Online Centralized administration of mailboxes and mail flow Synchronized calendar and free/busy scheduling Landing page to inform Exchange Online users who log in to the Outlook Web App onpremises URL about the appropriate Exchange Online Outlook Web App URL The Exchange Server architecture enables Microsoft IT to deploy messaging in a hybrid environment according to the needs of the business and desired project schedule. Microsoft IT moved mailboxes to Exchange Online after preparing the environmental dependencies such as identity management, security, and synchronization. In this way, Microsoft IT controls accounts, retention, e-discovery and other features in a unified way to ensure a centrally managed, homogenous environment. This white paper contains information for business and technical decision makers who operate an on-premises messaging solution and are evaluating possibilities of transitioning to a hybrid environment that incorporates Exchange Online. The paper assumes basic familiarity with concepts relevant to messaging technologies, such as Active Directory, Exchange Server, TCP/IP, and DNS. A high-level understanding of the capabilities of Exchange Online and Office 365 is also helpful. For more information about Exchange Online, see Note: For security reasons, the sample names of forests, domains, internal resources, organizations, and internally developed security file names used in this paper do not represent real resource names used within Microsoft and are for illustration purposes only. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 3

4 HYBRID ADVANTAGES The hybrid architecture results in many benefits for Microsoft IT not only in overall cost savings, but also in greater flexibility to accommodate business growth while saving time and money by not having to do capacity planning, update software, maintain servers, or manage hardware. Cost Savings Due to Cloud Efficiencies As a cloud service, Exchange Online provides the opportunities to reduce costs by eliminating the typical on-premises requirements of purchasing, deploying, and managing servers. These savings are possible due to Exchange Online features such as the following: Large 25 GB mailbox size With Exchange Server 2010, Microsoft IT eliminated backups and relies on a cost-effective, Just a Bunch of Disks (JBOD)-based storage. This solution offers cost savings over the previous Storage Area Network (SAN) approach, yet it is a high expense to deploy and operate the storage subsystem. Exchange Online frees Microsoft IT from the need to manage any storage hardware. Quota management. During the initial phases of using Exchange Online in a hybrid environment, it is important to manage quotas in case mailboxes need to move back to on-premises. Microsoft IT uses the same quotas for both environments to prevent the possibility of having to increase on-premises quotas for specific users, or asking them to reduce their mailbox size. Included technical support Exchange Online includes 24/7 phone support for the internal Microsoft IT support team, which helps to ensure timely responses and reliability. Automatic failover Similar to the on-premises solution, Exchange Online also provides automatic failover for resiliency. Highly available design Exchange Online includes mailbox resiliency technology, such as the ability to switch between database copies when disks fail, and automatic, database-level recovery from failures through database availability groups. Flexible growth and expansion As Microsoft grows and changes, Exchange Online makes it straightforward to add mailboxes by simply buying additional licenses. This requires no capacity planning, server purchasing, or deployment. Flexible Deployment and Management Exchange Online and on-premises overlap in terms of management functionality. Both use Role-Based Access Control (RBAC) for task delegation and administration via the Exchange Control Panel web-based console or through Windows PowerShell using the Exchange Management Shell. Microsoft IT uses the remote PowerShell capability for managing Exchange Online from within the on-premises network. Exchange Online gives Microsoft IT management capabilities relevant to messaging-as-a-service, including recipient policies and groups, whereas Exchange on-premises provides all management capabilities. A hybrid approach achieves the best of both worlds by enabling Microsoft IT to accomplish the following: Deployment on Microsoft IT s terms A hybrid approach offers Microsoft the flexibility to migrate mailboxes as needed to and from Exchange Online. As a way of validating the Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 4

5 hybrid approach, Microsoft IT started migrating the mailboxes of a small number of volunteer early adopters, and sets the pace of migration according to its needs. Infrastructure ownership and control Approaching messaging as a hosted service and an on-premises service gives Microsoft IT the flexibility to own the entire messaging continuum for the ultimate degree in infrastructure flexibility. Due to business needs, some mailboxes may remain on-premises, and others may be migrated to Exchange Online. If requirements change, Microsoft IT may move mailboxes from one environment to the other without affecting users. Centralized management Both Exchange Online and on-premises share a unified approach to managing mailboxes, policies, recipients, and other Exchange objects. In the hybrid implementation, Microsoft IT manages all messaging details in a unified and centralized way. Customer validation and dogfooding Validating hybrid performance and functionality as part of dogfooding efforts is one of Microsoft IT s key goals. Part of the design and deployment entailed working through many types of possible scenarios to work out any issues and fine-tune best practices. This goal went beyond implementing quick fixes and resolving bugs, to validating administrative and support paths to ensure the hybrid architecture was suitable for enterprise needs. Single namespace and unified experience Microsoft IT's hybrid design relies on auth headers in Exchange data, making communication appear internal to both on-premises and Exchange Online. As a result, Exchange features such as MailTips, and out-of-office (OOF) messages function and appear as expected to users and recipients. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 5

6 MICROSOFT MESSAGING INFRASTRUCTURE AT-A-GLANCE The Microsoft internal messaging infrastructure supports more than 200,000 mailboxes for employees, contractors, and business partners across three core divisions involving hundreds of products and services. As a company, Microsoft operates in more than 100 countries, with the majority of employees working in its Redmond, Washington headquarters. To support the corporate messaging environment, Microsoft IT manages multiple regional data centers connected by high-speed WAN links. The network dependencies have been refined and improved over time to where routing, DNS infrastructure, bandwidth, and other similar considerations are stable with high levels of redundancy and availability. You can find out more about the Microsoft Exchange Server 2010 architecture at Although the technological capabilities of Exchange Server 2010 have enabled Microsoft IT to reduce costs and increase efficiencies by taking advantage of server consolidation and more flexible and larger storage, additional opportunities exist with a hybrid approach that incorporates Exchange Online. On-Premises Messaging Architecture The Exchange Server 2010 topology and architecture continues a tradition of following best practices, incorporating product group recommendations, and meeting business needs based on real-world performance data. Figure 1 shows a high-level overview of the Microsoft onpremises architecture before implementing a hybrid infrastructure. On-Premises Environment North America, Dublin, Singapore Unified Messaging Office 365 Internet Client Access Mailbox Hub Transport Forefront Online Protection for Exchange (FOPE) Figure 1 On-premises messaging infrastructure The Exchange Server roles facilitate and separate the necessary functions of into servers that handle message filtering, transport, client access, mailbox storage, and unified messaging. As a best practice, Microsoft IT suggests deploying multi-role Exchange servers to support a hybrid infrastructure. For more information, including capacity planning, see Hybrid Messaging Architecture As a cloud-based offering, Exchange Online provides messaging-as-a-service with an architecture that abstracts dependencies such as message filtering into additional services. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 6

7 The Exchange Online architecture uses a similar role-based approach as on-premises, but driven by the following services instead of roles: Forefront Protection for Exchange (FOPE): as a first-tier message handler for Exchange Online, FOPE provides protection from viruses and SPAM. Microsoft IT has used FOPE as a service since 2007 as its message filtering solution. Office 365 directory Exchange Online uses its own directory service for user data. To handle authentication, the directory service relies on Microsoft Online ID. Exchange Online messaging As the core service that handles messaging, Exchange Online includes transport and storage functionality to house mailboxes and facilitate mail flow. Figure 2 shows Exchange Online in a hybrid architecture with Exchange on-premises. On-Premises Environment Office 365 Perimeter Internal Domain Controller ADFS Proxy Directory Sync ADFS Federation Microsoft Online ID Microsoft Federation Gateway Directory Hosted messaging service Internet Unified Messaging Client Access Mailbox Hub Transport Remote user authentication Directory synchronization ADFS trust Remote access user FOPE Figure 2 Hybrid architecture In a hybrid infrastructure, Exchange Online relies on the following additional services to enable cross-premises mail flow, synchronization, and unified management. Microsoft Federation Gateway As an intermediary between Office 365 and onpremises services, the Microsoft Federation Gateway provides an identity service that connects users to the hosted services they want to use. For more information about the Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 7

8 Microsoft Federation Gateway, see ADFS To enable single sign-on and communicate with the Microsoft Federation Gateway, Microsoft IT relies on ADFS. Microsoft Online Services Directory Synchronization tool To synchronize mailboxes, the global address list, and other data, Microsoft IT relies on the Directory Synchronization Tool. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 8

9 DESIGNING FOR HYBRID MESSAGING The successful deployment of a hybrid messaging infrastructure for Microsoft IT requires that all the dependent on-premises services such as ADFS operate reliably and meet business needs. These services perform the intermediary data handling between on-premises and Exchange Online, that make account and messaging synchronization possible, as well as enabling workers to continue using the Outlook client, Outlook Web App, and mobile devices. At a high level, Microsoft IT fulfilled the following requirements in the hybrid design: Service domain to facilitate single domain namespace To forward from onpremises to Exchange Online, Microsoft IT configured a new DNS service domain for coexistence named messaging.microsoft.com. Upon sign up, new companies are automatically given a customizable coexistence domain with the format <custom domain>.mail.onmicrosoft.com. On-premises federation through ADFS The ADFS infrastructure is the on-premises service that provides a trust relationship between on-premises and Exchange Online to make single sign-on possible. Exchange federation through Microsoft Federation Gateway The Microsoft Federation Gateway is the trust broker that enabled Microsoft it to establish a federation trust from Exchange Online to the on-premises Exchange environment. This enables synchronization and sharing of Exchange information, such as free/busy data. For more information, see The Microsoft IT environment is specifically designed for Microsoft business needs, yet the technical requirements and steps for deploying a hybrid environment are the same for all companies. For a guided lab that shows the steps of configuring on-premises and Exchange Online components, see Identity Management To make the experience seamless for administrators and workers, the messaging environment must support a single authoritative source of user identity, with associated authentication, authorization, and permissions management. In a hybrid approach, the technical solution for a single authoritative source is to populate the Exchange Online directory with on-premises users, and then keep the two directories synchronized. As shown in Figure 3, there are three technologies Microsoft IT uses for synchronization to take place: ADFS 2.0 To communicate between the on-premises Active Directory environment and Exchange Online, Microsoft IT relied on the established ADFS infrastructure and created a relying party trust relationship between the ADFS federation server farm and Exchange Online. This relying party trust is a conduit for authentication tokens to facilitate single sign-on. Microsoft Federation Gateway As an intermediary between Office 365 and onpremises services, Microsoft provides an identity service that connects users to the hosted services they want to use. Directory synchronization tool Exchange Online begins using the on-premises identities the first time that the directory synchronization tool is run. The directory synchronization tool synchronizes key data every three hours, including mail-enabled Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 9

10 contacts and groups, global address list (GAL), on-premises-based safe and blocked senders, and delegation details. On-Premises Environment Perimeter Internal Domain Controller ADFS Proxy Directory Sync ADFS Federation Microsoft Online ID Microsoft Federation Gateway Office 365 Directory Hosted messaging service Internet Exchange organization FOPE Figure 3 Identity, synchronization, and single sign-on technologies ADFS Architecture On-premises Exchange federation trust Exchange Online pre-defined federation trust Directory synchronization from on-premises On-premises to Exchange Online ADFS trust ADFS Exchange Online trust to Online ID The ADFS infrastructure at Microsoft supports single sign-on for over 300 line-of-business applications hosted on the cloud or by partners and vendors outside of the internal corporate network. ADFS handles claim requests to verify identities and returns tokens to the requesting party to enable applications to verify the identity of a user with Microsoft Active Directory credentials. ADFS relies on federation servers that authenticate users against Active Directory and issue claims, as well as federation proxy servers that reside in the perimeter network in front of the federation servers. Clustered SQL servers store configuration data, as shown in Figure 4. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 10

11 Perimeter First Data Center Business Continuity Second Data Center ADFS Proxy ADFS Proxy ADFS Proxy Internal ADFS Federation ADFS Federation ADFS Federation SQL Server Cluster (primary SQL Server Cluster (mirror Figure 4 ADFS architecture To accommodate the additional traffic to the ADFS infrastructure due to Office 365, Microsoft IT more than doubled the size of its ADFS infrastructure. In July 2011, when mailbox migration first began to Office 365, Microsoft IT operated 12 proxy servers and 12 federation servers. As onboarding accelerated, Microsoft IT added more servers. In March 2012, after increasing server numbers, Microsoft IT operated 56 servers, divided evenly between proxy and federation roles. The key metrics Microsoft IT uses to determine capacity planning come from the following product group recommendations shown in Table 1. Table 1. ADFS performance metrics Description Threshold Details Authentication token requests/sec Below 60 The number of client requests a proxy server handles per second during peak load times CPU load Below 50% Average CPU load The authentication requests per second is the major threshold. Microsoft IT tries to keep this at an even load of During March 2012, Microsoft IT migrated over 14,000 mailboxes, Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 11

12 with a plan to monitor performance of the existing ADFS environment, and then add additional capacity. Figure 5 shows an average of requests per second for March. Figure 5 Average requests per second As Figure 5 shows, after deployment, the average requests decreased by half from about 20 per second to 14 per second. The number of auth requests per user depends on the location of the user when making the requests to Exchange Online. Microsoft IT modeled three types of users, as shown in Table 2 to understand projected server load and plan for ADFS capacity. Table 2 User patterns for messaging-related ADFS load considerations Type of user Average daily auths Peak daily auths Light: Moderate: Power: Min daily auths Details Low usage worker who accesses only from within the corporate network. Exits Outlook at end-of-day. Even mix of access from within the corporate network and travel/mobile. Majority of Microsoft users. Usage patterns have many more peaks due to travel, access from multiple devices. More remote access from home. These models served as a starting point to determine how many more servers to add in order to accommodate the additional traffic expected from migrating mailboxes to Exchange Online. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 12

13 For more information about designing and capacity planning for ADFS proxy and federation servers, see and Usage Patterns and Bandwidth An important aspect for the hybrid design is modeling user patterns and user behaviors to understand how they affect the bandwidth requirements and user experience. The models Microsoft IT used for ADFS capacity planning do not necessarily address bandwidth and client experience needs related to messaging, calendaring and other Exchange traffic. To address these needs, Microsoft IT abstracted several user types, as shown in Table 3. Table 3. Microsoft usage models for sizing considerations Activity Per Day Light Medium Heavy Power User Messages sent Messages received Average message size 50 KB 50 KB 50 KB 50 KB Messages read Messages deletes Outlook Web App log offs/logons Client Outlook ,300 KB/ day 2,600 KB/ day 5,200 KB/ day 7,800 KB/ day Outlook Web App 6,190 KB/ day 12,220 KB/ day 24,270 KB/ day 36,330 KB/ day Microsoft IT's considerations for bandwidth requirements based on the user models followed established best practices of evaluating the connectivity at each gateway and monitoring performance. As migrations increase, Microsoft IT continues to monitor latency, jitter, collisions, utilization, and other network metrics to spot gateways and locations that need improvement. For more information about bandwidth planning, see bandwidth-needs-determination-a-refresher.aspx. One more performance consideration is the location of users relative to the Exchange Online data center, and the latency and bandwidth available between users and the data center. This is relevant both for the initial onboarding migration, due to the gigabytes of data transferred, as well as for ongoing needs, especially as Microsoft workers increasingly rely on mobile devices and work from home and while on the road. Because Exchange Online relies on Internet infrastructure for mail traffic between office locations and the Exchange Online data center, performance and SLAs cannot be guaranteed. It is important to gather performance statistics from your environment. Two tools Microsoft IT uses for validating connectivity are and Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 13

14 Client Performance Microsoft users are accustomed to high performance levels with messaging, expecting all message delivery to complete less than 90 seconds, maintain 99.99% or higher availability, as well as deliver fast operations to read and manage schedules and items. In an on-premises deployment, Microsoft IT controls the messaging infrastructure and its dependencies because all traffic flows internally within the corporate network, or between users accessing internal Exchange servers over the Internet. A hybrid deployment introduces additional variables that affect performance because users accessing Exchange Online from within the corporate production environment do so over the Internet, same as mobile and remote workers. The differences among gateways, client devices, and connectivity in Microsoft locations mean that user experience at times may not be consistent among all sites. Microsoft IT looks at two factors when considering client performance: the MAPI RPC latency and the overall client system indicators, such as CPU, disk, and file fragmentation. RPC latency includes round-trip latency to the mailbox server and server-side RPC processing. A helpful tool for determining these values is the connection status dialog accessible by holding down the CTRL key, right-clicking the Outlook icon, and selecting Connection Status from the Outlook context menu. Microsoft IT uses the following thresholds when analyzing latency: Max Avg Proc Time (Exchange RPC Latency) = 25ms Max Network RTT Time (Network Ping Time) = 300ms Max Avg Resp Time (Exchange RPC Latency + Network Latency) = 325ms For more information about client performance, see Service Dependencies At its core, Exchange Server has always and continues to deliver messaging and calendaring capabilities. Yet, Exchange Server 2010 integrates with other services and applications such as SharePoint, the Office suite, and Lync Server, both on-premises and through Exchange Online. This integration along with ADFS and directory synchronization helps to facilitate the following hybrid Exchange capabilities. Delegate permissions for administrators To maintain the delegate permissions that administrators need to support managers and executives, Microsoft IT migrates manager and delegate mailboxes together. Delegate permissions do not persist in Exchange Online unless all affected mailboxes are migrated at the same time. Free/busy sharing and synchronized calendaring As part of federated delegation, free/busy information is shared between on-premises and cloud-based users. After Microsoft IT establishes a trust through the Microsoft Federation Gateway and configures a sharing relationship between on-premises and Exchange Online, it is possible to share free/busy data. The user experience is transparent because the Outlook client communicates with the local CAS server, which requests a delegation Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 14

15 token from the Microsoft Federation Gateway, impersonates the user, and makes free/busy requests on each user s behalf. Public folders Exchange Online does not support public folders. This is not an issue for Microsoft IT because users whose mailboxes are identified for migration do not rely on public folder functionality. For more information about public folder best practices in a hybrid deployment, see Unified messaging Exchange Online supports unified messaging features for Exchange, including voic , automated attendant, Outlook Voice Access, speech-totext voic preview in seven languages, and inline playback. Outlook Web App redirection In the initial hybrid implementation, Microsoft IT created a landing page for users who access the on-premises Outlook Web App URL that directs users to the Exchange Online URL. If a user accesses from within the corporate network, only one login is required, whereas from the Internet, users see the need to authenticate twice. While working through the challenges, Microsoft IT collaborated with the Exchange Server product group to suggest improvements to streamline the experience. Exchange Server 2010 SP2 incorporates the latest changes with improvements to the Outlook Web App experience for hybrid deployments. For more information, see Mail Flow Over the course of planning for and deploying the hybrid environment, Microsoft IT validated possible mail flow scenarios and developed best practices to streamline hybrid deployment for clients. Many of these configuration options are included in the Exchange Server Deployment Assistant and as improvements in Exchange Server 2010 SP2 on-premises. The routing configuration in a hybrid deployment is relatively straightforward. It comes down to having on-premises or Exchange Online be the authoritative environment, and then relaying s to the secondary environment. In a hybrid configuration, both the onpremises and the Exchange Online environment see each other as an internal, trusted environment. Figure 6 illustrates the configuration and mail flow. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 15

16 On-Premises Environment Office 365 Global Catalog Hub Transport Transport certificate subject: mail.microsoft.com Mailbox Hub Transport to chris TLS to greg to chris, greg to greg FOPE Transport certificate subject: mail.messaging.microsoft.com s Delivery queue Internet Exchange Online mailbox [email protected] On-premises user mailbox [email protected] Mail flow Recipient lookup TLS encryption to [email protected] [email protected] Figure 6 Message flow overview To enable mail flow, Microsoft IT configured a dedicated send connector on Hub Transport servers secured by Transport Layer Security (TLS). That traffic traverses the Internet and enjoys the following protection measures: Channel privacy Exchange 2010 forces TLS encryption for all messages by requiring that a SAN or fully qualified domain name (FQDN) on the associated Secure Sockets Layer (SSL) certificate for the sending server is configured as authorized on the receiving server. Receiver and sender authentication To protect against impersonation, Exchange Server 2010 uses an encrypted auth header and domain validation, including validating the certificate of the receiving server against a revocation list with the certification authority (CA). Exchange Server appends the auth header to messages to mark internal messages as trusted and authenticated, making messages and MailTips appear as internal in both Exchange Online and on-premises. The header works together with the certificates and send connector to ensure mail flow happens smoothly between Exchange Online and onpremises. Figure 7 illustrates the role of the auth header. Because Exchange Server appends the auth header to all internal communication, features such as OOF notifications and MailTips work seamlessly for users. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 16

17 On-Premises Environment Office 365 Hub Transport XOORG TLS Data Global Catalog Transport certificate subject: mail.microsoft.com Mailbox Hub Transport 2 1 Certificate Subject 2 FOPE Transport certificate subject: mail.messaging.microsoft.com s 3 1 Delivery queue Internet Exchange Online mailbox [email protected] On-premises user mailbox [email protected] To on-premises Recipient lookup 1. Add internal auth header TLS encryption 2. Verify certificate subject, promote if valid Mail flow on-premises to Exchange Online Mail flow Exchange Online to on-premises Figure 7 Auth header To Exchange Online 1. Add internal auth header 2. FOPE records sender certificate subject 3. Verify certificate subject, promote if valid The auth header is relevant in the following mail flow scenarios for Microsoft IT: flow between Exchange Online and on-premises When an on-premises user sends an to a user whose mailbox resides in Exchange Online, the on-premises Hub Transport server verifies that the SAN or FQDN of the SSL certificate matches the configured value. If the certificate subject is valid, then Exchange appends internal header to the and sends it to Exchange Online. The message bypasses the Edge server on premises. The reverse direction follows a similar path where the DNS and SSL configuration along with the send connector on the Hub Transport server enable encrypted mail to flow. The built-in features of Exchange Server give Microsoft IT the functionality needed to configure mail flow. s between Exchange and Internet hosts For other communication to and from Internet hosts, Exchange Online and on-premises use the standard Simple Mail Transfer Protocol (SMTP) mail flow as detailed in Forefront Online Protection for Exchange (FOPE) In the classic on-premises architecture, an Edge server running in a perimeter network provides initial mail filtering for anti-virus and antispam protection as well as SMTP relaying. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 17

18 For Exchange Online, FOPE provides a similar service. FOPE includes high accuracy SPAM filtering with over 98% of SPAM filtered, and 100% of viruses filtered by using multiple virusscanning engines. FOPE also gives Microsoft IT a control center for advanced policy rules and reporting. Although it is possible to use an Edge server on-premises for mail filtering and SMTP relay in a hybrid architecture, Microsoft IT uses FOPE. The first contact point of handling messages is very important in the overall architecture, especially in the dependencies required when not using FOPE. The Exchange Deployment Assistant addresses this importance in the guidance it provides and accommodates both scenarios for initial mail handling. For more information about FOPE, see Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 18

19 MIGRATING MAILBOXES TO EXCHANGE ONLINE The exact process that Microsoft IT followed in migrating to Exchange Online entailed many months of planning, validating scenarios, and working to improve the client and user experience. Because its mission includes real-world validation and early adoption of Microsoft technologies, the deployment process did not follow a more typical path. These efforts support customer needs. For example, before the changes introduced in on-premises Exchange Server 2010 SP2, the configuration requirements entailed over 50 distinct steps, which SP2 reduces to just six. As Microsoft IT migrates more mailboxes to Exchange Online, the migration velocity can increase from 5,000 to 15,000 mailboxes per month. At a high level, the deployment entailed making the following changes: Configure single sign-on As a recommended prerequisite to a hybrid Exchange deployment, the on-premises credentials and user data should be used to authenticate with Exchange Online. Microsoft IT already operated an ADFS infrastructure, and configured it to support Exchange Online. On the Exchange Online side, after signing up for Exchange Online and verifying domain ownership, Microsoft IT configured the Microsoft Federation Gateway to work with its ADFS infrastructure through a trust relationship. Synchronize directories and data In order to onboard user mailboxes, users must exist in Exchange Online. Microsoft IT configures directory synchronization to populate Exchange Online with users from the Active Directory environment. Configure DNS and certificates Exchange relies on DNS entries for autodiscover, which is necessary for a seamless online migration with no user interruptions. After migration, Outlook uses autodiscover to detect the mailbox move, and upon restart uses the Exchange Online service. Microsoft IT configured the MX records to point to FOPE. Deploy/configure necessary on-premises Exchange dependencies To enable the full range of Exchange features and services, such as mailbox search, Outlook Web App redirection, MailTips, free/busy sharing, message tracking, and archiving, Microsoft IT made the necessary on-premises configuration updates to work with Exchange Online. The Exchange Hybrid Configuration Wizard in Exchange 2010 SP2 automates many of the configuration steps. Verify mail flow The auth header is crucial to bypass filters and mark internal messages as originating from trusted sources. Microsoft IT configured and verified mail flow between Exchange Online and on-premises, as well as Internet hosts. For deployment steps and instructions to deploy a hybrid environment, the best practice is to use the Exchange Deployment Assistant, which includes the latest steps. To access the Exchange Deployment Assistant, see Migration Approach and Process One of the advantages of a hybrid infrastructure is that it enables Microsoft IT to move mailboxes to and from Exchange Online without affecting availability, performance, or the user experience. The same core messaging and calendaring functionality remains available to users during the move without service interruption. The migrations are made as online mailbox moves, so users do not need to synchronize data after migration. In practical terms, this means Microsoft IT may schedule mailbox moves at any time if all the dependencies and Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 19

20 prerequisites are met for preparing and configuring settings, as well as informing users. The overall process is as follows: Test usage scenarios. Before Microsoft IT migrates mailboxes, it performs end-to-end system testing that includes all possible usage scenarios. This testing helps to discover and remedy system configuration and integration issues. Although the engineering staff audits configurations, sometimes real-world issues arise, especially with new changes. Thorough testing also enables Microsoft IT to better understand the environment and build a decision matrix to identify the users who can move their mailboxes to Exchange Online. Scope mailbox migration Microsoft IT creates a list of potential users to be moved, gathers statistics about their mailboxes, and makes decisions about the migration order based on a decision matrix. This decision matrix depends on the business and IT needs. For example, Microsoft IT made the decision to simplify infrastructure and operational support by adopting the default configuration and reduce customization as much as possible. This may mean not introducing some features and functionalities. One of the examples is not migrating any mailboxes who are using a legacy telephone system and only migrating mailboxes with Lync 2010 Enterprise Voice to Exchange Online. This decision saves Microsoft IT third-party gateway costs and associated support overhead. It also simplifies the Exchange Unified Messaging configuration, and enables Microsoft IT to focus its efforts in driving Lync 2010 Enterprise Voice as the default telephony and collaboration platform. Microsoft IT is working to transition majority of mailboxes to Exchange Online to reduce costs and still offer users the best experience. Verify configuration This includes ensuring that Exchange Online is prepared with the appropriate objects, directory synchronization functions, and mail flows between Exchange on-premises and Exchange Online. This step also serves as a safeguard to verify that there are no schedules service windows or current outages with dependent services. Update user computer To ensure that users have the latest Outlook client version and required software such as Microsoft Online Services Sign-in Assistant, Microsoft IT uses System Center Configuration Manager (SCCM) to package the required software and deploy it on user computers. Migrate mailboxes After notifying users of the migration schedule, Microsoft IT migrates mailboxes and sends notices upon successful completion. For more information about determining how many mailboxes to migrate, the anticipated migration timeframe, and other migration performance details, see the migration performance guide at Phases The rate at which Microsoft IT migrates mailboxes is closely tied to the rate that improvements and change requests from previous phases are implemented as features. Between the phases, Microsoft IT allowed for a period of one to two weeks to implement changes and constantly improve the user experience and migration process. The phases were as follows: Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 20

21 Phase 1: Environmental validation The purpose of this phase is to discover and fix any system configuration errors and integration issues by creating test accounts and performing usage scenarios. Phase 2: Early adopter validation The early adopter volunteers troubleshoot, gather logs, and provide constructive feedback to the project teams. In this phase, Microsoft IT migrates 10 to 20 mailboxes per week, stopping at approximately 100 mailboxes. Phase 3: Expanded early adoption During the expanded early adoption phase, Microsoft IT migrated the accounts of 1,000 additional volunteers who are eager to explore new options in technology. The migration proceeded in phases, stopping when major issues are discovered and resuming upon resolution. Phase 4: Executive opt-in To stress-test the approaches developed, Microsoft IT reached out to executives to migrate entire teams and reach the number of mailboxes necessary to perform larger scale performance testing. In this phase, Microsoft IT also introduced a stabilization period of 21 days where no changes are made, and statistics gathered to gauge availability and stability. Phase 5: Company-wide signup Following team migration, Microsoft IT opened up signups to volunteers company-wide, having resolved any underlying high-severity issues. Phase 6: Company-wide adoption Once the hybrid infrastructure meets the shared goals of Microsoft IT, product developers, and other infrastructure team members, Microsoft IT plans to migrate all mailboxes to Exchange Online, unless there is a business need to remain on-premises. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 21

22 SUPPORTING USERS During the transition to a hybrid infrastructure, Microsoft IT minimizes support tickets by informing users and designing architecture with the goal of least user impact. The typical process for any Microsoft IT improvement project includes a focus on user education. This entails a broad, multimedia approach of making help available to users on their own terms, including the following: Online help Microsoft IT developed online help to answer frequently asked questions, provide user self-help capabilities, and inform users about working with Exchange Online by suggesting best practices. s detailing project schedule and status As a best practice, Microsoft IT informs users personally when a scheduled task affects them, and follows up after completing the task with status details. Updated knowledge for front-line operators The support and escalation path remains the same for users due to the centralized controls that a hybrid infrastructure offers. However, as part of preparing for mailbox migration, Microsoft IT collects incident details and transfers the resolution specifics to internal front-line operators as well as the support team for Exchange Online to aid in issue resolution. To help facilitate this knowledge sharing, Microsoft IT established a supportability team to do deep analysis of each ticket and identify trends in order to support and prioritize change requests made to the Exchange product group. Validation team Due to the need to validate many possible customer scenarios and features for all the scenarios, Microsoft IT created a dedicated validation team. This team has oversight to validate possible customer configurations, record findings, recommend improvements, and create best practices. Exchange Server 2010 SP2 onpremises incorporates some of the findings of this team as product improvements to simplify customer hybrid deployments. This team also validates features and functionality for Microsoft users to ensure a smooth transition process. Feedback loop When Microsoft IT migrated the earliest mailboxes, this was done with the intention to obtain migration and usability feedback. The early volunteer users relied on a feedback portal to give real-time feedback as a smile, frown, improvement idea, or issue. This feedback loop complemented the one-week and one-month post-migration survey users filled out to help Microsoft IT gauge overall user experience such as migration experience and usage performance. This helped Microsoft IT to identify improvement areas for infrastructure, configuration and product design changes. Self-help tool. Microsoft IT treats both on-premises and Exchange Online as a single service, and the helpdesk supports both groups of users. It is important to be able to identify the environment that hosts the mailbox, therefore Microsoft IT created a Web portal that provides information about the mailbox location, Outlook Web App link, ActiveSync, and other information pertaining to that user. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 22

23 LESSONS LEARNED AND BEST PRACTICES Over the course of designing, deploying, and operating a hybrid Exchange infrastructure, Microsoft IT learned many lessons for the best approaches in running a hybrid environment. Although some of these are applicable to the Microsoft production environment specifically, the following best practices apply to hybrid Exchange deployments in general: Use available migration tools and wizards Many of the findings that engineers, architects, and implementers made are implemented in the configuration wizard and supporting tools that Microsoft makes available to anyone using Office 365. Whenever an easier solution of configuration step may be automated or implemented as a product change, Microsoft IT worked to transfer their knowledge into a standard for all customers. Focus on core architectural elements At first glance, a hybrid infrastructure takes a potentially complex Exchange architecture and topology, and introduces additional configuration requirements, and management overhead. Once the underlying dependencies, such as ADFS, Internet ingress and egress, and network latency are established and configured with adequate performance, Microsoft IT found that a hybrid deployment still maintains centralized administration, and introduces little architectural complexity while preserving a unified user experience. Adopt a services-based perspective With Exchange Online, every aspect (SPAM/virus protection, Microsoft Federation Gateway, messaging, and so on) is provided as a service and not as a feature or component. It is helpful in understanding a hybrid architecture to consider some aspects of on-premises as service counterparts, to abstract the architectural elements and understand their dependencies and relationships. For example, as a counterpart to the Office 365 directory, there is Active Directory. In any overlap that happens between services, it is important to remember that there must be a way to achieve a single, synchronized version that is transparent to the user. Enabling technologies such as ADFS and the Microsoft Federation Gateway facilitate this seamless integration. Audit common sources of misconfiguration Microsoft IT investigates upstream and downstream possible causes to isolate root causes and remedy them. When troubleshooting typical on-premises components, there is not always a corresponding cloud counterpart, which makes it challenging to do direct comparisons and remedy issues. Microsoft IT proactively audits the most common possible issues as a preventative measure to reduce the troubleshooting necessary. One useful tool already mentioned to help with common troubleshooting and auditing tasks is the remote connectivity analyzer located at Identify send as relationships Whereas users may specify delegate rights, administrators assign send as rights to grant someone control over a specific mailbox. These send as rights do not synchronize automatically as you migrate mailboxes. Microsoft IT determines on-premises send as permissions through PowerShell before migrating mailboxes, and uses PowerShell scripting to apply the same permissions after migrating mailboxes to Exchange Online, Engage infrastructure team early Mailbox migration to Exchange Online results in e- mail traffic traversing the Internet across provider backbone routers instead of internal WAN networks and internal routers. This change may require increasing capacity and sizing of the Internet proxy egress infrastructure, ADFS, bandwidth, gateway IP Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 23

24 configuration if doing Network Address Translation (NAT), Exchange on-premises hub transport configuration, and so on. It is important to engage the various teams responsible for all these services early and carry out capacity re-engineering to ensure project success. For example, Microsoft IT discovered that a cause of intermittent client connectivity to be caused by a flood threshold of the proxy array, and quickly reached out to the associated team to engineer and implement a new solution. Support the support department With a new service, support personnel must be trained on possible issues, and how to isolate and troubleshoot root causes. Having tools that identify mailboxes as on-premises or in the cloud helps when isolating root causes. Practice change management. With new technology adoption, users generally want to start using the new and exciting features. Yet with messaging, there is a high expectation that the service needs to be reliable with high service availability, which may not be possible at very early deployment stages. Microsoft IT mitigates this by ensuring users have all possible collaboration tools so that when one service is not available, workers may continue to carry out their tasks. For example, when service is unavailable, users can continue to collaborate with colleagues through Lync 2010 via instant messaging or voice call. They may also work on documents via SharePoint or send documents via Lync. At Microsoft, many early innovators and adopters are very keen to be early adopters because service outages do not severely affect their ability to work. After Microsoft IT achieves stability with a new service, it migrates the rest of the company. This methodology satisfies all user needs, creates high satisfaction, gives Microsoft IT the ability to support the developers in testing, and create a better product. Communicate with users Active communication to users via Web portal, newsletter and s keep users excited about the program and informs them about new features or issues. Microsoft IT rewards and recognizes users who provides the most constructive feedback and support, which maintains user motivation and commitment to dogfooding additional products and services. Audit gateway configuration Microsoft IT audited two configuration details for gateways: flood thresholds for TMG gateways, and Outlook client port exhaustion when using NAT. For more information about TMG configuration, see Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 24

25 FOR MORE INFORMATION For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) In Canada, call the Microsoft Canada information Centre at (800) Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to: The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. Unless otherwise noted, the example companies, organizations, products, domain names, addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, address, logo, person, place, or event is intended or should be inferred Microsoft Corporation. All rights reserved. Microsoft, Active Directory, Forefront, Lync, Outlook, SharePoint, SQL Server, Windows, Windows PowerShell, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners. Deploying a Hybrid Messaging Infrastructure by using Office 365: Exchange Online Page 25