DISASTER RECOVERY OF EXCHANGE SERVER 2010 DATABASE

Transcription

1 DISASTER RECOVERY OF EXCHANGE SERVER 2010 DATABASE 1 SURESH L, 2 NIRMALA S GUPTHA 1 2 Reva Institute of Technology and Management,Bangalore, India -surechdb@yahoo.com,nirmala.sguptha@gmail.com Abstract:- Maintaining continuity seems like an unnecessary expense, consider the cost of suddenly losing communications with stakeholders, customers, vendors. is becoming ubiquitous and has become the standard tool for communication in many enterprises, big and small. Microsoft is the dominant player in the messaging platform market through its Exchange Server. has become a mission critical application for most businesses today and it has long been a challenge to backup and restore information. When disaster strikes your Microsoft Exchange Server organization, your first instinct may be to reach for your backup. However, restoring a backup may not always be the best course of action. That's because when a failure occurs it is important to restore mail flow as quickly as possible, and restoring a backup can be time consuming. It is usually better to restore mail flow first, and restore the rest of the data later. So it is imperative for companies to effectively backup and recover data and protect them from huge losses in productivity and downtime. The Exchange Server database is hosted in an edb (Exchange database) database file with the potential of getting corrupted. This can be a crippling event as all of your users will suddenly lose access to their mail stores, calendars and contacts. Recovery of the Mails items from the offline backup database without Exchange server installed using the Disaster recovery of Exchange Server. Corrupt Exchange Server database Recovery can be done by converting from dirty shutdown to clean shutdown and removing the Secondary indexes. With the help of powerful disaster recovery of exchange server database method is immensely useful in recovering and restoring the user mailboxes, which have been accidentally or intentionally deleted. Disaster recovery method shows preview of all the recoverable mail items, before actually recovery and scans the corrupt Exchange server 2010 EDB database and lists all the recoverable mail objects from the inaccessible mailboxes. I. INTRODUCTION Microsoft Exchange Microsoft Exchange server is the communication hub for most organizations. Crucial flows through this database continually, day and night; business agreements, negotiations, and conversations. As the system of record for , the Exchange server environment is at the core of most discovery activities The fundamental capabilities of Microsoft Exchange Server 2010 are impressive. Improvements to security, reliability, and scalability enhance an already road tested and stable Exchange Server platform. Along with these impressive credentials comes an equally impressive design task. Proper design of an Exchange Server 2010 platform will do more than practically anything to reduce headaches and support calls in the future. Many complexities of Exchange Server might seem daunting, but with a full understanding of the fundamental components and improvements, the task of designing the Exchange Server 2010 environment becomes manageable. Designing Exchange Server used to be a fairly simple task. When an organization needed and the decision was made to go with Exchange Server, the only real decision to make was how many Exchange servers were needed. Primarily, organizations really needed only and eschewed any bells and whistles. Exchange Server 2010, on the other hand, takes messaging to a whole new level. No longer do organizations require only an system, but high level of system availability and resilience and other messaging and unified communications functionality. After the productivity capabilities of an enterprise platform have been demonstrated, the need for more productivity improvements arises. Designing Exchange Server used to be a fairly simple task. When an organization needed and the decision was made to go with Exchange Server, the only real decision to make was how many Exchange servers were needed. Primarily, organizations really needed only and eschewed any bells and whistles. Exchange Server 2010, on the other hand, takes messaging to a whole new level. No longer do organizations require only an system, but high level of system availability and resilience and other messaging and unified communications functionality. After the productivity capabilities of an enterprise platform have been demonstrated, the need for more productivity improvements arises. Exchange Server 2010 is the evolution of a product that has consistently been improving over the years from its roots. Since the Exchange 5.x days, Microsoft has released dramatic improvements with Exchange 2000 Server and later Exchange Server Microsoft then followed upon the success of Exchange Server 2003 with some major architectural changes with Exchange Server This latest version, Exchange Server 2010, uses a similar architecture to Exchange Server 2007 but adds, extends, and perfects elements of Exchange Server design Exchange 2010 database consists of a database file (*.edb), a transaction log stream (*.log), and a checkpoint file (*.chk). 18

2 Transaction logs record all the changes that have been committed to the in memory database, whereas checkpoint files record which logged transactions have been written to the on disk database files. In Exchange 2010, log files are limited in size to 1 MB each.exchange 2010 maintains a single set of transaction log files for each database. The transaction logs play an important role in backup and recovery operations. You must handle these logs properly for a backup and recovery application that uses the Volume Shadow Copy Service (VSS) to be implemented correctly. MoMT is still a code name and will have been changed before Exchange Server 2010 is released. The Mailbox Server Role does not route any messages, it only stores messages in mailboxes. For routing messages, the Hub Transport Server role is needed. This latter role is responsible for routing all messages, even between mailboxes that are on the same server, and even between mailboxes that are in the same mailbox database. For accessing mailboxes, a Client Access Server is also always needed; it is just not possible to access any mailbox without a Client Access Server. Mailbox Archive Exchange Server 2010 now contains a personal archive; this is a secondary mailbox connected to a user s primary mailbox, and located in the same Mailbox Database as the user s primary mailbox. Since Exchange Server 2010 now supports a JBOD (Just a Bunch of Disks) configuration this isn t too big a deal, and the Mailbox Archive really is a great replacement of (locally stored).pst files. As far as Active Directory is concerned, its minimum level needs to be on a Windows Server 2003 level, both for the domain functional level as well as the forest functional level. This might be confusing, since Exchange Server 2010 only runs on Windows Server 2008 or Windows Server 2008 R2, but that s just the actual server which Exchange Server 2010 is running on! The Schema Master in the forest needs to be Windows Server 2003 SP2 server (Standard or Enterprise Edition) or higher. Likewise, in each Active Directory Site where Exchange Server 2010 will be installed, there must be at least one Standard or Enterprise Windows Server 2003 SP2 (or higher) server configured as a Global Catalog server. From a performance standpoint, as with Exchange Server 2007, the ratio of 4:1 for Exchange Server processors to Global Catalog server processors still applies to Exchange Server Using a 64Bitversion of Windows Server for Active Directory will naturally also increase the system performance. Mailbox Server role The Mailbox Server role is the heart of your Exchange Server 2010 environment. This is where the Mailbox Database and Public Folder Database are installed. The sole purpose of the Mailbox Server role is to host Mailboxes and Public Folders; nothing more. In previous versions of Exchange Server, including Exchange Server 2007, Outlook clients using MAPI still connected directly to the Mailbox Server Role, but with Exchange Server 2010 this is no longer the case. MAPI clients now connect to a service called MAPI on the Middle Tier (MoMT), running on the Client Access Server. The name II. EXCHANGE SERVER ARCHITECTURE Database Components The design of Exchange Server is based on standard database technology. The system relies on an embedded database engine that lays out the structure of the disk for Exchange Server and manages memory. Information Store The information store, which is the key component for database management in Exchange Server, is actually two separate databases. The private information store database, Priv.edb, manages data in user mailboxes. The public information store, Pub.edb, manages data in public folders. JET Database Engine Exchange Server databases are based on the JET format, which uses log files to track and maintain information. Microsoft JET is an advanced 32-bit multithreaded database engine that combines speed and performance with other advanced features to enhance transaction-based processing capabilities. The EDB File and STM File Microsoft uses the Extensible Storage Engine (ESE) Database File (EDB) format for storing s, Calendar data, Contacts, journals and attachments of mailboxes. The Checkpoint File The database engine maintains a checkpoint file called Edb.chk for every log file sequence in order to keep track of the data that has not yet been written to the database file on disk. The checkpoint file is a pointer in the log sequence that indicates where in the log file the information store needs to start the recovery in case of a failure. The checkpoint file is essential for efficient recovery. Database Components for Exchange 2010 The Exchange database architecture has changed in four significant respects in Exchange Server 2007 and 2010 The streaming database (.stm) file has been removed from Exchange

3 Longer log file names are used, thereby enabling each storage group to generate as many as 2 billion log files before log file generation must be reset. Transaction log file size has been reduced from 5 MB to 1 MB to support the new continuous replication features in Exchange The database page size has increased from 4 KB to 8 KB in Exchange 2007 and 32 kb in Exchange III. DATABASE CORRUPTION IV. DATABASE STRUCTURE Exchange Server uses fault-tolerant, transactionbased databases to store messages and directory information before it is applied to the database.the mailbox database is the primary repository of the Exchange Server information. This is where all the Exchange Server data is stored. On disk it is normally referred to as mailbox database.edb ; in older versions of Exchange Server it is called Priv1.edb. The Exchange Server Structure is shown in the below diagram. Exchange may experience a failure, such as a hardware failure, that requires the system to attempt to get back to a consistent state. Because there are different types of database corruption with differing symptoms, different tools and techniques are required to diagnose and fix problems. There are several types of disasters that can happen to an Exchange Server such as database corruption and server corruption of the OS or Application. In some cases you may be able to recover from a database corruption by running the repair operation on the affected data store or calling Microsoft and finding a way to fix a severe OS or Application error on the Exchange server. However, in the even that the that a system completely dies where neither of the above options work or you simply loose an entire machine including all of the OS, Applications, Log files, and Data due to some catastrophic disaster, following the guidelines mentioned above is what is going to save you. There are two types of corruption: Physical corruption At the lowest level, data can become physically corrupted on the disk. This is usually a hardwarerelated problem that always requires you to restore from backup. Logical corruption Typical logical corruption occurs at the database level. For example, database engine failure can cause index entries to point to missing values. Logical corruption can also occur at the application level, in mailboxes, messages, folders, and attachments. For example, application-level corruption might cause incorrect reference counts, incorrect access control levels, a message header without a message body, and so on. When disaster strikes your Microsoft Exchange Server organization, your first instinct may be to reach for your backup. However, restoring a backup may not always be the best course of action. Fig 1Exchange Server Structure The major areas of improvement in Exchange Server 2010 include many of the concepts and technologies introduced in Exchange Server 2007 but expand upon them and include additional improvements. Key areas improved upon in Exchange Server 2010 architecture include the following: Database Availability Groups (DAGs) The Exchange Server 2007 concept of Clustered Continuous Replication (CCR) has been greatly improved and replaced with a concept called Database Availability Groups (DAGs), which allow a copy of an Exchange Server mailbox database to exist in up to 16 locations within an Exchange Server organization. Because Continuous Replication is no longer limited to two servers, there is no longer any need for concepts such as Standby Continuous Replication (SCR) or Local Continuous Replication (LCR) because they are all superseded by DAG technology. Transport and access improvements All client access is now funneled through the Client Access server (CAS) role in an organization, which allows for improvements in client access and limited enduser disruption during mailbox moves and maintenance. In addition, Exchange Server 2010 guards against lost s due to hardware failures by keeping shadow copies of mail data on Hub and Edge Transport servers that can be resent in the event 20

4 of loss. Integrated archiving capabilities Exchange Server 2010 provides users and administrators the ability to archive messages for the purpose of cleaning up a mailbox of old messages, as well as for legal reasons for applying a retention policy on key messages. In addition, a second archive mailbox can be associated with a user s primary mailbox, allowing seamless access to the archived messages from OWA or full Outlook. Users can simply drag and drop messages into their archive folder, or a policy or rule can be set to have messages automatically moved to the archive folder. Access anywhere improvements Microsoft has focused a great deal of Exchange Server 2010 development time on new access methods for Exchange Server, including an enhanced Outlook Web App (OWA) that works with a variety of Microsoft and third party browsers, Microsoft ActiveSync improvements, improved Outlook Voice Access (OVA), unified messaging support, and Outlook Anywhere enhancements. Having these multiple access methods greatly increases the design flexibility of Exchange Server because end users can access via multiple methods. Protection and compliance enhancements Exchange Server 2010 now includes a variety of anti spam, antivirus, and compliance mechanisms to protect the integrity of messaging data. Exchange Server 2010 also includes the capability to establish a second, integrated archive mailbox for users that is made available through all traditional access mechanisms, including OWA. This allows for older archived items to be available to users without the mail actually being stored in the individual s mailbox, enabling an organization to do better storage management and content management of mail messages throughout the enterprise. The data within a database is organized in a Binary Tree, or B+ Tree. A Binary Tree can be seen as an upside down tree where the leaves are in the lower part. The actual mail data is stored in the leaves. The other pages only contain pointers. This is a very efficient way of storing data since it requires only two or three lookups to find a particular piece of data and all pointers can be kept in memory..binary Tree used in Exchange Server. The actual data is stored in the leafs One or more Trees in a database make a table. There are several kinds of tables in an Exchange server: Folders table; Message table; Multiple Message Folder tables; Attachment table; The tables hold the information we can see in Outlook (Web Access). The tables consist of columns and records, the columns are identified as MAPI properties, the records contain the actual information. Multiple Trees exist in one database and sometimes Trees need to be split when the Tree fills up. Exchange Server s internal processes will reorganize all information from one Tree into two Trees. This is called a split. It is not possible to predict how many Trees will be in a particular database, but it will be hundreds or thousands of Trees. This can be seen in the header of the database V. DISASTER RECOVERY WITH MICROSOFT EXCHANGE SERVER DATABASE After the text edit has been completed, the paper is ready Backup is an integral part of any application and Exchange Server is no exception. Proper disaster recovery plan is highly recommended for any organization to avoid data and communication loss. Exchange Server storage groups should also be backed up together with databases as they share transaction logs. Exchange can be backed up online without the need to shut down any Exchange service or make server stores offline. Brick-level backups can be also preferred to back up individual mailboxes which use a MAPI connection to the Exchange Server store. However, the backup takes longer. Exchange store sizes also play a deciding role during Exchange disaster recovery. Bigger stores take longer time for backup and recovery. They must be adequately sized to meet expectations set by servicelevel agreements (SLAs). Exchange Server Enterprise Edition provides an extra advantage. Multiple storage groups and message stores can be used in this edition to disperse mailboxes in a manner that results in smaller stores. Recovery from Backup database without Exchange Server installed on a computer using the below Mail EDB Algorithm. Extensible Storage Engine Mail EDB Algorithm uses Extensible Storage Engine functions for recovery of mails from offline exchange database 21

5 The Extensible Storage Engine (ESE) is an advanced indexed and sequential access method (ISAM) storage technology. ESE enables applications to store and retrieve data from tables using indexed or sequential cursor navigation. It supports de normalized schemas including wide tables with numerous sparse columns, multi-valued columns, and sparse and rich indexes. It enables applications to enjoy a consistent data state using transacted data update and retrieval. A crash recovery mechanism is provided so that data consistency is maintained even in the event of a system crash. It provides ACID (Atomic Consistent Isolated Durable) transactions over data and schema by way of a write-ahead log and a snapshot isolation model. Transactions in ESE are highly concurrent, making ESE useful for server applications. It caches data to maximize high performance access to data. In addition, it is lightweight, making it useful for applications that serve in auxiliary roles. ESE is a DLL used by the Exchange store process to store records and create indexes within the Exchange database. It is the intermediary technology between the Microsoft Exchange Information Store service and the actual database. ESE allows applications to store records and create indexes to access those records in different ways. Although ESE does not accept structured query language (SQL) requests directly, it takes transactions passed to it from the client, in this case Exchange, and is the component that is responsible for the actual manipulation of the data. Because the engine is multithreaded and based on JET technology, it is optimized for fast data storage and retrieval ESE is for use in applications that require fast and/or light structured data storage, where raw file access or the registry does not support the application's indexing or data size requirements. It is used by applications that never store more than 1 megabyte of data, and has been used in applications with databases in extreme cases in excess of 1 terabyte, and commonly over 50 gigabytes. There are three ESE components in memory: transaction log buffers, data cache, and version store. There are two components on disk: log files and database files. Transactions move through the components as follows whether or not a backup is taking place: accessed by MAPI clients, and after a piece of data is moved to the.edb file and transformed into the Exchange native format, it stays that way. The.stm file houses all the content streamed over the Internet. Internet mail destined for review by a MAPI client first arrives through SMTP to the.stm file before it is promoted to the.edb file. If a message originates from another Exchange server, but is transported by SMTP, it is streamed into the STM and goes through an immediate promotion to the EDB. If the mail is never accessed by a MAPI client, it will stay in its native form in the.stm file. The splitting of these two files and the roles they play significantly reduce the hefty conversion overhead legacy Exchange products endured. VI. DISASTER RECOVERY OF EXCHANGE SERVER DATABASE Disaster Recovery of Exchange server database helps in getting the inaccessible user mailboxes by repairing the damaged Exchange Server database files. Mail Edb Algorithm recovers mailboxes, Repairs corrupt EDB files and restores various mail items, including s, contacts etc. The Algorithm is the first algorithm in the exchange server recovery to extract the mails from the offline database in any operating systems without the exchange exchange server installed. Disaster recovery method opens un mounted Exchange Mailbox OR Public Folder Information Stores (EDB, STM, LOG set) without using Recovery Storage Groups, (RSG's), ExMerge or setting up a dedicated recovery server. Simply browse to the location of the un mounted EDB for Exchange After scanning and repairing the damaged Exchange database file using the Disaster recovery method, it lists all the mailboxes with their recoverable mail objects in an Exchange tree. Supports EDB recovery from 64bit Exchange server databases. Repairs and recovers Unicode formatted EDB files.. Compatible with Windows 7, Vista, Server 2003 and XP The Corrupted Database can be recovered with converting the Dirty shutdown to state clean shutdown by replacing the dirty shutdown bits by clean shutdown bits in the first 16kb pages in Log buffers Log files Cache buffers and version store Database files The.edb and.stm files combine to make up the Exchange database. The.edb file stores all the data 22

6 In Message Table With reference id, Get the Mails in RTF and HTML Format.. In Message table, also get the Calendar data in vcal format and get Attachment reference id.. In Attachment table get the Attachments for corresponding Mails. Mails can be recovered by using the Extensible Storage Engine functions using the following algorithm Advantages of cdisaster recovery Open and Repair corrupted EDB files. Eliminate Brick-Level / Mailbox Backups NO changes to your Exchange environment Eliminate the need to use Recovery Storage Groups (RSG) and ExMerge Operating system independent. Avoids Brick level backup of Exchange Server mailboxes Issues in Disaster recovery Mail Algorithm Identification of row index for in MSG table for faster retrieval of mail. Offset movement for attachment files need to be studied. Mapping of exchange 2010 database tables to Mailboxes. Conversion of s to Pst format. Fig 3 Flow of opening Tables for Mail retrieval Mails can be recovered by using the Extensible Storage Engine functions using the below Exchange disaster recovery method Set the Page size for exchange server database 32kb depending on the exchange server format. Open the Mailbox table to get the Mailboxes of Exchange Server. Get the Reference id of Mailboxes for identifying the Folders of Mailboxes Open Folders table and using the reference id match the Folders of Mailboxes, Get the Table id to get the Folder summary of Mails. In Folder Summary table (Message headers table), get the Mails of Particular Folder such as Subject, To, From, Date of Mails and Get Mail Reference id. VI CONCLUSION Exchange disaster recovery method recovers mailbox items from a offline exchange server database. This eliminates having to perform timeconsuming, expensive single mailbox brick level backups. Recovery Method recovers complete mailboxes, folders or individual messages, Attachments, contacts, calendar items, Public Folders database items. Exchange disaster recovery converts dirty shutdown to clean shutdown and Removal of Secondary indexes to read the databases pages has explained. Recovery of Exchange server 2010 database, implemented by using the recovery method and recovered mails are converted PST format. VII REFERENCES [1] [2] [3] chitecture_and_design.aspx [4] [5] *** 23