Exchange 2007/2010 Journaling for Cryoserver

Transcription

1 Exchange 2007/2010 Journaling for Cryoserver How to set up Exchange 2007 / 2010 Journaling, to capture an archive copy of all inbound/outbound and internal mail, for delivery to the Cryoserver system February 2012 Forensic & Compliance Systems Ltd +44(0) info@cryoserver.com

2 Contents Overview... 3 Exchange 2007/2010 Journaling Terminology... 4 Envelope Wrapper... 4 TNEF (Rich-Text) Data... 4 Public Folder mail... 4 Duplicated Multiple Endpoint Journaling... 5 Configuring Exchange 2007/2010 for Journaling... 6 IMAP Collection... 6 Direct SMTP Journaling... 6 Step 1 Creating a Send Connector... 7 Step 2 Creating a Contact for the Cryoserver Address Test Step 3 Enabling Journaling Enable Global Journaling Mail-Store level journaling Remote Domains Controlling the classes of SMTP mail sent to Cryoserver Migrating Exchange 2003 to 2007/ Recommended migration steps Problems & Solutions Exchange Exchange 2010 journaling bug Journaling Loop P a g e

3 Figures Figure 1 - Adding a new Send Connector Figure 2 - Modifying the properties of a Send Connector Figure 3 - The address space and cost for the Cryoserver Connector Figure 4 - Entering the IP address of the Cryoserver Figure 5 - Cryoserver does not require any SMTP connection security Figure 6 - Adding a Contact for Cryoserver Journal mail Figure 7 - Modify the Contact settings to prevent Rich Text and Address Book visibility Figure 8 - Ensure the Contact is not restricted Figure 9 - Turning On Journaling at Hub Transport level Figure 10 - Turning on Journaling at the Mail Store level Figure 11 - Remote Domains (optional) Figure 12 Remote Domains - Another way to ensure rich text is not sent to Cryoserver P a g e

4 Overview A copy of every flowing through an Exchange (inbound, outbound and internal) may be captured for Archival purposes using a feature of Exchange called Journaling. There are two typical ways to get the Journal copies of every mail to reach the archive system: 1. Journal mail can flow direct to the Archive, over SMTP. Typically a connector will be needed to route the mail to a specific server inside the company. For cloud based archives, a standard address could be used and no connector would be needed. 2. Journal mail can flow to a standard Exchange User Mailbox. The archive would then need a method to extract the mails from this mailbox. Cryoserver uses an IMAP collection service. In Exchange 2007 & 2010, Journaling can be enabled for the whole organisation at the Hub Transport level. Or it can be enabled only for specific Mail Stores (as was required in Exchange 2000/2003). 3 P a g e

5 Exchange 2007/2010 Journaling Terminology Journaling is a feature of an system that records a copy of every original that is sent or received. To perform the Journal copy, the must traverse the Message Transport Agent (MTA). This will include inbound & outbound and internal , but will not include moved from one place to another in Outlook using, for example, drag-and-drop. Nor will new copies be created for that are edited by the end user in Outlook. Envelope Wrapper The Journal copy of each will be in a slightly altered format. The original will become an attachment to a new . This wrapping will contain the list of final recipients including: Bcc recipients All names from distribution groups The actual recipient after any redirection rules. This Envelope wrapper is vital, as otherwise the final recipient data would be lost from the archive indexes. NOTE: Under Exchange 2000/2003, this feature was enabled via the optional exejcfg utility from Microsoft. Under 2007 onwards, the Envelope wrapper is always added. TNEF (Rich-Text) Data TNEF (Rich Text) is a way for Exchange to pass an internal database entry representing any Exchange object ( , calendar entry, appointment, note, contact) to another Exchange system over the internet. It is NOT intended to be used outside of an Exchange to Exchange link. However, some circumstances cause this format to be used incorrectly. A TNEF can be identified as one that contains an attachment called winmail.dat or win.dat, and has a MIME type of application/ms-tnef or application/vnd.ms-tnef. NOTE: The winmail.dat IS the including attachments and original headers and so on. Cryoserver is able to decode simple TNEF s, so that they are search and displayable. But this is most certainly not a useful format for archive mail, and if discovered it should be turned off using the instructions shown later in this document. Public Folder mail Exchange 2007/2010 will journal to and from Public Folder accounts. This is unlike Exchange 2000/2003, which did not. Duplicated Unlike Exchange 2000/2003, Exchange may journal a single multiple times each copy with a different set of recipients in the envelope part. Due to the nature of Cryoserver, we cannot update an existing with additional recipient data and therefore we expect these mails to be duplicated within Cryoserver. For example: An sent to a local user, a local distribution group and an external recipient MAY result in 3 Journal Copies being sent to the archive. Exchange may defer the expansion of distribution list to a later time meaning that the archive may get copies for: 4 P a g e

6 1. The local user (s) the envelope wrapper listing ONLY these recipients. 2. The distribution lists the envelope warpper listing all actual recipients of these lists 3. The external recipients the envelope wrapper listing only the external addreses This is unlike Exchange 2000/2003, where mail is de-duplicated as it is dropped into the Journal Mailbox. Multiple Endpoint Journaling Normally when you use journaling you will set a single journal end-point (the archive). No matter how many Exchanges are in the domain, mail will only be Journaled once (allowing for the duplicate case discussed above). If you want two separate Archives, then you can set two different Journal end-points. You can only do this by setting Journaling at the PER MAILSTORE level. Now, when mail is sent from users in one mail-store to users in another mail-store, then Exchange will compare the Journal End-Points for the Sending and Receiving ends. If different, it will Journal separately for each end point. Thus the two archives will hold duplicates of some . You might need to do this where you have Exchanges in different countries, and each country wishes to archive to their own local systems. 5 P a g e

7 Configuring Exchange 2007/2010 for Journaling There are two ways to get Journal mail from an Exchange to the Archive. Either by direct SMTP delivery, or by IMAP collection. This section shows the configuration for both techniques. IMAP Collection In this case, Journal to a local user mailbox in the Exchange. Then use the Cryoserver IMAP/POP3 mail collector service to extract the journal mail from this mailbox. The collector uses a Read-And- Delete loop, so the mailbox should not grow in size. Step 1 Create a local user mailbox in the Exchange. Microsoft would recommend this to be on the least-loaded server / mail-store, or preferably the only mailbox in the store. Ensure that the mailbox has no/few restrictions applied to it (e.g. mail & mailbox size limits). Step 2 Enable Journaling (see later) to this user mailbox. Step 3 Install and enable IMAP service in the Exchange hub server. IMAP is no longer installed by default. When it is installed, it now requires TLS or SSL encrypted connections. Step 4 Add an IMAP collector connection in the Cryoserver. This is the recommended configuration for cloud based systems and is also suitable for many other situations. It has the advantage of surviving a prolonged period where the archive system is unable to collect the journal mails (as they just gather in the journal mailbox). Direct SMTP Journaling Unlike Exchange 2000/2003, where journal mail should flow to a local user mailbox in the exchange before being routed to the archive system, Exchange 2007 onwards will allow Journal mail to be delivered (over SMTP) direct to the archive system. Configuring Exchange 2007/2010 for Journaling involves these steps: Step 1 Creating a connector to route mail to Cryoserver. Step 2 Creating a contact for the Cryoserver address Step 3 Enabling Journaling, globally or per-mail-store. The key elements are: 1. To set an UNLIMITED path for journal mail. It must be able to journal all regardless of any limits (size and recipients) set elsewhere. 2. To ensure mail is NOT delivered as Rich Text. This is an internal Microsoft only format which should not be used outside of an exchange environment. Cryoserver does its best to handle this format but it should be avoided. 3. To ensure journal mail flows direct to the Cryoserver system without passing through any extra hops like mail filters; or leaking out to the public domain. By using the complianceinternet.co.uk address, any public-bound journal is guaranteed to be undeliverable. 6 P a g e

8 Step 1 Creating a Send Connector A connector creates a path for the exchange system to send outgoing mail for a specific domain (the part after symbol) or range of domains. Every recipient is checked, and where it matches a connector, a copy of the will be delivered to the specified end-point. In this case, the end-point will be the Cryoserver. If no connectors exist already, then a default connector must be created first. In this case, it will route all standard outbound mail (Address Space = * ). It MUST have a cost set to 2 or more this will prevent it from trying to route mail that would otherwise match any other connector (such as the Cryoserver one). The default connector will use the DNS MX records to determine the delivery routing. If you are upgrading from an older Exchange, and a Cryoserver Connector already exists PLEASE ADD A NEW CONNECTOR. Do NOT alter any existing one. The Cryoserver connector will route journal mail usually sent to cryouser@complianceinternet.co.uk to the Primary Cryoserver. A Cryoserver appliance, by default, uses the Postfix system to accept incoming SMTP mail for the complianceinternet.co.uk domain. Unless the Cryoserver system is modified for specific reasons, the mail will be rejected for any other domain is used. Also Note: This complianceinternet.co.uk domain has been registered by Cryoserver and does not have any MX records thus preventing mail from routing in the public address space. Use the following details and screenshots to guide you through the process. 1. Name the connector so that you understand its purpose. It is for your own records. The obvious name is Cryoserver. The Intended use setting determines the permissions that are applied to this connector. Set this to custom, though any of the options would suffice. 2. The Address Space lists the domain name(s) for which outbound with be delivered using this connector. Enter complianceinternet.co.uk. If your system uses Multi-Tenant features of Cryoserver, then you may tick the include all subdomains. That will allow you to use cryouser@company-tag.complianceinternet.co.uk. The cost must be set to 1 or ANY number less than the Default Connector (the one that handles normal outbound mail). 3. The Cryoserver is the smart host. Enter the IP address or DNS name of the Cryoserver. 4. Source Server: In a larger organization, you may have Exchange servers distributed over a wide area (e.g. different countries). Select the server(s) for which you need this connector to apply. After adding a new Cryoserver connector, it is worth modifying its properties to remove the size limit. 7 P a g e

9 Figure 1 - Adding a new Send Connector Figure 2 - Modifying the properties of a Send Connector 8 P a g e

10 Figure 3 - The address space and cost for the Cryoserver Connector Figure 4 - Entering the IP address of the Cryoserver. 9 P a g e

11 Figure 5 - Cryoserver does not require any SMTP connection security Step 2 Creating a Contact for the Cryoserver Address When adding the Journaling rule, you must select a valid user Mailbox (internal journaling) or a Contact (external journaling). Therefore, to journal to the Cryoserver a contact must be added to Active Directory to represent the Cryoserver Address. The Cryoserver address is cryouser@complianceinternet.co.uk. In Exchange 2007 you can add the Contact within the Exchange Management Console: Test You can test the path from Exchange to Cryoserver by sending a test to the Cryoserver Contact that was set up. View the monitoring page in Cryoserver (either via super-user web, or in version onwards, you can just access: name>/cryoserver/monitor.jsp). You should see this mail arrive in the spool queue and be processed. 10 P a g e

12 Figure 6 - Adding a Contact for Cryoserver Journal mail Figure 7 - Modify the Contact settings to prevent Rich Text and Address Book visibility 11 P a g e

13 Figure 8 - Ensure the Contact is not restricted Step 3 Enabling Journaling There are now many more options for Journaling in Exchange 2007/2010. In most cases however, the rule should simply be set to Journal ALL MAIL. Now check that you can see mail being delivered to the Cryoserver. Enable Global Journaling Set this at the Hub Transport / Journal Rules tab. 12 P a g e

14 Figure 9 - Turning On Journaling at Hub Transport level Mail-Store level journaling If you have an Enterprise addition of Exchange with multiple mail-stores representing different business units, then you may wish to enable journaling on a range of Mail Stores, instead of the whole exchange. Alternatively, if you find that sent looks corrupt when viewed in Cryoserver, then your Exchange 2010 may be showing the bugs relating to Journaling (it sends invalid formatted SMTP mail in the journal feed). One solution may be to set journaling at the per-mail-store level rather than at the global hub level. 13 P a g e

15 Figure 10 - Turning on Journaling at the Mail Store level Remote Domains Controlling the classes of SMTP mail sent to Cryoserver There is an optional facility that sets the classes and formatting of outbound mail from the Exchange. You may wish to create a specific setting for the mail flowing to Cryoserver which is different to the standard mail flow. Figure 11 - Remote Domains (optional) 14 P a g e

16 Figure 12 Remote Domains - Another way to ensure rich text is not sent to Cryoserver 15 P a g e

17 Migrating Exchange 2003 to 2007/10 If you are migrating from Exchange 2003 to 2007 or 2010, then the following tips should prove useful. Your Exchange 2003 should be journaling to a local user Mailbox (the Journal Mailbox ). Please check how mail is being moved from this mailbox to the Archive system. It will be one of: 1. Via a forwarding rule, set up using an Outlook client; OR 2. Via the IMAP collector feature in Cryoserver. The steps that will be shown below will not affect or alter this existing system. After the migration, this OLD system can be simply switched off. The OLD Journal Mailbox ( & connector, if any) SHOULD NOT BE MIRGRATED or touched in any way. Just ensure that at the end of the procedure it is empty and remains so (i.e. that no journal mail flows to it any more). With Exchange 2007 or 2010, journaling mail can flow directly to the Cryoserver or it can be delivered to a local mailbox for collection by Cryoserver (same as Exchange 2003). We recommend that the Mail Collection feature is used for Migration purposes, as explained next. Recommended migration steps This is how we would recommend that the Journaling facility is migrated over to the new Exchange: 1. Set up and configure your new Exchange. Apply any service packs! 2. Add a new Journal Mailbox to the New Exchange. 3. Install the IMAP service on the NEW hub transport server. 4. Enable Journaling on the NEW exchange to the new Journal Mailbox (at either Hub or Mail- Store level). 5. Configure a NEW IMAP collector in Cryoserver to collect mail from the new Journal Mailbox. 6. Test and check that mail flows correctly. Send a mail direct to this new mailbox, and search for it in Cryoserver. At this point you can migrate user mailboxes to the new Exchange. However the Exchange now has TWO journal end-points a mailbox on the OLD Exchange, and one on the NEW Exchange. This will cause a small level of duplication. If the migration is likely to occur over a very short period (1 or 2 days), then this duplication can be ignored. For longer migrations, we would recommend that the OLD Exchanges Archive all.. setting located in the property sheet of each and every Mail Store should be switched to journal to the NEW journal mailbox. 7. Migrate your user mailboxes to the New Exchange. 8. Make the NEW Exchange the Bridgehead (receiving/delivering the mail to the outside world) as soon as practical. Try to remove the Ex2003 as soon as possible from this role. 9. Use the Cryoserver Admin area to change the Outbound and Alerts settings so mail raised by Cryoserver is now routed to the NEW exchange. 10. If applicable, Remove the IMAP collection connection to the OLD exchange (once it has been de-commissioned). 16 P a g e

18 Problems & Solutions Exchange 2007 There was a bug with the original Exchange 2007 journaling (fixed in Service Pack 1) which means that the original in the attachment is formatted in a Microsoft specific format known as Transport Neutral Exchange Format (TNEF). Cryoserver has been adjusted to accept and re-format these s so that they become fully readable. Please use the latest set of service packs to ensure that your Exchange is free of these issues. Exchange 2010 journaling bug Under certain conditions in Exchange 2010 SENT mail becomes corrupt in the Journal feed, when sent direct to Cryoserver. It seems to affect systems upgrading from Ex2003, where both Exchanges perform journaling. Ex2010 SP1 was released in Sep 2010 and this corrects the bugs in this area. Alternative solutions include: Set Journaling at the Mail-Store level, not at the global hub level. Journal to a standard Ex2010 user mailbox and set Cryoserver to collect the using IMAP. Journaling Loop If your Archive grows dramatically, then try to find any potential Journal Loops. A loop can occur if the Journal mail is sent direct to the Archive via SMTP but this is routed via a gateway server that may also have a Journaling feature. This could cause each and every journal copy destined for the Cryoserver to cause another journal copy, which could then cause another copy and so on. This is very rare and an easy solution to try is to use the IMAP collector service instead. 17 P a g e