Archiving Technical Overview. A Technical Overview Paper Describing Key Functions And Capabilities Of The Managed Archiving Service

Transcription

1 A Technical Overview Paper Describing Key Functions And Capabilities Of The Managed Archiving Service

2 Contents Introduction Feature Overview The Archiving Process Archiving Boundary using the Filtering Service Archived Audit Trails (meta-data extensions) Archiving Internal Blind Carbon Copy (Bcc) Routing Journaling Routing Messages to the Managed Archiving Service Securing Connectivity Infrastructure Redundant Infrastructure Data Availability Mirrored Storage Availability of Data Access Data Security Data Store Management Retention Periods Storage Management Aspects of Scalability User Types Compliance Interface End User Interface Summary About Webroot Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

3 Webroot s Archiving Service is designed as a mission critical service, offering organizations the option to quickly and easily deploy an archiving solution requiring no additional hardware or software. Driven through compliance requirements, business continuity and the need to continually manage storage costs, the Archiving Service offers a fully featured offering that is able to meet the requirements of all organizations, both large and small. Although simple to deploy, prospective purchasers will wish to better understand solutions that are handling their sensitive data. This paper provides a high level technical overview of the offering which aims to address the key technical concerns that any organization may have. Feature Overview The Managed Archiving Service is able to archive messages passing through the Webroot Managed filtering Service, as well as capturing internal mail traffic sent through mail servers such as Exchange, Sendmail, Postfix, etc. If using Exchange as the internal mail server, the solution takes advantage of Exchange extensions to preserve Bcc recipients and also record the authoritative sender and recipient envelope addresses. A fully compliant data storage and access solution is provided supporting the three key criteria for Data Protection: data protection through encryption, restricted access and audited access. All data is encrypted when stored within the service, ensuring that the privacy of the data is both retained and protected from malicious attack. Restricted access to the data is provided through a secure Web interface which authenticates individuals within the context of service-defined user credentials. Finally, any search requests made to the system are audited. If a user has permission to search within accounts other than their own, they must quote a reason for the search and acknowledge the terms and conditions of the search. Data Custodians can be notified by to ensure that any and all search requests are proactively monitored. An audit event log is provided with a search tool that allows authorised individuals to review recorded audit actions. The service provides a comprehensive set of search tools designed to assist the user in finding the required information in a fast and efficient manner. In addition to searches based on addresses, dates, subjects, etc, enhanced indexing facilities enable users to perform full text searches on messages bodies and attachments. When messages are indexed, common compression types, such as zip files, are decompressed whilst all enclosed contents (Word, Excel, PDF, RTF files) are indexed. A tiered search interface enables the user to approach the search task from a simple interface which ranges from a single intelligent entry box to an advanced interface which allows detailed search criteria to be constructed over a multitude of indexed fields. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

4 The search results interface provides a message based view, allowing the user to view all items at a message level, along with a document view which enables the user to view the search results according to the sought document type, revealing the actual attachment or message body that was found by the search. Tools are provided to restrict search results to those that are relevant to the user, e.g. by document type, relevance, sort order, etc. Archive capability is also provided for the End-User, which is able to query and view his or her messages in exactly the same way as the compliance administrator can. Access is provided via a Web Interface the only difference being that End-User access is not audited. End-Users may view messages and, if required, can release messages back for delivery to their inbox. The same archive store is used for both compliance and End-User access, thus minimizing storage overheads. Much of an organizations data will be stored within local PST folders. The service provides the capability to import PST folders into the archive. This is achieved by enabling the End-User or organization to upload PST files to the Managed Service, where it will be imported. All messages are stored on mirrored disk farms across two data centres, thus protecting against loss of data in the event of a disaster at either one of the data centres. Each disk farm is RAID enabled to provide maximum up-time. Messages are distributed to the disk farms in real-time. Search engine technology more commonly associated with internet search engines is used to implement a highly scalable and distributed indexing system, which is able to support searches over many terabytes of data at lighting speed. The Archiving Process The Managed Archiving Service has been designed to operate transparently with no hardware or software required on-site. Archiving can take place in two locations; either on the boundary of the organization or internally. INTERNET YOUR NETWORK FILTERING AND MANAGEMENT SYSTEMS ARCHIVE QUEUE TLS JOURNAL SERVER SEARCH HTTPS STORAGE ARCHIVE DIRECTOR INDEX STORAGE ARCHIVE DIRECTOR MIRRORED DATA STORE INDEX Figure 1 - Archiving Process Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

5 If an organization has subscribed to the Managed Filtering Service for virus, spam, etc, then both inbound and outbound messages can be archived as they pass through the mail filters. Alternatively, if users haven t subscribed to the filtering service, or if they wish to archive internal as well as boundary , the system can be configured to capture generated through journaling or Blind Carbon Copy (Bcc) Routing. Archiving Boundary using the Filtering Service Archiving boundary that enters and leaves the organization is, for most, a fundamental requirement. In light of this, the function to archive boundary is a built-in component of the Managed Filtering Service. By simply enabling the archiving facility for each domain, all messages passing through the filters, in either direction, are also routed to the archive service. Rules can be defined for each domain, enabling policy decisions to be applied to certain classes of information. For example, messages detected as spam may or may not be archived and, if archived, it is possible to specify for how long. The flexibility to make such decisions enables a stricter compliance model to be adopted which balances the need to retain information with the need to efficiently manage storage space. Archived Audit Trails (meta-data extensions) Much of the time, the filter service is deleting or modifying mails as they pass through the filters. However, just because a message is filtered out, doesn t necessarily mean that it shouldn t be archived. Moreover, once a message is archived, it is important to record what actually happened to the message. The filtering service therefore offers a number of critical extensions that form the basis of an archived audit trail. Specifically, attributes are stored with the archived message that indicate what type of content was detected, e.g. spam, virus, etc, what the delivery outcome for each recipient was, including an indication of any additional recipients that were added to the message by the content rules, and retention policies. This information is available to the Compliance Officer when the message is pre-viewed, and assists in supporting claims relating to nonrepudiation of mail for example. Archiving Internal Intra-organizational mail generated between mailboxes within an organization can be captured and archived by the Managed Archiving Service. The process to archive internal is, in fact, very simple. Firstly, mail messages must be captured using one of two techniques; either Journaling or Blind Carbon Copy (Bcc) Routing. Secondly the captured messages are then routed via the organization s infrastructure to a virtual mailbox located within the Managed Archiving Service data centres, using standard SMTP protocols. The approach was selected to comply with the core design ethos of being able to provide a managed service that requires no additional hardware/software on site, thus reducing risk, complexity and cost. Furthermore, the approaches that have been taken allow a wider set of mail systems to be targeted. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

6 Blind Carbon Copy (Bcc) Routing The most basic form of capture is to use Blind Carbon Copy (Bcc) Routing. Effective use of this mechanism will be dependant upon the facilities provided by the organization s internal mail servers. Bcc Routing works by configuring the mail server to automatically add an additional delivery recipient to every routed . The additional delivery recipient is an address that is hosted on the Managed Archiving Service, thus causing a copy of the message to be routed to the service via standard SMTP. Journaling Journaling is a capability that is commonly associated with Microsoft Exchange. Provided as a built-in capability of Exchange, it was specifically designed to capture a copy of every message that is exchanged between individuals located in an Exchange Server environment. In its most basic form, it is simply a Blind Carbon Copy Routing solution, which makes another copy of the message which is routed to a mailbox for servicing by an archiving product. However, recent compliance pressures from the US Financial Services market have forced a number of extensions to the core journaling service, which resulted in an enhanced Envelope Journaling solution. Similar to the Bcc solution, Envelope Journaling delivers a copy of the original message to a mailbox, to which additional meta-data is added. This has two key advantages over the Bcc Routing mechanism: > The real envelope senders and recipients are retained. Using Bcc Routing, the best guess on who sent and received the message is based on the headers of the message which can be spoofed, non-existent, or different to the envelope. > bcc recipients specified by the original sender of the message (not to be confused with Bcc Routing) are also retained. The Managed Archiving Service supports the Envelope Journaling extensions and is therefore able to extract the relevant meta-data in order to correctly re-create, store and index the messages. As far as capacity planned is concerned, Microsoft estimate that journaling may introduce up to a 20% overhead on the Exchange Server. Routing Messages to the Managed Archiving Service In both cases of Bcc Routing and Journaling, captured internal is routed to the Managed Archiving Service using the same approach. As each message is captured, it is routed to a virtual mailbox which is located on the Managed Archiving Service using standard SMTP protocols. Dedicated Archive Servers within the service are designated for this purpose and, as such, special routing rules may be required in the organization s edge servers to ensure that messages are routed to the correct server within the service. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

7 Securing Connectivity contains material of a sensitive nature. A natural concern relates to the confidentiality of internal as it is transferred from the organization s infrastructure to the Managed Archiving Service. To counter these concerns, TLS (Transport Layer Security) is used to secure the messages from the organization s edge mail server to the Managed Archiving Service s mail server. TLS is supported as standard by the majority of main stream Mail Servers and allows an encrypted pipe to be setup between two MTAs. At the most secure level, the encryption can be mandated at the Managed Archiving Service level to ensure that only TLS connections are accepted for mail upload. Infrastructure The Webroot Managed Archiving Service is hosted over two data centres that are physically separated, providing a full Disaster Recovery architecture. As already discussed, can be captured for archiving either directly from the filter mail servers as they are relayed on behalf of the organization, or by routing internal mail from the organization to dedicated mail journaling servers hosted in the data centres. Redundant Infrastructure The Webroot architecture is a fully redundant N+1 grade solution at data centre level. Two data centres operating in an Active-Active mode are backed up by a third buffer data centre. The N+1 redundancy model means that an entire primary entire data centre can be lost and service will continue without interruption. In keeping with the core architecture, the Archiving Service has been designed to work within this N+1 model, meaning that all archiving infrastructure, including storage devices are duplicated over both the primary data centres. Redundancy is a key element of the entire service, and as well as providing a fully redundant N+1 dual data centre solution, the majority of the systems within each data centre also operate at this level. This includes redundant environmental controls, dual power feeds with battery and diesel generator UPS elements and redundant internet connectivity. Redundancy is also applied at the server level which includes redundant power supplies, RAID support for storage arrays and in certain cases, redundant NICs. Data Availability Organizations require access to their stored data on a 24 x 7 basis. In order to provide this level of availability, the Managed Archive Service has been designed to utilise the architecture to store data in a redundant fashion using mirrored storage. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

8 Mirrored Storage Data mirroring is a core design feature of the entire Webroot service, which uniquely ensures that all mail in transit, or logged to a message area, is replicated in real-time. This concept has been fully extended to encompass the Message Archiving element of the service as follows. All messages received by the service s journal or filter mail servers are immediately replicated in real-time to a mirrored peer in the opposing data centre on reception of the message. Such an action ensures that the message is protected in case of primary server failure or complete data centre outage. The data centres operate in an Active-Active mode, thus allowing data to enter either site, but with full bi-directional replication. Once a decision has been taken to commit the message to archive storage, the primary centre will again split the message, causing a copy to be sent to the primary site and a copy to the secondary site. At this point, the messages are essentially processed as two separate entities by separated archiving servers, using dual separated storage technology. The result is two copies of the stored message, and two updated indexes in physically separated centres. The dual separated storage approach ensures action will not be mirrored to the secondary site if a data store or index become polluted through corruption, thus preserving the integrity of the data. This is in contrast to technologies that rely on real-time disk level replication, which run a higher risk of data corruptions affecting the redundant stores. All data is stored directly onto spinning disks, as opposed to backing off to tape. This gives near instant access to s held in the archiving system. Availability of Data Access With stored in multiple locations, the architecture enables High Availability user access. In the event that a data centre or server suffers an outage, or planned maintenance is required, availability is maintained by re-directing the access request to the data stored in the secondary location. In any archiving solution, the Index tends to be the window into the data store. Loss of an Index means downtime whilst it is re-built - if indeed it can be re-built. The Webroot Managed Archiving Service mitigates the impact risk of Index corruption by maintaining a mirrored index in addition to the mirrored storage of . Data Security Security of the data is a key requirement for any organization. The integrity of the archive is maintained by recording each message with the key meta-data items. The loss of an index means that it can be totally re-constructed from the message and its associated meta-data. The data centers are also continually monitored to ensure that the integrity of the data stores are maintained, using each store as a reference to the other. In the event of any integrity issues, network engineers are immediately alerted. All messages are encrypted upon storage with a 128-bit symmetric key. This level of encryption protects the data against data theft and prying-eyes. Each customer s data is stored in a logically separated data store (Figure 2 - Segregated Customer Data), which is identified by a unique business identity that is allocated by Webroot when the archiving system is provisioned. Every message has the Business Identity built into its naming and storage scheme. Prior to presenting a message or a search result containing a message reference to a user, the Business Identity of the message is correlated with the identity of the user to ensure that he/ she is authorized to access it. As already discussed, encrypted channels using TLS are used to protect the message on transit from the organization to the Data Centers, to ensure that sensitive internal is transferred in a secure manner. Access to data is protected through user authentication to the service over an HTTPS links to the Control Panel. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

9 ARCHIVE DIRECTOR CUSTOMER A CUSTOMER A CUSTOMER B CUSTOMER B CUSTOMER C CUSTOMER C STORAGE MIRRORED STORAGE Figure 2 - Segregated Customer Data Data Store Management It is a continual balancing act to determine how much to delete versus how much to store in order to comply with relevant legislation. Data Store Management functions are critical to ensure that archives don t become bloated, yet the letter of the law, whether it is legislative or corporate policy, is maintained Retention Periods The key to Data Store Management is the retention policy. Many archiving solutions only enable a single retention period to be set, which determines how long a message is to be kept before it is deleted. However, for increasing amounts of legislation this is not sufficient. The Webroot Archive Service is able to define both a minimum amount of time a message must be stored for and the maximum amount of time after which it must be deleted. These retention periods are set at storage time and cannot be post-modified, thus preventing the deletion of data that may be regarded as evidence within an impending enquiry. An automatic purge process runs daily to delete all that has reached its maximum purge period. Data may be stored for as long as the organization requires. Most legislation features clauses that require certain data to be stored for as long as seven years. Storage Management To ensure compliance, messages are stored in their original form. Consumption of storage space is minimized by enabling the archiving solution to use the same message for both the End-User and compliance system. All data is compressed before storing, which gives an average compression ratio of about 50%, however on a per message basis this is dependant upon content. Unlike many solutions which only index and display the HTML component, the solution indexes and displays the text parts as well. Since text and HTML can be very different in content, this is essential to preserve the integrity of the system from a compliance perspective, ensuring that data cannot be hidden. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

10 Aspects of Scalability Techniques employed by high-end Internet search engines are used to ensure that the system is highly performant and scalable. The system has been designed to scale over many petabytes of data without introduction of delays in search performance. When a search request is issued by a user through the Web interface, the search criteria are submitted to a high performance distributed search cluster in the currently active data centre. To ensure optimum performance, both in terms of data insertion and query response times, indexes are constrained in the size they may grow to. Once an index reaches its designated capacity, a new index is created. Multiple search processors are then employed to query these distributed indexes in parallel. Results from each search processor are collated by a master processor and returned to the user, which allows results to be typically returned within seconds. Based on the search results, the user may require access to one or more of the messages. Messages are stored in storage locations, designated by a unique storage allocator. Access requests are passed to a Storage Director, which is responsible for accessing the correct storage location, finding the stored message, decrypting the message and then passing it through to the user. The solution is highly efficient in that the message is exploded before being passed out, allowing the user to access key elements of the message in a quick and efficient manner, rather than requiring a complete download of the message. HTTPS RETRIEVAL SEARCH & RESULTS SEARCH PROCESSOR SEARCH PROCESSOR SEARCH PROCESSOR STORAGE INDEX INDEX INDEX Figure 3 - Scalable Search Architecture Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

11 User Types The Archiving Service supports two key user types: the Compliance user and the End-User. The Compliance User accesses the messages through the main Control Panel and has the ability to access the entire archive if suitably privileged. The End-User accesses messages from the End-User site, which already provides access to the End-User spam logs, etc. As already discussed, both the Compliance Administrator and End-User have access to the same physical messages to maximize storage utilisation. However, message access is managed in a very different way for each, relative to the user s login credentials. Compliance Interface The compliance interface provided through the standard Web Control Panel, supports a strict compliance model. Only authorised users are granted access to the archive system, which is fully audited. Every search request must be accompanied with a quoted reason, which is stored in the audit log before the search is actually applied. In fact, if for any reason the audit system cannot generate an audit activity, the activity will be denied. Optionally, a set of Data Custodians can be defined, which cause an notification to be automatically sent to the Custodians when a search request is performed. The security of the Custodian system is such that when the system is first provisioned, Webroot will configure the Custodian list for the organization. Thereafter any changes to that list will result in a notification sent to the Custodians, informing them that they may have been deleted. Coupled with the audit logging, the system is protected against unauthorized use. As in keeping with the redundancy of the rest of the service, all audit logs are replicated in real-time between data centres to mitigate risk of loss. The compliance interface also provides full view of the stored meta-data parameters associated with a message - including retention periods and information regarding the original message filtering decisions. End User Interface End-Users neither have any visibility of the meta-data options such as retention periods, filter message modifications, nor do they have access to the audit data associated with a message. The message is logically regarded as belonging to the End-User who is therefore allowed to search and view as required. Ownership of a message is established by the logged in End-User having either been a sender or a recipient of the message. Note that in the case of internal messages that have been captured through Blind Carbon Copy Routing, End-User access will not be possible, as the original sender and recipient information is no longer available. Webroot, Inc th Street, Boulder, CO U.S.A. Tel:

12 Summary Messaging is a mission critical environment. It is estimated that up to 60% of critical business information is conveyed via , and with estimates placing the growth of at as much as 80% year on year 1, the case for an archiving system is clear. Effective solutions to meet compliance legislation, minimise risk through business continuity solutions and manage storage in a cost effective manner are three key drivers pushing archiving up the organization s IT priority list. As demonstrated in this technical paper, the Webroot Managed Archiving Service is more than capable of meeting these needs. Redundancy and scalability are two of the key building blocks for the system, ensuring that when your data is stored with us, it is safe and available on a 24 x 7 basis, using a full N+1 redundant infrastructure. Comprehensive search and analysis ensure that individuals can quickly and easily find the data that they need, whilst working in the knowledge that the fail safe auditing system is protecting the organization from abuse of the data. Finally, the entire solution can be deployed with no hardware or software required on site, thus simplifying the entire proposition, mitigating risk and removing what can be a very real headache from the IT Manager. About Webroot Webroot Software, Inc. provides industry-leading security software and services to consumers, enterprises and small to medium-sized businesses worldwide. The Boulder, Colorado based company s newest software-as-a-service (SaaS) offerings, Webroot Security SaaS and Webroot Web Security SaaS provide better manageability, better value and better protection than any other or web security solutions. Webroot s award-winning endpoint products, Webroot AntiSpyware Corporate Edition and Webroot AntiSpyware Corporate Edition with AntiVirus are comprehensive, centrally managed solutions that aggressively block, detect and eradicate malware on desktops across the network. To find out more, visit or call Messaging Technology Report 2003 [Radicati] 2008 All rights reserved. Webroot Software, Inc. Webroot, the Webroot icon, and the Webroot tagline are trademarks or registered trademarks of Webroot Software, Inc. in the United States and other countries. All other trademarks are properties of their respective owners. NO WARRANTY. Analysis based on research conducted by Webroot Software, Inc. The information is provided AS-IS and Webroot makes no warranty as to its accuracy or use. Any use of the technical documentation or the information contained herein is at your own risk. Documentation may include technical or other inaccuracies or typographical errors. Webroot reserves the right to make changes without prior notice. Certain data is available upon request. Webroot, Inc th Street, Boulder, CO U.S.A. Tel: