June Advanced Data Reproduction and Storage System Data Center in a Box

Transcription

1 June 2012 Advanced Data Reproduction and Storage System Data Center in a Box

2 Contents Introduction... 2 Steps for Successful Document Management... 2 Phase 0: Survey... 4 Phase 1: Architecture... 4 Phase 2: Application... 6 Phase 3: Process Implementation Control Group... 6 Optional Phase 4: Complete Initial Hardcopy Digitization Execution... 7 Optional Phase 5: Long Term Digitization Capability... 7 Architecture Overview... 7 Cost ROI Factors... 8 The Data Solution... 9

3 Introduction Without a better solution, every agency s abilities to archive and analyze huge amounts of data will spin out of control, and archiving policies will be increasingly unaffordable and un-executable. The ability of government agencies to provide continuous, secure, effective and efficient document management in support of operations, analysis, planning and archiving is critical. Consolidation of facilities and operations has severely challenged organizations to meet data management requirements. Effective and efficient means to consolidate data, control it and make it available for use by operators are bottom line requirements. Additionally, data collection and access speeds, accountability, facility space, power and maintenance, all of which affect the total cost of ownership, are key management and governance factors for senior leader consideration. In summary, to ensure success, government agencies need advanced technical capabilities and processes that enable full and uninterrupted document management, security and access with due consideration of total cost of ownership factors. Another strong need is the government agencies requirement to safeguard historical data and imagery, as mandated by law. The current storage strategy of nearline and offline tapes requires a constant refresh of tapes and migration of data combined with constant hard drive replacements for online storage due to the relatively short functional lifetime of hard drives and tape systems. Again, this significantly affects the total cost of ownership and is a key management and governance factor for consideration by senior leadership. Hie Electronics TeraStack Solution with Active Archive will automatically tier historical data to the most cost effective tier: storing infrequently accessed data or sleeping data onto robust archive media and storing hot, frequently accessed data for immediate access on fast access memory such as SSD or HDD. Classified data presents another storage retention requirement, as well. Chain-of-custody is critical, assuring the authenticity of all data. This data must also be cataloged, stored, and managed for the duration of its mandated retention period on a storage medium that can guarantee the integrity and authenticity of the data. The TeraStack Solution has built-in file level security and chain-of-custody software to track who has accessed data and when, thus eliminating unauthorized access and corruption. Steps for Successful Document Management Digitization will reduce the hardcopy volume in government agencies, along with decreasing the time required to search and acquire records. Hardcopy libraries are inherently inefficient, are susceptible to unintentional damage, and typically require an individual s direct experience with the specific archive to extract critical information. When a digitization capability is implemented as a Specialized Business Practice utilizing the TeraStack Solution, advantages include: Preserves critical data Provides workflow for digitization of all future data Improves data usability across enterprise Alleviates burden of hardcopy storage space requirements Provides permanent, non-volatile storage media for digitized content on archive grade Blu-ray

4 The digitization team must provide these benefits: Feature Digitizes legacy hardcopy data including text documents, maps, images Digitization contents bolster content security and access tracking TeraStack physical storage footprint for greater volumes of data Provides framework for content digitization of newly generated content Risk mitigation implementation approach Benefit Data is permanently archived in a non-volatile format Always available to authorized users: never misplaced, misfiled, lost or damaged Provides modifiable text and image markup Secure solution easily integrates into agency environment Readily expandable to meet future needs Proven low total cost of ownership Eliminates future build-up of large hardcopy libraries Ensures the right data is always available to users when they need it Data period provides agency with clear checkpoints and decision gates Reduces the impact of cost, schedule and technical issues Let s consider a project approach for the digitization of the paper archives of a Federal Agency which is relocating its headquarters or another office. This agency wants to eliminate its paper documents and using the building or base relocation (BRAC) move as a change agent for it s document control and access. The goal is when the new office environment is commissioned that the employees start to use a Digital Asset Management Electronic Document Management Information environment. The digitization approach follows five phases which are discussed in greater detail in this section of the white paper. Phase 0: Survey Perform a survey and analysis among the agency user population at the current agency facilities to assess and validate the scope and magnitude of the amount of hardcopy material required to be digitized. Phase 1: Architecture Pilot program to validate the solution architecture, including scanners, OCR capabilities, document management application, and the TeraStack Solution storage system architecture. Phase 2: Application Development Conduct advanced development of the enhanced OCR engine, image meta-tag and image overlay capability applications to increase search, archival, retrieval and analytical capabilities. Phase 3: Process Implementation Control Group Conduct scanning of hardcopy material from 2% of the agency users. This phase provides a refined basis of estimate to complete the full digitization effort. Optional Phase 4: Complete Initial Hardcopy Digitization Execution Full scope completion of the initial hardcopy digitization requirement. Provides the processes and capability to support

5 the agency s continued digitization efforts that are offered in the Optional Phase 5 to preserve hardcopy materials. Optional Phase 5: Long Term Digitization Capability Continuation of operations from Optional Phase 4 to preserve new hardcopy materials. Phase 0: Survey The survey and analysis phase will determine the volume and types of existing hardcopy materials at all identified locations and validate the assumptions regarding quantity and type of data. Digitization Team will survey agency users to quantify the volume and type of data to be digitized. Digitization Team will review the hardcopy material repositories and conduct interviews with holders of the material. Digitization Team will document the results and perform a gap analysis of the initial assumptions. Digitization Team will prepare recommendations for exceptions to the assumptions. Digitization Team will brief the agency on the results, recommendations, and any modifications to schedule, cost, and technical approaches. This phase will provide the end-users with the guidance to prepare their material for the digital scanning that will take place in later phases. At the end of this survey period, the digitization team will have real data on the amount of hardcopy data identified for digitization to report to the agency. From the survey, a schedule of users will be determined for participation in the Phase 3 Control Group; work with this group will be the baseline for the project. Materials not requiring digitization can be destroyed. Phase 1: Architecture In this phase, the architecture described in this whitepaper that provides the digitization capability will be integrated at one location. This architecture includes the scanners, servers, TeraStack Solution storage solution, and the Document Management Application (DMA) to support the scanning, storage, archival, and retrieval system. The Agency chooses the DMA tool.

6

7 Phase 2: Application Phase 2 focuses on providing enhanced analytical capabilities by providing text searchable and metatagged documents and images. The following are typical types of documents that can be digitized. Text Documents - This phase focuses on enhancing the OCR engine to extract editable text from scanned hardcopy documents. This provides high fidelity OCR modifiable documents that can then be processed by standard desktop application including Microsoft Word, notepad, and textual entity extraction. The OCR capability could also be used in conjunction with enterprise textual entity extraction capability to automatically extract textual entities from the OCR document and apply metadata to the source document. This additional metadata will greatly improve the speed and efficiency of document searches, increasing the analytical capabilities of the agency users. Images - During this phase, the Digitization Team will develop an imagery metadata extraction capability that identifies one or more text blocks included on imagery documents, and translates that information into actual machine editable data. This data can be applied as metadata to the digitized imagery document and the enhanced OCR engine will have the capability to identify fields in forms for processing and making it readily searchable. Maps - The Digitization Team will implement a map overlay creation capability, allowing users to create overlays from their digitized imagery and mapping documents. These overlays can then be imported into and used with Google Maps and others to add additional detail and information. Users will be able to create and edit map overlays based on digitized mapping documents; this includes the ability to mark specific points in their overlay to rectify the image, and add additional information to existing map overlays. Phase 3: Process Implementation Control Group During Phase 3, the Digitization Team will take a limited set of agency users, refine the processes to be used in the Phase 4 and provide revised cost and schedule estimates. The Control Group digitizes approximately 2% of the estimated hardcopy information. They will work individually with each of the users in preparing the hardcopy material, and then scan the material, complete Meta-tagging and place the digitized data onto the system. They will perform digitization Quality Control validation and testing to ensure the material responds correctly to user queries and associated search requests. At the conclusion of Phase 3, the digitization team will provide a refined benchmark schedule and detailed cost estimate to the agency to complete the remaining 98% of the digitization proposed as Optional Phase 4 and will form the basis for the ongoing services included within Optional Phase 5.

8 Optional Phase 4: Complete Initial Hardcopy Digitization Execution In Phase 4, the remaining 98% hardcopy material is digitized, meta-tagged and marked-up, and made available on the system. During this phase the Digitization Team will provide productivity metrics detailing the progress and Key Performance Indicators (KPIs). The Period of Performance for this full production phase will continue until the accumulated hardcopy data is digitized. Optional Phase 5: Long Term Digitization Capability Once the current inventory of hardcopy data is digitized, the agency must maintain an ongoing digitization capability to preclude the buildup of large hardcopy archives in the future. Phase 5 advances the digitization capability into an operational process providing the infrastructure to maintain a digitization capability at the agency. This ensures that the agency will optimize its capabilities well into the future, while minimizing the likelihood of new paper-based archives. Architecture Overview The following is a sample architecture for the digitization initiative that would be validated during Phase 1 of the project. Hardware and software utilized will be Xerox and Hie Electronics products. Scanning Capability. The Digitization Team will use high-capacity and fidelity Xerox scanners. The Xerox DocuMate 4799 Scanner will scan up to 100 pages per minute and boasts advanced sensor systems: ultrasonic double fee detection, stapled document detection, and intelligent feed control. This will be networked to the meta-tagging and markup databases, document management applications and the TeraStack Solution storage devices thru the TeraStack Solution s indigenous enterprise server. OCR and Mark-Up. The Digitization Team will initially use Xerox tools, and will migrate to an enhanced OCR and Mark-up capability developed under this effort. DocuShare Products has a complete package for OCR and Mark-up, easily added on to the DocuShare Document Management Software package and hosted on the TeraStack Solution indigenous enterprise server. Once the images are processed, they are fully indexed and searchable. Document Management. Digitized data will be available through Enterprise Management Software (DMA) that stores meta data relative to each data set. A flexible and easy to use content management platform can scale to tens of millions of documents and tens of thousands of users. This software would be installed onto to the agencies server hardware. No unique software will be required on the agency client desktops, as a web-based client would be utilized to access digitized content. Storage Devices. The Digitization Team will use the Hie Electronics TeraStack Solution storage system. All digitized hardcopy data is stored on Blu-ray optical media within one of eight 6.25 TB TeraStack managed optical volumes, which can be removed and stored offline or inserted into a separate system, extending an individual systems overall archive capacity into the petabyte (PB) range. Multiple systems may be clustered to provide increased data accessibility as needs dictate. The TeraStack Solution provides quick random access and retrieval of all documents archived through its enterprise-class, industry-standard searchable content database. The media integrity is guaranteed by the manufacturers for 50 to 100 years, compared to 3-5 years for hard drives and 5-7 years for magnetic tape. With a

9 footprint 60% smaller than other storage devices, the TeraStack Solution integrates well into any space constrained environment well into the future. Networking and Operating Systems. The connectivity and networking functions are provided by industry standard, non-proprietary components, devices and software. Cost ROI Factors A sample ROI calculation from 2008 follows. A government agency presented that an estimated 55,000 cubic feet of hardcopy material existed that was required to be preserved through digitization. For the purposes of ROI calculation, storage space cost analysis is based on the volume of data and the associated cost per square foot. In addition, there is an increase in the growth of hardcopy volume over the timeframe from 2009 through 2015 as well as floor space cost and infrastructure increases to provide a cost savings basis. The results are shown in the table below: Description Base hardcopy estimate per the agency (ft 3 ) 55,000 75, ,000 Typical file cabinet capacity at 6 x3 x2 (ft 3 ) The average space require for file cabinet (ft 2 ) Additional floor space required for file cabinet (ft 2 ) 15,280 20,830 27,770 Estimated cost per square foot (each year) $30 $35 $40 Annual cost avoidance $458,400 $729,050 $1,110,800 Total cost avoidance for 3 years $1,375,200 $2,187,150 $3,332,400 Accumulated cost avoidance from $1,375,200 $3,562,350 $6,894,750 The cost savings are provided for representative purposes, and will change when considering actual costs and volume of data produced. However, it is worth noting that the Air Force Satellite Control Network digitized its entire legacy and current hardcopy technical data library* which totaled approximately 640,000 engineering drawings and over 3,000 complete documents. The digitized data is available through a secure web-enabled portal called the Central Data Library (CDL). The ROI for the entire effort was three years, and the new business practices employed to maintain electronic access reduced the time to field changed drawings and documents from up to one year to just minutes. The CDL has been operational for more than 10 years. * Included microfiche, aperture cards, E-sized engineering drawings, Technical Orders and Specifications.

10 The Data Solution Hie Electronics is the leader in Active Archive data storage technology and the manufacturer of the TeraStack Solution, a data backup and archiving system. This system features application hosting and up to 142 terabytes of data storage using hybrid architecture of hard drive and optical media. A leader in Sustainable IT technology, the TeraStack Solution product line delivers a 90 percent energy cost savings when compared with that of current technology. This system provides extraordinary capabilities to rapidly archive the vast amounts of sensitive agency documents; shrink them down to a manageable and easily transportable digital format in managed optical volumes; and provide easy, dynamic and searchable access to that data whenever it is needed. Either write-once, rewritable, or both types of media may be used in the TeraStack Solution to facilitate long term archival or short term backup business use. All data is stored on removable, transportable TeraStack cartridges containing 6.25TB (with 50GB Blu-ray discs) or 12.5TB (with 100GB Blu-ray discs) of data each. Each stack is contained and secured in a hard metal casing to prevent tampering and damage to the 125 Bluray optical discs contained inside, each rated with years of data integrity. Compared to 5-7 years for tape and 3-5 years for hard drives, the integrity of your data will be assured long past its archival requirement without continuous data migration from one tape media to the next. Once stored onto the TeraStack Solution, the TeraStack cartridges are easily removed and stored and secured per appropriate security classification guidelines. The TeraStack Solution allows for the quick location and retrieval of all documents through an industry standard searchable content database. Stored documents and data can then be quickly restored from the TeraStack Solution to any system architecture as needed, in the same format that they were initially stored on TeraStack Solution. The TeraStack Solution s Java enterprise middleware services provide a completely open system architecture that easily integrates with virtually any software application. The TeraStack Solution includes a high-end application server capable of hosting the most demanding applications. Backed by 100TB of nearline, non-volatile, long life Blu-ray media and a 42TB RAID buffer, the application server helps simplify your data management needs into one simple solution consuming a maximum of only 425 watts of power to operate and no special power or HVAC requirements. The TeraStack Solution enables the automated storage and management of a vast library of nonvolatile dense optical storage media. The Hie Electronics TeraStack Solution s technical features translate to leap-ahead operational and support capabilities for document imaging and reproduction, and data management: Manages huge amounts of data with storage, search and archiving for months or years; both recordable and rewritable formats are available; archiving periods are based on customer requirements. - Support fast collection and use of information, with little time and manpower resources.

11 - Employ and enable expansion with available best of breed cutting-edge applications, with the vast amounts of information those programs will be able to access when supported by this system. - Data warehousing and mining techniques. - Employed as a stand-alone system or integrated into existing network architecture or clustered environment as needed. - Supports positive data chain-of-custody using hardware write-once media and native or third party encryption software to assure authenticity of data. Low total cost of ownership (TCO): - Low power consumption (425 Watts, equivalent of wall outlet plug) - Low heat production (no special raised floor or cooling requirements) - Minimal facility space (1/3 rack 15U) - Low maintenance costs - Increased data automation - Significantly reduced data migration costs - Open systems architecture based upon Java 2 Enterprise tools, Oracle, MySQL Bottom Line: The TeraStack Solution along with scanning hardware and Agency Choosen Digital Asset Manatment (DMA) provides users and data managers with the significantly enhanced capability to minimize the footprint and operational overhead associated with moving all of their hardcopy documents: images, maps, video, film and other sources. Most importantly, the mountain of documents and media will be reduced to a manageable volume of optical discs. Scanning Hardware for Paper Documents: Scan large volumes of business documents (100 pages per minute) Digital Asset Manatment (DMA) Software: Manage large volumes of business documents Support thousands of users Search through large volume, long-term archives in an efficient manner Automate high volume, document centric business processes TeraStack Solution: Store large volumes of business documents Host best-of-breed applications with large, long-term data requirements Provide chain-of-custody control for sensitive and classified material (Varonis) Enterprise data backup/archive with the added assurance of years of Blu-ray integrity TeraStack Solution Additional Features: Fully scalable solution, including centralized management and SAN support Benchmark management of sleeping data Active Archive Solution Integrated Application Hosting Server (Scalable) which can fully operate with Agency Data

12 Security Policies 42TB of high-speed internal RAID5 buffer capacity 100TB of nearline storage, extendable to petabytes of total storage capacity with hotswappable media volumes and in a clustered configuration Eight interchangeable, hot-swappable TeraStack cartridges per unit storing 6.25TB (50GB discs) or 12.5TB (100GB discs) of data each Fully configurable user logins, groups, and permissions preventing unauthorized access to controlled data on the TeraStack Solution Data throughput rated at 190GB/hour Burst data rate of up to 600GB/hour Technology roadmap anticipates Blu-ray capacity and speed increases throughout the next decade. Scalable to Many Systems clustered together in a Cloud Service Environment