Reference Architectures for Repositories and Preservation Archiving

Transcription

1 Reference Architectures for Repositories and Preservation Archiving Keith Rajecki Education Solutions Architect Sun Microsystems, Inc. 1

2 Agenda Challenges Solution Architectures > Open Storage/Open Archive > Cloud Computing Customer Success Stories Summary Next Steps 2

3 Challenges 3

4 Value of Reference Architectures Minimizes cost, complexity and deployment time >Lowers administrative costs via automated data management and migration across storage tiers >Cost-effectively matches the value of data with the appropriately priced media >Economical and power-friendly cost of operation Flexibility to build performance, economy or mixed archive repository >Infinitely scalable archive management 4

5 Sun Reference Architectures Develop Collaborative, Replicable Reference Architectures Fedora Fedora/Drupal (Islandora) DSpace EPrints Duraspace (Cloud) Ex Libris Rosetta VTLS VITAL SAM/QFS Internet Archive in a Sun Modular Datacenter Tessella Safety Deposit Box* 5

6 Sun Open Storage Product Positioning StorageTek 7410 SAM/QFS StorageTek7210 Identity Management and SOA Digital repositories for data and metadata storage Fedora, EPrints, and D-Space communities Ex Libris Rosetta and VTLS VITAL applications Large scale preservation projects needing policies Digital asset management eresearch databases Federated repositories 6

7 Virtualized Repository Appliance Digital Objects Single Virtualized Server Virtual Machine 1: Repository Entity Preservation Index Creation Metadata Management Security Search Engine Archive Repository App. Oracle, MySQL Solaris + ZFS Virtualized Server Physical Storage Virtual Machine 2: Archive DB Policies Metadata Virtual Machine 3: Open Storage Mgmt Storage Preservation Physical Storage 7

8 Open Repository Tiered Components User Digital Objects Application Server Entity Preservation Metadata Management Relationship connections Security Search Engine Policy Driven Open Storage Storage Preservation Abstraction OpenSolaris, ZFS, SAM Physical Storage Components Media Migration DB Server Digital Asset Policies Metadata Tape Libraries Full Sun portfolio of supported tape libraries 8

9 QFS Standalone File System TCP/IP Metadata Data FC or iscsi LUNs 9

10 SAM-QFS Archiving File System TCP/IP Network-Attached Tape Library Metadata Data FC or iscsi LUNs 10

11 SAM Archiving Configuration QFS Solaris SAM-QFS Solaris Copy 5 Copy 6 Copy 3 Copy 4 NFS File System or Appliance TCP/IP Copy 2 SAM-QFS Solaris Copy 1 11

12 Sun s Infinite Archive System Approach Factory integrated, solution tested, simple, scalable, and economical Library DB Surveillance Web Unstructured IP Communications Protocol Infinite Archive System (IAS) Models: Value and Midrange X4200 T5220 Intelligent, Policy-Based Automated Archive IAS GUI, Storage Archive Manager SAM-QFS, Sun Cluster 3.2, Solaris Infinite Archive System (IAS) Tier 1 + Tier 2 Disk IAS Options 2500 series SAS 2500 series SATA Scalable, Eco-Efficient Tape Tier Options SL8500 SL3000 LibrariesEncryption SL500 Access & Capacity Drives 12

13 Single Node IAS Components Software X4540 Server Disk Cache Disk Archive Internal Tape (SL500/LTO4) + Standard Optional SAM-FS Migrators QFS ID Manager Mgmt GUI Solaris 10 Sun Infinite Archive System Sun Libraries and Tape can be attached externally for unlimited scalability 13

14 Dual Node IAS Components LAN Software FC Disk Cache T5220Servers SATA Archive Internal SAN + Optional Internal Tape (SL500/LTO4) Standard Optional SAM-FS Migrators QFS ID Manager Mgmt GUI Solaris 10 Sun Infinite Archive System Sun Libraries and Tape can be attached externally for unlimited scalability 14

15 15

16 Cloud Computing 16

17 Emerging Cloud Deployment Patterns Test and Development Functional Offload (Batch Processes TimesMachine) Functional Offload (Storage SmugMug) Augmentation (Temporary Load Animoto) Web Service 17

18 Storage Service What It is > On-demand, API-based access to storage on the network Features > Ability to store and retrieve data as objects or files > REST API with open, AWS S3-like semantics for > > > > object storage WebDAV for file storage Fast and inexpensive cloning of objects and files High availability Detailed metering of storage used, I/O requests, bandwidth, etc. Customer Benefit > Scalable, highly available storage without big hardware investments 18

19 TACC: World s Top Supercomputer Sun Fire X Systems 1.7 petabytes 64.8 GB/sec total bandwidth Sun Fire X systems 800 cores SunBlade ,936 blades 15,744K CPUs 62,976 cores 125 TB/RAM Sun Data Center Switch 3456 Dual redundant 110 TB/sec bisectional bandwidth The world s largest largest computing system in the world for open science research Sun Constellation Linux Cluster and Sun StorageTek Mass Storage Facility Tflops peak performance 19

20 SAM-QFS Recipe for Success Library of Congress Customer Challenges Convert millions of recordings, video & film clips, and photos to digital form Improve its ability to acquire and provide public access to audiovisual content Archive for The life of the republic Solution Business Results A robust storage area network based on Significantly increase the: Sun disc and tape storage technology >Rate at which it can acquire new content Sun SAM-QFS storage management >Amount of content it can store software >Time horizon for preserving content 20

21 SAM/QFS Customer Success Story Digital Asset Management w/open Text Artesia > Retrieve, reuse, repurpose content realtime > Implemented in 4 months > Improved production productivity 10%-40% 21

22 SAM-QFS Customer Snapshot: Healthcare/Life Sciences Cleveland Clinic Customer Challenges Archiving 10 TB of image data per week Accessing 20 TB of image data per week on a read-basis Access diagnostic images from any site worldwide as soon as the exam is completed Complete business continuity... in the event of a disaster Solution SAM QFS is a key technology that lets us do some very critical things - Robert Cecil, PhD, Cleveland Clinic s network director Business Results Sun's approach with... SAM-FS and QFS software is the core of our digital imaging storage strategy. "Data loss at the institution is so small, that it can t be measured"... a tremendous advantage in terms of data recovery and data availability 22

23 SAM-QFS Customer Snapshot: Media, Entertainment Major League Baseball Customer Challenges Create a high-performance, scalable and available data center infrastructure Deliver real-time and archived audio and video content Develop an on-demand digital asset management system Solution Two new data centers and a digital asset management system powered by Sun technology Sun SAM and QFS software manage shared file and archiving capabilities Business Results Over one billion minutes of streaming media Over 2,000 full-length games Over one billion visitors 10 million page views per day No application downtime in two years 23

24 SAM-QFS Customer Snapshot: Media, Entertainment & Internet Services HBO Customer Challenges Transition 5,000 hours of programming from videotape-to digital storage Reduce use of costly videotape and tape machine support costs Enable cost-effective, secure and highly reliable delivery of digital programming to millions of subscribers Solution Business Results Standardize on Sun and Grass Valley Near-seamless content delivery to technology for a server-based storage and broadcast and on-demand subscribers play-out system with percent system availability Digital repository for SD and HD programming eliminated 80% of existing videotape equipment Sun QFS file-sharing software to provide the scalability and powerful Significant savings in both labor and performance required to meet HBO's maintenance costs demanding throughput goals 24

25 SAM-QFS Customer Snapshot: Government Federal Ministry of Finance, Germany Customer Challenges Develop and implement automated tariff and local customs handling systems (ATLAS) for customs processing Provide a secure, scalable, and highly available infrastructure with mirroring capabilities Solution Business Results A new three-tier mirroring system Customs clearance is now completed more architecture with Web browsers leveling the accurately and much faster than before first tier Exceeded server expectations and delivered Data stored on Sun systems 10 km apart solution six months ahead of schedule with Sun StorageTek 6540 arrays for We have a zero failure rate... disaster recovery Sun QFS file sharing software 25

26 Customer Success Story Digital Content Archiving Industry leader in video post-production Locations in US and EAME Digital Media Environment > > Implemented tiered storage solution from Sun > Managing massive amounts of shared digital content View, edit, store uncompressed data between global facilities SAM-QFS, 6540, X4500, SL8500, T10000 Streamlined digital file-based workflows > > Archiving content cost effectively Generating new revenue streams 26

27 Internet Archive Highest Density Integrated Storage Architecture Key Requirements Build a server infrastructure to support massive amounts of data 2 PB of storage, growing by 1 PB per year Provide an efficient, reliable, and scalable datacenter Keep space, energy, management and maintenance costs low Web snapshot 100 TB of data - approximately 4 billion Web pages. Sun Solution Client Results Sun Modular Datacenter S20 Sun Fire X4500 Server Solaris 10 with ZFS Sun Remote Operations Management Gained a reliable and flexible datacenter that supports multiple PB of storage Increased storage capacity of its servers Reduced space and energy needs for lower costs Superior data integrity to guard against data loss Rapid time to deployment Sun MD unit delivered in less than 45 days Support up to 500 user queries per second. 27

28 Next Steps Sun Edu Essentials On-going Discounts Try & Buy 60 days up to 40% off Sun Products Open Archive Architecture Assessment 28

29 For More Information Storage Archive Manager Join the Sun Preservation and Archiving Community Join the OpenSolaris Storage community Open Storage Open Storage Servers 29

30 Thank You Keith Rajecki Education Solutions Architect Global Education & Research Sun Microsystems, Inc. 30