Avid ISIS

Transcription

1 Avid ISIS

2 Table of Contents Overview... 3 Avid ISIS Technology Overview... 6 ISIS Storage Blade... 6 ISIS Switch Blade... 7 ISIS System Director... 7 ISIS Client Software... 8 ISIS Redundant Infrastructure... 8 Avid ISIS 7000 Technology Comparison... 9 Integrated File System and Storage Infrastructure... 9 Shared File System on General Purpose Storage Hardware... 9 Storage and Scalability Summary Conclusion

3 Overview Avid ISIS 7000 is highly-scalable storage system based on a parallel file system that meets the latency constraints of media applications found in the most demanding postproduction and broadcast environments. The Avid ISIS 7000 solution offers support for up to 330 real-time clients with simultaneous access to up to 192 TB of usable mirrored storage or 288 TB in RAID6 configurations. Avid ISIS 7000 provides uninterrupted facility-wide media access, powerful, easy-to-use management tools, and an open architecture supporting both Avid and non-avid editing clients. Avid ISIS 7000 is uniquely capable of delivering frame accurate media with strict latency requirements even at high system load. Today s collaborative environment, tight budgets and HD storage and capacity requirements mean that a media network must deliver more than shared storage alone. An Avid ISIS 7000 network comprises scalable storage to accommodate growth needs, coupled with Quality of Service (QoS) and high performance 1 Gb/s and 10Gb/s Ethernet. Deploying Avid ISIS 7000 provides the ideal storage foundation for even the largest post production and broadcast environments. Avid ISIS 7000 delivers economic scalability and blazing performance based on logically segmenting system functions. Media storage is distributed across Avid ISIS 7000 Storage Blades (ISBs) and Avid ISIS 7000 Engines (chassis) while metadata is stored on redundant System Directors. Integrated 1 & 10Gb/s Ethernet provide superior connectivity while redundant power and fan systems complete the infrastructure. Avid ISIS 7000 client software provides intelligent, distributed media storage that is highly redundant and scales easily as capacity is added. Each Avid ISIS 7000 Storage Blade is a micro-server in its own right, including dual disks, processor, memory and dual network interfaces. Mirrored protection provides the highest levels of throughput and system performance. RAID6 dual parity protection increases storage efficiency to 75% usable capacity and maintains protection from two simultaneous disk failures. ISBs are allocated to storage groups configured for mirrored or RAID6 protection. Maximizing performance (mirroring) or storage efficiency (RAID6), each Avid ISIS 7000 can have multiple storage groups as required by the desired workflow(s). The Storage Blades communicate with the clients to provide explicit guarantees for frame delivery latency; if an ISB cannot deliver within the required timeframe, the ISIS client will request media from the mirrored copy. To provide fast recovery while still delivering real-time media, data redistribution in the event of a drive failure is performed by all ISBs working in parallel. A pair of Avid ISIS 7000 System Directors manages the metadata of the file system storing information such as directory structure, access rights, and file creation times. In contrast to most other file systems, the System Directors do not store the location of the blocks making up a file, which are determined algorithmically by the ISIS file system client software. Thus the meta-data server s scalability is independent of the amount of system storage. 3

4 Avid ISIS 7000 offers the following unique combination features and benefits: 1. Real-time Performance: ISIS implements several mechanisms enabling it to deliver real-time performance. a. It allows a workstation client to explicitly state the priority and real-time performance needs through deadlines for each read or write request. The scheduling algorithms take these deadlines into account when ordering disk requests. b. It provides a mechanism used by applications to pre-fetch data into storage system cache from disks without using host memory. As the size of the Avid ISIS 7000 storage system grows, the amount of cache increases. c. It implements highly tuned data transfer protocols built on top of industry standard Ethernet which minimize memory copies. These protocols work over 1 Gbs & 10 Gbs Ethernet. Direct 10 Gb/s connectivity supports processing of uncompressed HD resolutions, or delivers multiple streams of compressed high definition video. d. The data distribution algorithm implemented in Avid ISIS 7000 distributes portions of data with a pseudo-randomness among the available storage blades. It also reduces the probability of hot spots among blades. e. Only drives that have passed Avid s rigorous drive qualification process are used in ISIS storage blades. This results in the use of drives that guarantee bounded I/O command times; it is a requirement that prevents the disruptions of real-time operations due to specific drive characteristics such as, retries, ECC, block mapping, internal diagnostics and calibrations. f. It additionally delivers real-time performance even during redistribution of data due to drive failure. These mechanisms allow ISIS to deliver real-time performance while maximizing the performance capabilities of Avid ISIS 7000 hardware. By contrast, systems without such mechanisms can only meet real-time requirements at low utilization rates. This is why ISIS is able to support a large number of concurrent clients. 2. Scalability: The size of an ISIS system ranges from 32 disks (16 storage blades in one Avid ISIS 7000 Engine, 2 disks per blade) to 384 disks (12 Avid ISIS 7000 Engines). Customers can start with the smallest system and grow it by adding more storage without disrupting their workflows. To avoid burdening the System Director, Avid ISIS 7000 computes file chunk location information, both at the Storage Blade and client. This leverages capabilities of the intelligent storage blades and allows Avid ISIS 7000 to scale up in capacity and performance as the system expands. The Avid ISIS 7000 redistribution algorithms allow capacity to be added on-the-fly without interrupting active use. Avid ISIS 7000 also supports the dynamic resizing of workspaces with a simple slider-bar user interface. A benchmark of all Avid ISIS and Unity media network implementations, this feature allows administrators to quickly and easily reallocate available capacity between projects as workloads necessitate. 4

5 3. Scalable Recovery Throughput: The Avid ISIS 7000 patent pending data distribution and redistribution mechanisms allow the recovery rate from disk failures to scale up as the size of a system increases. Moreover, the impact of a failed disk on user workload also decreases with increase in the size of the system. Leveraging the micro-server processing capacity of each blade, each Avid ISIS 7000 storage blade participates in data recovery. The elapsed recovery time is further improved by exploiting the system awareness of which blocks belong to a file. Not only are those alone recovered, but recovery of a 1 TB blade can be completed in 30 minutes in a 12 chassis system. 4. High Reliability and Availability: a. Making the system resilient to many component failures, active hardware components are redundant. b. When mirrored protection is used, Avid ISIS 7000 can protect against loss of data due to a storage blade failure. ISIS 2.1 also implements RAID6, enabling it to tolerate the failure of two storage blades. These mirroring and RAID6 implementations are unique. They preserve the real-time performance and scalable recovery benefits described above, where storage blades work in parallel to recover data. Moreover, Avid ISIS 7000 unique redistribution technology begins protecting new content immediately upon drive failure. c. Designed with the changing storage needs of customers in mind, an ISIS system can be expanded while it is online. User workspaces that partition the storage can also be expanded or reduced while the system is online. d. Field replaceable components can be interchanged without bringing the entire system down. 5. Efficient, Flexible Storage Utilization: Administrators allocate Avid ISIS 7000 Storage Blades (ISBs) to storage groups configured for either mirrored or RAID6 protection. Storage within a storage group is assigned to workspaces mounted by client hosts. Delivering highly efficient use of raw storage capacity, Avid ISIS 7000 deploys 6+2 RAID6. This provides dual parity protection plus high performance read/write operations. Alternatively, storage groups can be configured for mirroring to deliver the highest level of performance. 6. Integrated Networking: Each Avid ISIS 7000 Engine chassis includes two integrated hot swappable Ethernet switching modules that provide 1Gb/s & 10Gb/s Ethernet networking and 12 Gb/s dedicated interconnection between Avid ISIS 7000 Engines. 7. Open Using Industry Standards: Avid ISIS 7000 supports integration with industry standard protocols to simplify use in any IT-centric environment. a. ISIS is compatible with Microsoft s Active Directory services through the use of the LDAP protocol, which streamlines administering ISIS users, user group accounts, and associated credentials. b. For client workstations that do not require real-time QoS, ISIS workspaces can be shared through a Common Internet File System (CIFS) layer, using one or more ISIS File Gateways. This option eliminates the need for loading ISIS client software on those workstations wishing only to access ISIS storage for basic file operations. 5

6 c. Avid ISIS 7000 provides support for selected non-avid editor applications, providing a single shared storage environment for a diverse range of user needs. 8. Ease of Management: ISIS provides a web-based GUI for the centralized and integrated management of the file system as well as hardware infrastructure. This GUI can be used to configure the system, add and remove capacity, diagnose system problems, and initiate remedial action when problems are diagnosed. Avid ISIS 7000 Technology Overview ISIS Storage Blade The Avid ISIS 7000 Storage Blade (ISB) is the foundation upon which ISIS massive scalability and unrivaled performance are built. Each ISB is comprised of 2 disk drives, a CPU, memory and dual Gigabit Ethernet network interfaces. The ISIS micro-server software runs on the ISIS Storage Blade. Optimized to store larger media files than are typical of ingest, edit, and playout workflows, Avid ISIS 7000 files are stored on disk as fixed size chunks (256 KB or 512 KB) and distributed across the ISBs of a storage group using the data distribution algorithm. Balancing throughput and latency requirements, these chunk sizes are ideally suited to the media environment. The distributed approach for mirroring and RAID6 avoids rebuilding 100% capacity of a failed drive. When a write of a chunk is completed on the primary ISB, the mirrored copy is written to the alternate location, such as another ISB in the same Avid ISIS 7000 Engine or in another Avid ISIS 7000 Engine. Both copies can be accessed by clients. This allows the ISIS client to select the copy that meets their latency needs. All Storage Blades participate in data redistribution initiated whenever blades are added to, or removed from, a storage group. This parallel approach to data redistribution leverages the CPU, memory and internal network capacity to return to a fully protected state as quickly as possible. Due to the additional number of disk drive spindles and micro servers operating in parallel, time to restoration decreases as the size of the storage group is increased. When the configuration grows, each drive is responsible for a smaller proportion of the protected data. Managing quality of service of all storage operations, Avid ISIS 7000 provides unequalled delivery of high load with low latency, while simultaneously performing data redistribution. 6

7 ISIS Switch Blade Providing redundant connectivity to Storage Blades, additional chassis, clients and external network switches, each Avid ISIS 7000 Engine has two integrated switch modules. The iss2000 is the Standard Switch Blade with 16 Gigabit Ethernet interfaces to internally interconnect all ISBs in the chassis. A 12Gb/s internal chassis interconnect interface is used to provide cost effective expansion between two Avid ISIS 7000 Engines. An additional 8 Gigabit Ethernet interfaces provide interfaces for directly connected clients or switches. A 10Gb/s Ethernet interface provides high bandwidth connectivity for high resolution clients or to connect a higher number of clients using qualified external switches. To expand beyond 2 engines, the ISIS Expansion Switch Blade, ixs2000, is used to deliver 11 x 12Gb/s interconnect interfaces, building an ISIS storage array of up to 12 Engines containing 192 storage blades. All interconnection between ISIS Engines are built with 12Gb/s interfaces. ISIS System Directors (SD) are connected to ISBs through ISIS Switch Blades. Connectivity to 10GigE high bandwidth editors and Gigabit Ethernet connectivity is provided via dedicated external Ethernet switches. certain Ethernet switches from Cisco Systems Inc. and Brocade Communications Systems (Foundry).have been qualified and approved by Avid. Network design rules recommend deployment of two separate Virtual LANs (VLANs) for each set of ISIS Switch Blades (all of the left switch blades in each Engine are on one VLAN and all of the right switch blades on the another VLAN), extending out to the external switches. All ISBs are connected to both VLANs, along with both System Directors, while clients can be connected to a single VLAN or both VLANs for highest availability. Client workstations may be connected directly to ISIS switch blade ports or through external edge switches that are connected in turn to ISIS switch blade ports. Clients achieve maximum performance when they are directly connected or connected via edge switches using Layer 2 routing. Layer 3 routing is permissible, but, depending on what other network traffic is on a routed network, performance may not be guaranteed. ISIS System Director The Avid ISIS 7000 System Director (SD) stores metadata information for up to 10 million files, including directory structure, access rights, and file creation times. Chunk location, however, is determined by the data distribution techniques implemented in the client and ISB micro-server. This metadata is protected by ISIS through the use of multiple copies on mirrored, hot pluggable disks in each System Director. To provide maximum metadata availability, system directors are typically deployed as a redundant pair with dedicated redundant network connectivity. 7

8 ISIS Client Software The Avid ISIS 7000 client uses standard operating system calls for basic read and write operations, allowing applications that aren t optimized for ISIS to access files stored on ISIS. Additional Avid extensions provide control of QoS and optimal I/O size parameters. Latency requirements and frame delivery deadlines are communicated to the ISIS infrastructure by the ISIS client software installed on the editor workstations. Avid media editors utilize the frame delivery parameters to provide predictable response without congesting network switches. Whether or not its latency requirement for a particular chunk of media can be met, the protocol between the ISIS client and the microservers on the ISBs allows an ISB to communicate to the client. Otherwise, if mirrored protection is used, the client requests the chunk from the ISB that has the other copy of the chunk. ISIS Redundant Infrastructure The Avid ISIS 7000 Engine is designed with multiple levels of redundancy. Redundant power supply modules with separate power inputs provide power continuity to all internal devices. All major system components are redundant and can be hot swapped, including ISIS Storage Blades, Switch Blades, and Power/Cooling Blades. Redundant System Directors are deployed with redundant connections to ISIS. Client connections are also redundant. In addition, all ISIS blades have redundant OS and file systems images. System wide health monitoring reports real time statistics and provides event notification. 8

9 Avid ISIS 7000 Technology Comparison Few technologies offer the package of features and benefits that ISIS offers. Competing technologies fall in two classes: 1) integrated file system and storage, and 2), shared file system on general purpose storage hardware, e.g., a RAID array. Integrated File System and Storage Infrastructure Some commercial products that appear to use a similar architecture to ISIS offer an integrated file system in a clustered or grid architecture. However, these cannot enable the facility to express real-time requirements. While these products claim to deliver high throughput they cannot, as a result, provide guaranteed ability to meet real-time constraints since they have no latency controls. While clustered or grid systems often claim scalable recovery, their implementations offer limited performance benefit. All Avid ISIS 7000 storage blades participate in recovery delivering fast redistribution while the recovery computations of alternative systems are limited to a few director blades and, therefore, extend the duration of the unprotected state. A large ISIS system can recover at the rate of 1 GB/second whereas typical alternatives only recover at below 200 Mbytes/second. Shared File System on General Purpose Storage Hardware General purpose file systems that use commercial storage arrays for storing data can be implemented for shared media storage. These systems rely on RAID storage arrays to provide data protection against disk failures. This approach clearly separates file system functionality from data protection functionality, allowing some of these file systems to run on a variety of storage hardware. However, when compared to Avid ISIS 7000, these systems have the following drawbacks: 1. No ability to express real-time constraints: Since file system clients typically access the storage array using the SCSI protocol and the storage arrays control disk scheduling, it is difficult to add functionality to these systems. 2. Since these systems rely on the storage array for recovery after a disk failure they are limited to the capabilities of the RAID array controller. While Avid ISIS 7000 offers a scalable recovery method that improves recovery rate as the size of the system increases, traditional storage arrays are limited in their recovery rate by the rate at which data can be written to a single disk typically performing at less than 100 MB/s. Moreover, the rates achieved in practice are far less than this. In addition, the storage array recovers the entire failed disk because it does not have information about which blocks of the failed disk were in use. The advantage of Avid ISIS file system is that it is integrated with the storage infrastructure, so only blocks that were a part of a file need to be recovered. 3. Administrators end up dealing with the file system and the storage infrastructure as two separate entities, often requiring a system integrator to serve as point of contact for problem resolution. 9

10 Storage and Scalability Summary No ability to express real-time constraints: Since file system clients typically access the storage array using the SCSI protocol and the storage arrays control disk scheduling, it is difficult to add functionality to these systems. Since these systems rely on the storage array for recovery after a disk failure they are limited to the capabilities of the RAID array controller. While Avid ISIS 7000 offers a scalable recovery method that improves recovery rate as the size of the system increases, traditional storage arrays are limited in their recovery rate by the rate at which data can be written to a single disk typically performing at less than 100 MB/s. Moreover, the rates achieved in practice are far less than this. In addition, the storage array recovers the entire failed disk because it does not have information about which blocks of the failed disk were in use. The advantage of Avid ISIS file system is that it is integrated with the storage infrastructure, so only blocks that were a part of a file need to be recovered. Administrators end up dealing with the file system and the storage infrastructure as two separate entities, often requiring a system integrator to serve as point of contact for problem resolution. One ISIS Engine 32 TB Twelve ISIS Engines 384 TB Resolution Mirrored Hours Streams Mirrored Hours Streams DV , DV - 50 / IMX , DNxHD , * DNxHD , Uncompressed HD ^ * 120 Clients, 2 streams each + 84 clients, 2 streams each ^ 16 10bit 1080i clients, 2 streams 10

11 Conclusion Avid ISIS 7000 leads the market in providing collaborative storage solutions for professional media environments. Performance: Avid ISIS 7000 delivers the highest possible performance with its unique latency controls and ability to read from primary or backup copies based on instantaneous load. This allows workgroups of all sizes to share media assets without compromising productivity. Redundancy: Avid ISIS 7000 architecture prevents media loss and down time due to unavailable media. This ensures that projects are completed on time and on budget. Scale: Avid ISIS 7000 offers massive scale up to 384 TB to support ever-increasing online media storage requirements. 11

12 Allowing creative staff to focus on content creation using their choice of tools and media, Avid is committed to supporting the broadest range of media formats, resolutions and third party editing products in order to keep pace with growing media storage demands and ensure your storage solution remains future-proofed. You can be assured your media assets are protected and accessible at all times with Avid ISIS 7000 Corporate Headquarters AVID (2843) Asian Headquarters European Headquarters Avid Technology, Inc. All rights reserved. Product features, specifications, system requirements and availability are subject to change without notice. Avid, the Avid logo, and ISIS are either registered trademarks or trademarks of Avid Technology, Inc. or its subsidiaries in the United States and/or other countries. All other trademarks contained herein are the property of their respective companies. AIWP0810 To find your regional Avid office, visit