Huawei Video Cloud Storage Solution

Transcription

1 Huawei Video Cloud Storage Solution

2 Contents Trends in Video Surveillance Storage Huawei Video Storage Solution Growth with Customers 1

3 Trends in Video Surveillance Storage Applicable to smallscale surveillance systems with a modest data amount, most sophisticated technology, in use for a long time, and remaining sales opportunities Early 21 st century Applicable to small-scale environment that does not require scalability or high performance 2004 Applicable to medium-sized and large surveillance systems requiring modest performance TV wall 2006 Media server IP SAN LAN/WAN Client Image capturing 2008 DVR/CVS DVR/CVS Primary surveillance center Remote surveillance NVR stack Applicable to various surveillance systems, highly adaptive, yet unreliable After 2010 Storage Intelligent Video server cluster analysis Video surveillance server cluster network Gov. administration Secure user/resource peripheral platform (video channel) Public security Gov. video resource access and Video video access platform External video sharing platform application access platform server cluster Storage device divided into storage pools, on-demand capacity allocation, smooth upgrade, easy management, and applicable to municipal security system construction and video surveillance over a large network District or county public security user Platform server monitoring system DVR local storage system DAS storage system Network storage system NVR storage system Cloud storage system 2

4 Surveillance Challenges Facing Traditional Storage Fast growing camera market and HD videos generate a huge number of video files, complicating storage planning. Multi-service systems require multiple storage platforms with a high building cost and upkeep. As a large number of videos are restored, data recovery and file retrieval both take a long time. Big data analysis and intelligent video analysis demand high bandwidth and performance. 9:45 12:00 18:00 3

5 Huawei Video Surveillance Cloud Storage Massive storage space Able to store a huge number of videos Brilliant performance High concurrence and data sharing among different systems and departments Storage pool Intelligent Convergence Intelligent storage Distributed parallel architecture for fast retrieval and intelligent analysis Robust security and reliability Guaranteed service continuity of diverse systems 4

6 Contents Trends in Video Surveillance Storage Huawei Video Storage Solution Growth with Customers 5

7 Video Surveillance for "Safe City" Project Video management platform Intelligent storage Cloud storage Primary surveillance center Branch surveillance center Cloud storage Client Client Cloud storage Branch surveillance center Surveillance spots Business district Residence Road Hotel Subway 6

8 Video Data Streams on Surveillance Platforms General surveillance center TV wall Cloud storage Cloud storage Intelligent analysis server Forwards to TV wall. Media server Branch surveillance center 1 Stores videos. Media server Intelligent analysis and retrieval Client viewing Stores Videos. Forwards to surveillance center. Uploads videos. IP Network Management server Forwards to TV wall. Client Client Media server History viewing Real-time viewing TV wall TV wall Cloud storage Branch surveillance center 2 The storage system in the general surveillance center provides longterm storage and archiving of all camera-recorded data. The storage systems in branch surveillance centers provide short-term storage and of partial camera-recorded data. 7

9 Huawei Cloud Storage Features Simplified management No instant maintenance needed No data planning needed Automated alarm notification SafeVideo Outstanding performance High disk density high utilization 6400 Mbit/s bandwidth/node Parallel architecture 40 PB single namespace Single node added in 60s Smooth upgrade to 288 nodes Excellent scalability Global view Free from instant maintenance Global data redundancy for automated restore Disk failure rate down by 70% 1 TB/h reconstruction speed Robust reliability 8

10 Huawei Cloud Storage System Summary:OceanStor 9000 HDFS Mode NFS/CIFS mode Ethernet Switches GE/10GE IP Network Storage Nodes OceanStor 9000 Storage Nodes Distributed Architecture Multi Protocol Support The OceanStor 9000 employs a distributed architecture, where many storage nodes are stacked to meet the needs for high concurrency and bandwidth of video surveillance.. 9 File system protocol : NFS/CIFS/HDFS Management: SNMP/NTP/SSH

11 Cloud Storage Advantage: Architecture Advancement Cloud Storage SAN Storage Ethernet Switch Controller Enclosure GE Storage Nodes... Disk Units Distributed architecture: All nodes participate I/O, eliminating performance bottleneck. Scale-out expansion improves performance with volume. Max capacity of 40PB meets the requirement of video surveillance such as large volume, high band and concurrency. Hardware of X86 simplifies O&M. Surveillance manage software can be installed on the hardware platform, saving the cost of servers. Architecture of 1 drives N: One controller drives N disk units. Only controller is responsible for I/O which often becomes the performance bottleneck. Scale-up expansion can t improve performance when capacity expanded. Only the controller adopts X86 architecture. 10

12 Cloud Storage Advantage: Reliability Comparison Cloud Storage With SafeVideo patent technology, no matter how many disks breakdown, if there is one fault-free disk video data can be read from the normal disk. SAN Storage In the SAN storage,if two or more disks of one RAID5 set breakdown at same time, all the video data stored in this RAID set will be lost. SafeVideo : make fault-free disk space readable The total video data Cloud Storage has higher reliability,fault data recovery rate up to 1TB/hour. And data recovery rate is 0.1 TB/hour in SAN storage. SAN Storage : after 2 disk breakdown all data are lost. 11

13 Cloud Storage Advantage: Service Reliability(2) Cloud Storage SAN Storage 1 File is devided into M pieces, N pieces of parity produced then. All the pieces are written into M+N nodes m 1 m 1 m 2 m 2 m 3 m 3 m 4 In case of node failure, damaged data can be restore through algorithm with data in other normal node. m 4 2 n1 n2 n1 n2 XOR calculation data A0 B0 C0 D0 A1 B1 C1 E1 A2 B2 D2 A0 A1 A2 A3 P4 B0 B1 B2 P3 B4 C0 C1 P2 C3 C4 D0 P1 D2 D3 D4 P0 E1 E2 E3 E4 Cross-node RAID is super set of RAID,more reliable and flexible than RAID. Multi node participate parallel recovery of failed node, improving recovery speed to 1TB per hour. Cloud storage can endure multi disk failure without immediate replacement. Data protection usually adopts RAID5 : N+1(parity)+1(hot backup) After one disk failed, the data is recovered to hot backup disk, which will take about 22 hours for 4T data recovery and make performance decline more than 50%. Failure disk must be replaced immediately. So maintenance is heavy. 12

14 Cloud Storage Advantage: Management Convenience(1) Cloud Storage SAN Storage Controller Enclosure GE Ethernet Switch... Disk units Scale up with complex configuration Scale out with simple configuration Rapid deployment by Plug and Play, unnecessary to plan RAID and LUN; Capacity can be expanded in 3 levels: disk, node and domain. It take only one minute to expand one node; Capacity expansion has no impact on surveillance service system. When deployment, LUNs must be planned and formatted. It will take more than 20 hours to format a LUN and more than 100 hours to format all LUNs of a SAN storage; Capacity is increased by adding disk units. When number of disk units exceeds a limit, a new controller should be added which means a new storage system. After storage expansion, new volume should be configured to surveillance management system. 13

15 Cloud Storage Advantage: Management Convenience(2) Cloud Storage SAN Storage Service servers Service servers POSIX interface Global view /root NFS NFS NFS /root/d1 /root/d2 /root/d3 Storage infrastructure IP SAN Storage Node A Storage Node B Storage Node C Cloud storage manages cross-region resources and presents global unified view to service servers. Storage load can be balanced within cloud storage system. Physical resources are shielded to surveillance management and run as a storage pool to upper system. Every IPSAN is managed respectively, no load balance. Storage systems couple closely with surveillance management system. Two parts impact each other when configuring and managing, improving management cost. 14

16 Scalability Comparison(1000 lines) Cloud Storage SAN Storage Media Server Global Namespace /root POSIX Interface Added Server Media Server Added Server /root/d1 Cloud Storage Platform /root/d2 /root/d3 Original storage Node Add 2 storage domain, still be one storage system ; Add 2x10GE ethernet link; /root/d5 /root/d4 Add storage Node Add 7 media server,assign new work directory for new add server. 15 /root/d6 Original disk array(9) Add 5 controller enclosure,35 disk enclosure,up to 14 storage system; Add 15xGE ethernet link; Add Disk array(5) Add 7 media server, assign LUN for each server and spend 30 day / person to format the LUN. Cloud Storage is easier to expand, need less time and less network interface, no need to interupt the application.

17 Contents Trends in Video Surveillance Storage Huawei Video Storage Solution Growth with Customers 16

18 Over a Decade of Sustained Investment and Accumulation Developers 5 Research centers Sharing core technologies and platforms of Huawei chips, networks, hardware and software New T Series unified storage launched Dorado5100 solid state storage launched 18500/18800 high-end Storage, UDS Cloud Storage, Oceanstor 9000 big data storage 2012 launched New T series SAN published Full SSD SAN Dorado 2100 commercial launched 2011 Media & Entertainment, data protection, Cloud solutions New generation SAN series commercial launched Cloud storage solutions Cloud storage were deployed in some Tele operators SAN products were widely deployed in Tele systems SAN and NAS were widely used in ISP industry High end and Middle range SAN, NAS Industry leading performance of SPC-1 and SPEC Full series NAS/SAN/VIS Product launched Commercial launched the 4Gb SAN product. Commercial launched the first FC SAN product F800. Joint Venture from Huawei and Symantec Storage research The first generation DAS product: D100 17

19 Constant Innovation, Contribution to Storage Standards and Patents 600+ Over 600 storage-related patents authorized and accepted 30+ More than 30 proposals made official standards 10+ Responsible for formulating standards in over 10 Chinese and international storage standardization organizations Joined standardization organization Administration role in standardization organization International China Post Standards documents SNIA Over ten proposals to SNIA INCITS SPC SPEC PCI SIG NVMe CNITS SOA CCSA SNIA TC member Joint chairman of SMI Core TWG Deputy leader of SOA cloud computing (including storage) group, CNITS Two proposals to INCITS Seven CCSA standards documents Six proposals to IETF IETF 18

20 Committed to Customer Success Through Partnership Committed to Customer Success Industry Partners Consulting / Design Partner ICT partners (100 +) 19

21 Global Practice of Huawei Safe City Solution Russia Belarus Hungary Kazakhstan Mongolia Turkey Tajikistan China Mexico Ecuador Venezuela Algeria Egypt Morocco Mali Niger Senegal Nigeria Ghana Angola Chad Ethiopia Kenya Tanzania Zambia UAE Pakistan Laos Malaysia Indonesia PNG Brazil Mozambique South Africa Djibouti Argentina Huawei s Safe City solution services over 40 cities in more than 30 countries. Huawei s Safe City solution services over 40 cities in more than 30 countries. 20

22 Middle East KC "Safe City" Project Front-end device DVR/DVS Surveillance client Recorded videos Surveillance LAN Surveillance platform Video playback data Management node Customer Requirements The city needs 1980 cameras and 6.8 PB storage capacity. The storage system needs to provide central storage and management. The system implements multi-level reliability protection to prevent data loss. Storage capacity can be flexibly expanded without affecting services. Huawei Solution The large-scale distributed clustered system, integrating computing and storage capabilities of massive nodes, provide parallel processing and up to 6.8 PB video storage by horizontal expansion. The unified management platform enables automated deployment and capacity allocation of a thousand storage nodes Powerful real-time remote backup ensures data reliability. Customer Benefits Storage domain Storage domain Storage domain Unified storage resource pool: automated restore, inexpensive and simplified maintenance, and guaranteed maintenance personnel safety Reliable cloud storage design: failure rate down by 30%, integral monitored data, and reduced maintenance cost 21 Scale out: linear expansion of capacity and performance

23 Middle East XX "Safe City" Video Surveillance Project Metadata nodes Management server Media server 16 x 10GE Customer Requirements XX district government invests $20 million in building a video surveillance system to cope with political tension, collect civil information, and improve public security. A total of 1200 cameras are needed to cover the entire city. The system capacity is around 1.5 PB and meets 6-month storage needs. Huawei Solution The solution, combining LTE, IVS, and cloud storage products, delivers storage system competence. The large-scale distributed clustered system, integrating computing and storage capabilities of massive nodes, provide parallel processing and 1.5 PB video storage by horizontal expansion. Customer Benefits Cloud storage backup of 6 months, 1.5 PB usable capacity The end-to-end solution improves overall delivery efficiency. The cloud storage architecture meets subsequent requirements for large capacity expansion, saving expansion investment. Unified management reduces TCO. 22

24 Why Huawei Storage? Unified hardware platform ensuring maximum ROI Over 10 years' innovation and expertise in Storage technologies Vendor neutral or integrated without platform lock-in Constant pursuit of best performance and costeffectiveness 23

25 HUAWEI ENTERPRISE ICT SOLUTIONS A BETTER WAY Thank You! Copyright 2012 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice. Page 24

26 OceanStor 9000 Networking Recommendations Back-end and front-end 10GE networking Management server GE switch 10GE switch (provided by the customer) 10GE switch 10GE IPMI Application server Front-end network OceanStor9000 Back-end network Flexible internal and external networking choices Redundant switch configuration IPMI independent network 25

27 Intelligent Load Balancing Linear Scalability Elastic Space High Reliability Management Simplicity Intelligent load balancing among all nodes OceanStor 9000 Automatic capacity and performance balancing WushanFS Free from manual intervention or configuration modification WiseLink WiseLink WiseLink WiseLink Transparent to applications node node node node 26

28 Dynamic Storage Tiering, Improving Resource Utilization Linear Scalability Elastic Space High Reliability Management Simplicity WushanFS Automatic migration of non-hotspot data, improving utilization P node WiseTier B node WiseTier File-level fine granularity, accurate hotspot locating based on I/O access frequency. The migration process is fully transparent to users, without affecting applications. Hot data Cold data 27

29 Efficient Global Cache, Improving Access Efficiency Linear Scalability Elastic Space High Reliability Management Simplicity Application server A B C Global cache Up to 55 TB cache pool WushanFS CPU CPU CPU Memory Memory Global cache Memory Increasing the cache hit ratio by 50% Storage Storage Storage Improving cache space utilization Node Node Node 28

30 WushanFS: Largest Single File System in the Industry Linear Scalability Elastic Space High Reliability Management Simplicity Application server Separate expansion of performance and capacity Symmetric distributed architecture without individual metadata nodes Even distribution of metadata for eliminating performance bottleneck of metadata Up to 40 PB of single file system WushanFS 29

31 On-Demand Expansion, Decreasing Procurement Costs Linear Scalability Elastic Space High Reliability Management Simplicity Application server Scalable from 3 to 288 nodes linearly Expansion of a single node completed within 60 seconds Free from manual intervention or any modification 30

32 Distributed RAID, Providing Highly Reliable Data Protection Linear Scalability Elastic Space High Reliability Management Simplicity Switch Disk utilization up to 95% +1 ECC +3 ECC +2 ECC +4 ECC N+1 to N+4 data protection File-level data protection 1 TB/hour data recovery 31

33 Comprehensive Visualization, Unified Resource Management Linear Scalability Elastic Space High Reliability Management Simplicity WiseQuota: sets quotas for nested directories to facilitate space management. 100 TB Fast automatic deployment 10 TB 20 TB 40 TB 30 TB Comprehensive performance monitoring Unified topology display User User User group User group 20 TB 10 TB Centralized management of a single file system of 40 PB 32 User User

34 SafeVideo: Proprietary Technology for Video Data Protection Outstanding Performance Excellent Scalability Robust Reliability Simplified Management A complete video sequence Tradition data protection: All data is lost if two member disks of the RAID 5 group fail. Advantages The failure or damage of any number of disks does not affect the video streaming. Only data on the failed or damaged disks is lost. The risk of all data loss caused by disk failures is greatly reduced. SafeVideo: Data in normal space can be read in the event of a RAID failure. 33