Market Landscape Report

Transcription

1 Market Landscape Report Object Storage By Scott Sinclair, Analyst June 2015

2 Market Landscape Report: Object Storage 2 Contents What Is Object Storage?... 3 Limitations of RAID and the Need for Object Storage... 3 Object Storage Provides a New Level of Scale and Protection... 4 Additional Considerations about Object Storage... 6 Market Situation: Technologies, Challenges, Opportunities... 6 ESG Research Findings on Object Storage... 8 Historical Market Challenges of Object Storage... 8 Making an Educated Investment Object Storage: Core Functionality Considerations Object Storage: Considerations by Use Case The Bigger Truth ESG Object Storage Coverage Vendor Capsules All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at

3 Market Landscape Report: Object Storage 3 What Is Object Storage? If there is one universal truth when it comes to information technology over the past decade, it is that data growth is inevitable and unstoppable. The sharp rise in digital assets has been a consequence of multiple cultural shifts over recent years. Whether driven by personal or professional motives, individuals are simply creating more digital assets than ever before. Regardless of the industry, the success of corporate operations depends on the ability to utilize digital assets. Whether it s media and entertainment leveraging higher resolution video or developing more realistic digital effects, energy exploration firms capturing detailed 3D or 4D seismic data, security systems capturing higher resolution security footage, or online content distribution, the creation and the efficient utilization of digital assets is critical to staying competitive across nearly every line of business. Along with this surge of digital content, and the fact that our corporate culture is focused on litigation, comes an increase in compliance regulations and internal controls and audit processes that require organizations to be more responsible than ever when storing and curating content. In some cases, organizations have responded to the possibility of future litigation by instituting a culture where data is kept for decades and potentially longer. With the emergence of machine data and with the Internet of Things (IoT) on the horizon, content creation promises to expand well beyond the previous limitations of content created by just the human populace. The net result is a future where nearly everyone and everything will be creating some form of data that will have to be stored and protected for some period of time (if not indefinitely). Object storage, more than any other storage architecture, is designed to store and protect the resulting massive content repositories. Just as solid-state technology has emerged in response to the increased demand for performance, spinning media has begun to specialize in high capacity and cost-optimized storage with larger capacities, with the recent availability of 6 TB hard drives, and higher capacities on the horizon, such as Western Digital s 10 TB drive announced in September of Recent drive advances have greatly reduced the price per capacity for storage hardware infrastructure, enticing organizations to keep more data longer. These relative savings, however, will likely be offset in aggregate by the sheer amount of data growth expected to take place. The net result is expected to be a perfect storm that couples exponential data growth with an increase in organizational policies requiring that all data be kept online virtually forever. Despite cost reductions of the underlying storage hardware infrastructure, storage system capacities have reached a tipping point, a threshold where traditional storage and protection technologies, such as file systems, are no longer viable options. Object storage architectures provide the necessary capabilities critical to storing and protecting high capacity content environments, and with the projected growth of digital content over the next few years, object storage looks to become a much larger portion of enterprise storage deployments. Limitations of RAID and the Need for Object Storage Object technology delivers a direct response to the challenge of storing and protecting large amounts of unstructured data. Traditional storage arrays, which often leverage RAID technology, hit serious limitations at scale. ESG recently conducted an in-depth research survey of 601 IT professionals concerning their organizations IT spending plans and priorities for With a flurry of multiple recent high-profile security breaches, it was not surprising that information security initiatives was the most popular response provided. Managing data growth, however, was the third most-cited response, up from fourth in Upon closer examination, concerns over data growth seemed to increase with the amount of capacity managed. Over one-third (35%) of organizations with at least 1 PB of disk-based storage capacity cited managing data growth as a top IT priority, compared with only 22% of those with less than 100 TB. 1 As data grows, the challenge of managing that data increases. This is where object storage is designed to assist. 1 Source: ESG Research Report, 2015 IT Spending Intentions Survey, February 2015.

4 Market Landscape Report: Object Storage 4 Figure 1. Top Ten Most Important IT Priorities over the Next 12 Months Top 10 most important IT priorities over the next 12 months. (Percent of respondents, N=601, ten responses accepted) Information security initiatives 34% Improving data backup and recovery Managing data growth Increasing use of server virtualization Desktop virtualization Using cloud infrastructure services Regulatory compliance initiatives Business continuity/disaster recovery programs Building a private cloud infrastructure Improving collaboration capabilities 26% 26% 25% 25% 25% 24% 23% 22% 22% 0% 10% 20% 30% 40% Source: Enterprise Strategy Group, There are also nuances associated with protecting data at scale. While traditional storage systems with RAID were designed primarily to protect against single drive failures, larger hard drive capacities have translated into an increased likelihood of multi-failure events, which leaves the organization exposed to the danger of losing data should a second drive failure occur during the rather lengthy rebuild time associated with high density drives. Multi-failure scenarios arise in a couple of different ways: additional drives could fail during a rebuild or bad sectors on a previously designated healthy drive could be uncovered during a rebuild and result in unrecoverable data. The likelihood of unrecoverable bits used to be low. However, with larger capacity drives, the chances of not being able to recover some data have increased significantly in recent years. As a result, the industry has shifted from recommending RAID 5 to recommending RAID 6, which protects data in the event of a dual drive failure. While the addition of a second parity bit has alleviated some concerns about finding an unrecoverable bit during the rebuild process, the introduction of larger drives is also resulting in longer rebuild times, increasing the amount of time data is exposed to the possibility of experiencing a secondary or tertiary failure. And while the odds of a multifailure event are still low, they can increase to undesirable levels for organizations that have hundreds or thousands of drives. These risks are further exacerbated by the fact that massive capacity environments are often too large to back up using traditional backup methodologies, meaning that in many cases these repositories may store the only copies of the data available. Object Storage Provides a New Level of Scale and Protection This section intends to provide some technical insight into object storage and to provide some discussion on the background and history of object storage technology. Traditional file systems are designed to store content in a hierarchical manner, often in a tree of files and folders. In these systems, users access a file by following a path to a specific location. While this method can be intuitive for storing a few files, when content storage explodes to billions or even trillions of files, a hierarchical access method can create too much complexity and in some cases overwhelm traditional file system storage architecture. Some file system-based solutions can run into limitations

5 Market Landscape Report: Object Storage 5 once a certain number of files and folders is reached. Additionally, some file system architectures are unable to effectively scale metadata, greatly impacting the performance of metadata operations once the system exceeds a certain amount of capacity or number of files. Object systems are designed using an alternative approach with a single and massively scalable flat address space where file access is provided via a unique identifier. An analogy that can help describe this difference is that accessing a specific file in a file system is like following a set of directions to find a location for example, take the first left, then the second right, etc. Object storage, on the other hand, is like using GPS coordinates. This more efficient manner of identifying specific content helps enable object storage to scale to higher capacities than can be achieved by traditional file systems. Another unique element of object storage is the option for more advanced metadata. Metadata is a term used for the data that stores information about particular digital data assets. File systems only support simple metadata information: basic file type and location information. Object storage, on the other hand, provides the ability to store advanced and/or custom metadata, which allows for more information to be stored with specific content. For example, object storage can store protection and retention requirements or search-related information with the object itself, improving the ability to run analytics or customize protection schemes. An example might be a medical archive of x-ray content, where the patient s information and diagnosis are stored in the image s metadata. The richer the metadata, the faster and more valuable search activities can be, which might be especially needed in environments with large content requirements. In summary, object storage architectures are designed with massive scalability in mind, increasing the scalability to near infinite levels along with the ability to support more efficient search mechanisms. In addition to these advances, object storage also provides unique capabilities to ensure that data is protected at scale. Object storage solutions were designed to solve the challenges of protecting massive capacity environments where traditional backup is often not an option. Object storage architectures provide robust protection by expecting multi-failure scenarios to be common occurrences. Object storage provides several innovations to help accomplish a more robust and bulletproof ecosystem: Replica or erasure coding versus RAID: Instead of traditional RAID, object storage commonly leverages the use of object replication, erasure coding, or a combination of the two to provide data protection. These technologies provide several advantages over traditional RAID, including the ability to leverage commodity hardware, the ability to apply specific protection schemes to specific objects or groups of objects rather than on a volume level, and the ability to evolve the protection scheme over time, e.g., to reduce protection from four replicas down to three over time. Additionally, by using replicas or erasure coding, object storage can automatically expand protection across sites by spreading data across multiple drives, nodes, and even geographies. Rebuild into free space: The majority of traditional RAID storage environments are typified by the hot spare, a free drive waiting to take over after a production drive fails. However, in the event of a failure, only one drive is the write target for rebuilding the data, creating a significant bottleneck that is further exacerbated as drive capacity increases. The vast majority of object storage solutions provide the ability to rebuild data from a failed drive into free capacity across many drives and nodes in the system, significantly speeding up the recovery process thanks to the massively parallel nature of leveraging many drives (versus one), and therefore greatly minimizing the time the system is in a degraded or vulnerable state. Self-healing: Another innovation designed to greatly speed up recovery time and nearly eliminate the risk of encountering unrecoverable bits during the rebuild is the ability to self-heal. Object storage solutions often provide background processes that read presumably healthy data, and verify that the contents are intact. When an unrecoverable bit is identified, the object is rebuilt in available free space, ensuring that data is always healthy and readable. Additionally, since it is common for drives to fail a few sectors at a time rather than all at once, with self-healing, it is likely that most of the data will already be rebuilt to healthy sectors on other drives when the drive in question finally is determined as failed.

6 Market Landscape Report: Object Storage 6 Automatic geo-protection: Traditional storage arrays often rely on file system or volume/lun-based replication for multi-site protection. These traditional storage containers are also often bound by some architectural capacity limit, 16 TB for example. Providing multi-site protection for large capacity content storage environments with these limitations can result in the unwieldy management of scores of replication policies, if not more. Multiple object storage solutions support a massive, near-infinitely scalable flat address space, with the capability for automatic multi-site protection built in, greatly simplifying multi-site protection and greatly reducing the associated management costs. Enhanced scalability and resiliency are not the only advantages of object storage infrastructures, but they are key architectural capabilities that separate object technology from more traditional file and block storage methodologies. Despite a number of advantages, however, object storage is not a panacea for all storage concerns. Additional Considerations about Object Storage Object storage systems are designed around storing and retrieving entire objects, as opposed to block-based or file system-based technologies that are designed to read and write individual blocks of data. The most significant impact of that difference is experienced in write performance, especially when a user or application wishes to modify a file. Instead of simply applying the modified blocks of data, object storage systems require the entire object be read before it can be modified, and, once modified, the object is rewritten in its entirety to the storage platform. This architectural difference led object storage systems to originally house workloads that were often write-once-read-many (WORM) environments, such as medical imaging, energy exploration, or other workloads with image or video media content. These environments are typified by content that is created and rarely modified, but that needs to be kept online and at some point retrieved. Heavy transactional workloads are typically better suited for file or block storage environments. However, advances in processing and memory technology along with the integration of solid-state have improved the performance capability of object storage solutions considerably. Some object storage providers, such as Scality, have taken advantage of these performance advances, expanding use case support for more transaction virtualization workloads. In its RING 5.0 release, Scality claims a 95% improvement in performance over the previous generation and support for up to 200 IOPS per VM. In several cases, object storage solutions can be designed to serve transaction use cases, although perhaps not as efficiently as other storage technologies. Another consideration and potential limitation for object storage is limited storage protocol support. Traditionally, access to object storage is available via programmatic access over the HTTP protocol. While HTTP access can offer some advantages, a lack of more popular storage protocols has limited application support. This challenge is discussed in more detail later in this report, but it is important to note that recently a majority of object storage solutions have expanded protocol support to include more common storage protocols such as NFS and CIFS. Additionally, S3 has emerged as a more widely supported object protocol. Despite inefficiencies in high transactional workloads or limited protocol support, as organizations experience increases in data growth, object storage systems are becoming a more significant portion of their IT storage strategies. As such, a growing number of object storage offerings that look to solve the challenges of high capacity content storage are available in the market. Market Situation: Technologies, Challenges, Opportunities Various object storage players have emerged over the past few years. While participants often differentiate on delivery model and on advanced functionality, all object storage players are epitomized by the ability to massively scale and protect at scale. For this report, the focus will be on on-premises solutions. While off-premises cloud object storage solutions exist, such as Amazon s S3, those solutions are outside the scope of this report. When segmenting various on-premises object storage solutions, deployment model differences, along with whether the technology is open source or proprietary, provide some clear delineation. When it comes to deployment model, there are essentially four different options: Proprietary hardware-based object solutions

7 Market Landscape Report: Object Storage 7 Proprietary software-based object solutions Open source object storage solutions Scale-out file system solutions that support object protocols Proprietary hardware-based solutions are delivered as hardware-only solutions; are delivered as either appliance or traditional dual controller, storage-based solutions; or require a shared storage infrastructure as part of the solution. Software-based solutions are available as software-only solutions that can be deployed on traditional commodity server hardware. While these solutions may also be offered as appliances, their availability as softwareonly options enables their inclusion in this category. Additionally, several available open source solutions are supported by an open source community, even though versions of these solutions may be supported by or purchased from software or storage vendors in certain circumstances. Finally, some scale-out file system solutions are starting to offer object-like functionality. The capabilities may be in the form of erasure-coded protection or object-based protocol access. These solutions, however, are based on a hierarchical file tree access model, and as such do not offer all of the capabilities of traditional object storage. For this paper, these file systems were included in the analysis since they are commonly seen as options for the same workloads and use cases that are common to object storage. Table 1. Four Object Storage Market Groupings Proprietary hardware-based (appliance or traditional storage) object storage solutions Proprietary software-based (available as a software-only option) object storage solutions Open source-based object storage solutions Data Direct Networks WOS Exablox OneBlox Huawei OceanStor UDS Imation Assureon NEC HydraStor Quantum Lattus Amplidata Himalaya Caringo Swarm Cleversafe Hyperscale Storage Software Platform Cloudian HyperStore EMC ViPR (also available as Elastic Cloud Storage appliance) Hitachi Data Systems Hitachi Content Platform (HCP) NetApp StorageGrid Tarmin GridBank Scality RING Storiant Basho Riak CS OpenStack Swift Redhat Ceph SwiftStack Swift Scale-out file systems that support object protocols EMC Isilon IBM Spectrum Scale Source: Enterprise Strategy Group, 2015.

8 Market Landscape Report: Object Storage 8 ESG Research Findings on Object Storage Despite the exponential growth and the subsequent pressure on traditional storage environments, object storage adoption currently remains at a relatively low level. During the second half of 2014, ESG conducted in-depth discussions with IT professionals responsible for their organizations data storage environments on a wide range of storage-focused topics including several next-generation architectures and solutions. This research identified a number of interesting trends related to object storage. First, the top data storage challenge identified in the study was data growth and the associated costs of storing and managing growing capacity demands. With 28% of the respondents storing over 10 PB of data, and 56% storing over 1 PB, a large percentage of those polled managed significant data capacities, suggesting that even larger capacity environments are seeing significant rates of data growth. 2 Second, while we expected the high number of large capacity environments identified in the research to translate into a relatively high adoption of object storage, that was not the case. ESG s overall impression from the study is that there is still a high level of uncertainty and/or confusion in users minds around object storage. A high percentage of participants were found to be block storage-centric in their understanding of data storage technologies. Naturally, these environments tended to have significant Fibre Channel storage footprints. When asked to discuss their thoughts on and impressions of object storage, many of the respondents demonstrated further confusion around the differences between file system and object storage technologies. While these results may seem negative on the surface, there is cause for optimism for the object storage market. ESG found that despite a significant level of confusion, those individuals who understood the details and the potential benefits of object storage technology showed an extremely high level of interest often associated with an active investigation into object storage offerings. Twice as many respondents identified themselves as evaluating/planning to use object storage, as the number that identified themselves as currently using. The use cases that received the most mentions by those evaluating object storage deployments were general-purpose unstructured data content storage, on-premises cloud storage, and storage for big data analytics. The end takeaway from these data points suggests that while the benefits of object storage resonate really well with potential customers (i.e., all of the participants who understood object storage stated that they were interested in the technology), an accurate object storage message is not reaching a wide number of potential customers. Additionally, for those organizations that were evaluating object storage, the most common reason stated was that the scale of unstructured data storage had exceeded the limit of what their existing NAS resources could support. Challenges with existing solutions were tied to protecting including limitations of RAID and time to replicate and managing data, together with the challenge of managing multiple file systems at scale. These results can be summarized as: Data growth is a major challenge even for large environments, many organizations still do not understand the benefits of object storage, those that understand the benefits are highly likely to actively investigate and deploy object storage solutions, and eventually some organizations reach a tipping point where object storage becomes a necessity. Historical Market Challenges of Object Storage This section intends to provide some additional commentary to help explain the current and relatively low levels of adoption of object storage across the industry. Then in the following sections, this paper will call out emerging trends that hold the potential to likely increase object storage applicability and adoption over the next few years. Early object storage deployments found success in low write and heavy read archive use cases. The long-term storage capabilities made the technology an ideal for archive, compliance, and vertical application content storagefocused solutions. Historical concerns about write penalties incurred by object storage architectures could be partly responsible, but the use cases were predominantly driven by the areas where organizations initially suffered the most pain from the combination of data growth and the preservation of content for long periods of time. 2 Source: ESG Research Report, Next-generation Storage Architectures, March All ESG research references and charts in this Market Landscape Report have been taken from this research report, unless otherwise noted.

9 Market Landscape Report: Object Storage 9 Use cases that saw the most immediate demand for object storage included industries where the primary means of doing business relied on the combined demand of long-term storage and relatively fast retrieval and analysis of massive quantities of data, such as health care and energy exploration, along with media and entertainment archive repositories, where operations required data to be available and online instead of stored offsite on tape media in a vault. Especially when usage patterns required the data to be secure and online, as in the case of a hospital where any patient may come in at any time, records needed to be readily accessible in a reasonable timeframe. Customized metadata tagging allowing for data to be categorized more effectively made object storage particularly useful in compliance workloads, where objects were designated as immutable for a certain period of time, such as seven or ten years, before being destroyed. However, despite many advantages for large-scale content storage, object architectures struggled to take market share from file system storage likely due to a few chief concerns. Lately, object storage providers have resolved many, if not all, of these concerns, enabling a more promising outlook for the technology: Adoption of standard protocols: Many of the initial object storage solutions provided connectivity via the HTTP protocol along with a programmatic REST interface. Without a standard interface, many applications required some level of customization and certification to provide support for object storage, and in many cases support for one solution did not translate to supporting all or even multiple object storage solutions. This fact limited the number of applications supported and could lock applications into a single storage offering. Early solutions were limited both by the flexibility of storage choices as well as the applications supported, leading deployments to often be specialized and tackled on a one-off basis when compared with other more general-purpose storage solutions. More recently, however, object storage solutions have added support for more standard protocols, such as NFS and CIFS. Additionally, S3 and Swift APIs have seen increased application support, with S3 fast becoming a defacto standard. Both of these movements have increased the application support of object storage solutions and reduced, or eliminated, the inhibitors described above. When evaluating object storage solutions, the greater the number of protocols supported, the broader the level of application support and speed of deployment as custom coding is reduced or eliminated. Wider application support: With object deployments tied to specific use cases or applications, object storage deployments often found themselves tied to application deployments. When object storage was deployed, it was often as part of a combined solution deployment where the storage vendor and application vendor worked hand in hand. In many cases, this limited the ability of object storage solutions to expand to other workloads in the data center. From the storage administrator s point of view, these solutions were often seen as directly tied to applications and not part of the traditional storage ecosystem. During ESG s research into next-generation storage architectures, ESG identified the fact that, among those organizations already using object storage, there was a tendency to strongly tie the storage solution to a specific application (often industry-specific). In fact, the data suggests the alignment was stronger for object storage solutions than would typically be seen for block or file storage, which may be attributable to application vendor recommendations (or even a prepackaging of the object storage solution with the application) taking the decision out of the hands of the traditional (storage) infrastructure decision makers. Data from individuals who identified themselves as currently evaluating object storage solutions, however, suggests that the trend appears to be changing, with their anticipated use cases being focused on more general-purpose storage infrastructure deployments than specific applications. The use cases that received the most mentions by those evaluating object storage deployments were general-purpose unstructured data content storage, on-premises cloud storage, and storage for big data analytics. Improved market perception: With block storage and file storage, was a third storage solution truly necessary? Apart from the specific content application and compliance need, the answer (whether founded or not) was often no. While many solutions are easy to use, organizations often perceived complexity and that additional skill sets would be needed to deploy object storage. Many of these challenges are greatly alleviated by the increase in standard protocol support by object storage providers. Also, with advances in storage performance technology, object storage is able to serve a wider range of workloads. Additionally,

10 Market Landscape Report: Object Storage 10 object storage historically saw limited support by the major players: While EMC offered Centera and then later Atmos, NetApp, Dell, HP, and IBM all promoted non-object-based solutions. While NetApp did acquire Bycast in 2010, potentially recognizing the future potential for object storage, the company did not start actively promoting the technology until recently. In similar fashion, EMC has released its Elastic Cloud Storage appliance based on the ViPR object storage technology. IBM has rebranded its General Parallel File System (GPFS) to Spectrum Scale and added object functionality. Additionally, Dell and HP have partnered with object storage providers. In addition to resolving many of the key challenges for object technologies, several new technology trends and use cases are driving demand for massive and affordable content storage technology. These trends hold the potential to greatly increase object storage adoption over the next few years: The Internet of Things and machine-generated data: Whether it is the Department of Defense gathering geospatial data for drones, a mobile application developer collecting GPS coordinate information, an airline monitoring aircraft maintenance information, or any of countless other solutions, data is not just being created by people anymore. Everything from construction and mining machinery, through home appliances, to clothing and apparel are becoming connected to the Internet and generating and creating data assets. All that data needs to live somewhere, be protected, be available, and be searchable in order to provide insights. Organizations looking at storing machine data will require infrastructures that can scale indefinitely, can store and protect massive amounts of content, and be affordable and easy to search and analyze. Object storage systems were designed to handle these very workloads. Hyper-scale and the private cloud: The emergence of public cloud storage has introduced several new concepts for the storage administrator, such as the ability to pay for only the storage used and the ability to leverage commodity hyper-scale hardware infrastructure. The promise of cloud solutions has enticed multiple executives to push IT organizations for recommendations and strategies. As organizations look to deploy on-premises private cloud solutions (particularly with OpenStack or CloudStack), a number object storage solutions combine scalability, a programmatic interface, and the ability to be deployed as software on commodity hardware, fitting the mantra of cloud and hyper-scale infrastructure. On the road to offpremises storage solutions, organizations are finding that some object storage solutions are delivering the economics of cloud storage in on-premises solutions. Future-proof with object storage for hybrid cloud storage: For off-premises public cloud storage deployments, archive storage is often seen as a potential use case for cloud storage. However, off-premises archives are being held back by security concerns and lack data governance. During ESG s in-depth discussions with IT professionals on next-generation storage architectures and solutions, multiple respondents cited a lack of data governance, specifically the inability to determine critical from non-critical data, as the top contributor to retaining data onsite. This inability to categorize unstructured data and isolate critical or sensitive material limited their organization s capability to identify which sets of data posed a lower level of risk and therefore could be safely moved off-premises. For many organizations, this is in part a technology issue and in part a process issue. The technology either isn t available to classify secure information and/or the internal policies and processes are not in place to define and restrict sensitive information. As organizations maintain content on-premises, but work to deploy the necessary data governance processes, object storage solutions that support cloud protocols, such as S3, can allow internal application development to future-proof applications by making them off-premises cloud-ready once internal policies and processes are put in place. The rise of big data and the data lake/hub: Often coupled with the rise of the Internet of Things, but not limited to it, more organizations are looking to analyze the data being collected, in addition to simply storing and protecting it. As a flurry of big data applications rise to the occasion to inspect and gather insight on massive pools of unstructured data, the data lake or hub is one potential method for storing all of this business-critical information. A data lake solution presents a single massive pool of storage that accepts content from multiple sources and protocols. The principle is to consolidate data from disparate sources in a single location, making future analysis much easier by reducing transformation costs and preventing

11 Market Landscape Report: Object Storage 11 critical data from being isolated to individual silos. The time it takes to move the large amounts of the varied content required for analytics from one location to another can significantly impede decision making and the time to results, driving up costs and hindering the potential benefits of analytics. If you can store all of that data in one place and perform analytics where the data resides in its native format, then it can save much needed time and money. The ability to store and access vast amounts of varied data can provide higher value analysis than that which data warehousing alone can provide. Object storage provides an excellent solution to store and protect that data as it scales. Considerations around data lakes or hubs: Given the relative infancy of the big data industry, many organizations are still deciding on the right storage solution. One of the benefits of a data lake/hub model is that the storage resides in its native format, allowing organizations to skip the costly data transformation step. Recently, some critics have expressed concerns that bypassing data transformation shifts the responsibility of understanding the context of the raw content to those performing the analysis. While it is important to understand these concerns, often the cause can be more the result of a lack of internal policies and processes for data governance than anything specific to the storage solution. This is where the descriptive metadata capabilities of object storage can provide a significant advantage. However, organizations must implement the internal controls necessary to ensure that the organization can achieve the maximum value from descriptive metadata. The key message is that while a data lake/hub solution can help solve many of the challenges with big data implementations, it cannot solve all of them. Other concerns around data lake solutions relate to performance. Low-latency performance can be critical for analytics. As mentioned previously, object storage architectures may not provide the performance desired for more frequent data analysis. One potential remedy to achieve the benefits of both worlds is to perform more basic analysis inside the data lake and then use those findings to isolate the smaller data sets that should be moved to another platform for more in-depth analysis. Data lake or hub solutions, when deployed using object storage, can enable organizations to store everything, then distill and analyze as needed, ensuring that the data is available when required. Making an Educated Investment When evaluating object storage solutions, there are a number of considerations to include in the investigation. This section will review specific considerations and trade-offs for core object storage capabilities, and a later section will review feature and functionality implications for specific use cases. The goal of this section is to provide a framework to use when evaluating object storage solutions in both general and use-case-specific evaluations. Included in this section are examples of functionality provided by various object storage solutions. However, organizations must always ensure that the latest information is used when evaluating any object storage solution. Object Storage: Core Functionality Considerations Object storage is defined by the ability to scale and protect at scale. However, all solutions do not protect or scale data in the same way. These several categories of capabilities should be considered during a technology evaluation: Replication versus erasure coding: The vast majority of object storage solutions leverage a replication methodology, an erasure coding methodology, or some combination of both to protect data. The replication protection scheme replicates an object when it is written to the object store and stores multiple copies (often three or more) of each object across multiple nodes in the environment. It is therefore common for object storage systems to require at least three nodes in a pool to provide the necessary level of resiliency. By proving three separate physical nodes, the solution can lose two without risking data loss. The trade-off of this scheme is that the raw capacity for the system is required to be three times the capacity of the content being stored. While object storage systems often couple this protection scheme with the ability to leverage lower cost hardware, using only a replica-based protection scheme can lead to a substantially large hardware infrastructure deployment. While the addition of deduplication or compression can help alleviate the raw capacity concerns, solutions that only offer replica-based protection require a significant increase in the raw capacity when compared with similar erasure-coded solutions.

12 Market Landscape Report: Object Storage 12 Erasure coding, on the other hand, is similar to RAID where failure protection can be provided without a direct multiplication of the raw capacity by using parity information. The difference between erasure coding and RAID is that erasure coding is more flexible and can be distributed across multiple drives, nodes, or even sites for multi-site failure survivability. While erasure coding provides multi-failure protection with far less raw capacity than leveraging a replica-based protection scheme, erasure coding requires more processing power to calculate the coding schemes, which can impact performance, or may require more processing hardware to achieve the same level of performance. Additionally, reading an object requires accessing content from multiple nodes and possibly multiple sites, potentially impacting the read performance. However, erasure-coded schemes can provide the flexibility to protect against a large number of simultaneous failures, if desired, which may not be feasible when using a replica-based protection scheme. For example, Scaility s RING architecture supports protection for up to thirteen nines of reliability, and Amplidata s Bit Spread technology can be configured to support up to fifteen nines and can tolerate up to 19 simultaneous failures with no data loss. Of course, the more failures the system is configured to withstand, the larger the hardware investment, so it is important to understand the impact of higher resiliency on the cost of the solution and balance accordingly. In response to the trade-offs, some solutions offer a combination of erasure coding and replica capabilities, allowing administrators to select between improved performance and improved capacity optimization. For example, Caringo s Swarm, DDN s WOS, HDS s HCP, NetApp s StorageGRID, and Scality s RING offer the ability to select between replication and erasure-code-based protection. Areas of differentiation arise in the level of granularity organizations are able to select for the protection scheme. Some solutions only offer replication, some only erasure coding, and even if a solution offers both, administrators may only be able to select one scheme per pool. A couple of solutions, however, allow granular protection control, down to control over specific objects. DDN s WOS, for example, allows protection to be defined as granularly as on a per-object basis. Caringo s Swarm allows for erasure coding and replication schemes to reside on the same node. Greater granularity of control translates to greater flexibility and value for an organization. In addition to granular protection capability, some object storage solutions also provide the ability to evolve protection schemes over time for example, providing the ability to protect against four simultaneous failures at an object s creation and then reducing the level of protection over time. Caringo, NetApp, and Quantum are a few of the solution providers that advertise the ability to evolve protection schemes over time. Geo-dispersion and multi-site protection: Replication, asynchronous or synchronous, has been a common feature in traditional storage systems for a while. While some applications may be able to get by relying on backup alone for protection, for massive capacity environments, backup may not be an option as the time to back up may exceed even the widest of acceptable windows. As discussed earlier, the chief concern as capacities increase is managing the multitudes of replication policies that may be required across an environment. Multiple object storage solutions provide automatic multi-site protection via either a distributed erasure coding scheme or a replication capability. It should go without saying that if your organization supports multiple sites, or requires multi-site protection, you should ensure that any object storage solution considered supports automatic multi-site protection. However, object storage solutions can offer multi-site protection in different ways. Whether the base protection scheme leverages a replica-based or an erasure-code-based model, a multi-site configuration may simply extend the base protection scheme to include nodes on other sites in addition to nodes on the primary site, or may apply a separate layer of replication on top of the base protection scheme. For example, if the system uses erasure coding to provide data protection, the architecture may span the erasure-coded objects across multiple nodes that reside across physically separate sites or the solution may simply replicate an erasure-coded portion of data to a secondary or tertiary site. Additionally, if the solution leverages replicas as the primary protection methodology, the system may simply always ensure additional replicas are placed on nodes in physically separate sites.

13 Market Landscape Report: Object Storage 13 For example, Amplidata offers GeoSpread for multi-site protection, which spreads erasure codes for protection across multiple sites with the potential to save over 60% of raw storage capacity required. While some schemes may require more raw capacity than others, the performance impact of multi-site protection is an important consideration. If a system uses erasure coding and the codes are spread across multiple sites in a manner that requires reads to occur on the remote site to read the object, there is likely a substantial read performance penalty. Multiple erasure-coded solutions ensure that enough codes always reside on the primary site to protect against these performance penalties. Some solutions such as Cloudian, DDN, and Scality ensure that reads and writes are performed locally, then that the data is replicated across multiple sites. Cleversafe, on the other hand, uses a combination of its SmartRead technology, where reads are always performed from the fastest responding node, and WAN optimization techniques, such as leveraging parallel concurrent connections, to improve performance. The configuration that provides the best results will likely depend on the specific environment. Additionally, automatic site protection can be very useful in content distribution or collaboration use cases, as content is automatically distributed to remote sites. While multiple object storage providers offer some level of geo-distribution capability, solutions like Scality s RING, which provides one-to-one, one-to-many, many-to-one, and mesh (many to many) replication options, can assist in content distribution or collaboration use cases. Multi-generation architecture: A side effect of massive scale environments is that they are very difficult to migrate. Large-scale content stores and the underlying object storage infrastructure can t exist in the continuous three-year upgrade cycles of traditional storage infrastructure, and, as such, object storage systems are designed with the ability to support multiple generations. The ability to incorporate multiple hardware generations into a signal pool enables object storage to take advantage of new hardware technologies, while eliminating the need for big data migrations. As nodes need to be retired, migration is done only for the individual node on a rolling upgrade basis. Because of the massively parallel nature of the system, the retirement process is relatively quick and easy compared with traditional SAN and NAS migrations. When a node is retired, the data from that node is simply copied to free space throughout the cluster, leveraging the ability for multiple drives to write data in parallel. There are a couple of different ways that object storage solutions deliver multi-generational support. One way is to offer multiple appliance options, continually developing and offering new appliance solutions as hardware technology evolves. Each generation of the appliances is able to mix and match with previous generations in a single pool. This option can provide a simple deployment model, but is ultimately limiting in terms of the hardware options that are available. In some cases, appliance models may reach the end of their supported life, requiring at least a partial migration. While this method provides multi-generational support, it is less flexible than the second method, which is to simply provide the object storage technology as software and allow that software to support virtually any hardware option. This software approach provides superior flexibility and choice in hardware infrastructure along with some added benefits of software-defined storage. Software-defined versus appliance: Multiple solutions across the storage industry leverage the softwaredefined storage nomenclature, each with different definitions. But one segment of software-defined storage technology provides deployment flexibility by providing storage functionality in this case, object storage functionality as a software-only package. Solutions in this segment can be deployed on commodity server hardware, providing flexibility in hardware selection that offers a number of benefits for IT organizations. By providing the ability to select hardware, IT organizations can directly lower either their capital or operational expenses. In some cases, an organization may choose to select lower-cost commodity hardware, while in other cases, organizations may choose to leverage server hardware from a familiar manufacturer in order to reduce the management complexity and associated costs. Also, while traditional storage systems recently have been leveraging more commodity components, server systems often offer a faster hardware refresh cycle than storage systems. In some cases, new technology is available as quickly as every 12 or 18 months as opposed to every three to four years for storage systems.