Scale-out NAS Unifies the Technical Enterprise

Transcription

1 Scale-out NAS Unifies the Technical Enterprise Panasas Inc. White Paper July 2010

2 Executive Summary Tremendous effort has been made by IT organizations, and their providers, to make enterprise storage more efficient and effective. Unfortunately, to date these efforts have not focused on the technical computing side of the organization. For many organizations the technical and engineering work is the center of value creation for the company. These organizations are primarily engaged in research or the development of products or other innovations that drive the success of the organization. This paper details the compelling benefits created by improving the efficiency and performance of the Technical Enterprise with a Scale-out NAS storage infrastructure. Scale-out NAS enhances the traditional file server to provide an unlimited ability to scale capacity and performance across the organization. Unlike traditional IT efforts which largely focus on reducing costs, the primary goal of IT for the Technical Enterprise is to increase the organization s top line driving new research output and developing new products and services that increase revenue. Scale-out NAS helps achieve this by: 1. Improving application performance - accelerating the time it takes to produce results means organizations can perform research, design and analysis faster and better. Improved application performance = increased ROI. 2. Improving collaboration - Today s research and development depends on multiple users and groups working together to achieve specific results, and storage systems must support them to achieve common goals. Providing a single common pool of storage for all users and applications drives a faster workflow and better results. Scale-out NAS also has the potential to improve the bottom line by increasing efficiency in the Technical Enterprise. 1. The limitations of traditional NAS causes users to create copies of data sets for use by different applications to achieve better storage performance, or simply because they ve run out of space. Scale-out NAS eliminates the need to duplicate data caused by traditional storage solutions. 2. The islands of storage created with traditional NAS also increase the amount of storage administration required. Each system needs to have capacity and performance monitored, access managed and users re-balanced. With a Scale-out NAS solution these activities are done once and wasteful administrative overhead is eliminated. What is the Technical Enterprise? Technical organizations are the foundation for modern business, performing the research and developing the technology that underpins the products that make companies successful. High performance computing (HPC) has traditionally been associated with science, engineering, R&D and analytics applications used for simulation or modeling and performed on Linux clusters. As computing power had increased exponentially in recent years, new technical applications have emerged that enable a broader set of users to create the inputs and analyze the outputs of the simulation or model that are required to complete the research or design workflow. Taken together HPC and Technical Applications form the core of the Technical Enterprise (see diagram below). Unlike traditional business applications focused on cost reduction, Technical Applications drive innovation and increase revenue. 2

3 Enterprise Applications From developing new autos, aircraft or semiconductors for manufacturing companies, to finding and extracting hydrocarbons for energy companies, to developing new drugs for the medical industry, all of these businesses have groups of scientists and engineers with specialized software, networking, storage, and compute hardware to make them productive. In all of these disciplines, time to results is the key to better research and faster product development. Some examples in various industries are detailed below. 3

4 One of the key elements differentiating the Technical Enterprise from the rest of corporate IT is the type of data created and used. The Technical Enterprise is built on unstructured data. Corporate IT is focused on structured data which resides in various databases and describes the operations of the modern organization, from accounting to inventory to customer relations. Storage products used in this environment are designed for structured data where servers own a particular data set. Storage Area Networks (SAN) are popular because they maintain the relationship of a server and application owning the data as if it is inside the server. This makes the storage array easier to maintain by virtualizing the disk and allowing the capacity to be easily distributed across servers and applications. Researchers and engineers in the Technical Enterprise live in a world of unstructured data needed to solve complex problems. Some use sensor data to understand a problem, as happens with a genome sequencer in assessing the genomic difference between research subjects. Others create models or simulations to prototype a design, creating a model of an airplane wing instead of spending time and money to actually produce and test many prototypes. Unlike a company s books and records which stay fairly constant, as simulations and sensors get more sensitive and models become more refined, the data that they generate has exploded in volume. The growth of unstructured data in enterprises is rapidly outpacing transactional/structured data. (See chart below.) 2010 Data Growth vs. IT Spending Forecast Source: Data growth (IDC), IT spending (Forrester) 4

5 Technical organizations are often expected to use standard Enterprise IT storage products which are not designed to meet their unique needs. Network-Attached Storage (NAS), particularly NFS file servers, have been adopted as the backbone of technical organizations for the last two decades by providing a pool of storage that can be shared by all of the users and applications. While typical Enterprise NAS servers have supported data sharing, the limitations on the size of traditional file server storage pools have forced IT administrators to create silos within the Technical Enterprise that work to slow results and increase cost. (See diagram below.) Storage Silos Slow Time to Market for Products and Research Storage silos are often driven by different application requirements. For example, in a semiconductor environment the design verification process requires extremely high IO for very small files, while the tape-out or optical proximity correction process requires high concurrency with high bandwidth. A typical Enterprise NAS server cannot support both types of activity, and often there are different storage vendors who specialize in different performance areas. This leads to silos of storage by both vendor and application. Often applications are differentiated by whether they run on a large number of servers in a Linux cluster, described as High Performance Computing (HPC), for use in modeling or simulation. The other category of technical application is for engineers or researchers to create the input for the model or to analyze the results. The table below describes the major difference between the two main categories of applications and their IT implications. 5

6 IOPs There are important implications for choosing storage systems based on specific application requirements. Storage designed and tuned for the high IOPs requirements of Technical Applications may have insufficient bandwidth needed for many HPC applications. In addition, the problem of keeping large data sets on high performance storage can be costly when the data needs to remain available for further analysis or processing. The differences between types of storage become especially important in the realm of Scale-out NAS, where the key value of the storage system is to consolidate data across the Technical Enterprise. An important requirement for Scale-out NAS is the ability to create multiple storage pools with different price/performance capabilities, yet provide a global namespace and single point of administration. Silos of storage build walls between users and applications in the technical environment and make it difficult to share data across disciplines. In Life Science, for instance, most of the difficult problems are cross-discipline where the geneticist must work with the biologist and the chemist to identify a new disease pathway and potential compounds to address it. Workflows involving many applications, users and servers are the norm in technical disciplines, whereas in typical enterprise applications, each discipline may have a single central database that supports their work. In the Technical Enterprise there are usually several separate applications, running on different servers, with the data residing on multiple NFS servers. In a workflow, multiple disciplines bring their expertise to bear at relevant stages on a shared problem. That problem represents a data set shared across the workgroup. If a company is building a new model of jet aircraft, the wing design team and the engine team have different roles and skill sets. Sub-groups may work in CFD and finite element analysis, but they must ultimately integrate their design, testing and manufacturing plans to achieve a successful result. If these groups work on multiple file servers, crossing storage silo boundaries slows the time it takes to move data between workgroups or between projects. In the worst cases it can lead to insights not made, or mistakes caused, by data sets that have aged and are out of sync (sometimes called data drift ). 6

7 Silos of Storage Increase IT Costs A Technical Enterprise built on many silos of separate storage systems will substantially increase IT costs. Some costs are easy to see, some are more subtle and pernicious. Each storage silo increases the support costs associated with maintaining the storage infrastructure. Tasks such as adding storage capacity, backup, and disaster recovery are multiplied with each additional storage silo. Even more expensive are the costs and complexity arising from the need to support multiple file servers. Some examples include mount table management, multiple monitoring tools and user credentials management, which are all driven by the requirement to support multiple servers. Cost and complexity increase geometrically as additional file servers are added. As we saw earlier, many storage systems are quite specialized for application workloads and are procured from different product lines or from different vendors altogether. This works to further increase the administration workload by requiring specialized knowledge and training on different storage systems, each with a separate learning curve and possibly different administrator. A hidden cost often created by storage silo environments is the duplication of the same data on multiple systems. Sometimes duplication is caused by organizational behavior, with each group feeling the need to have the data sets on their own servers. More often duplication is used as a way to increase performance because the primary server is overloaded or has poor performance characteristics for that data set or application. The costs of duplication on primary storage are amplified as the data is replicated for disaster recovery or archived to secondary storage or tape. Worse is the potential for the data to drift over time, getting out of sync between servers, causing errors and mistakes in the project that are difficult to reconcile. Data duplication can also contribute to slowing the progress of the project or workflow. Instead of using the data where it initially resides, the duplication process adds time to copy to a new location, and with large unstructured data sets, may cause delays of hours or days in a project. Scale-out NAS Integrates the Technical Enterprise Scale-out NAS improves on traditional NAS by providing a mechanism to increase the capacity and the performance of the storage network to meet the increased needs of the Technical Enterprise. Scale-out NAS allows technical users to minimize or eliminate silos of storage, unlike the 1st generation Enterprise NAS which created them. The ability to scale capacity from Terabytes to Petabytes is an important benefit to technical users who need to support the massive increase in fidelity of models describing financial risk, hydrocarbon reservoirs or airflow over Formula 1 vehicles. Similarly organizations that use instruments or sensors to support national security, astrophysics or weather/climate data are capturing exponentially more data as their sensors increase in sensitivity. In order to analyze the data or execute the simulation, Moore s law is doubling the capability of the compute infrastructure every 18 months. The performance of the storage systems must scale with the compute servers and workstations in order to take advantage of the additional capability. Scale-out NAS promises to extend storage performance continuously as needed, rather than the traditional requirement of rip and replace when a new, more powerful generation of file servers becomes available. 7

8 Scale-out NAS offers significant benefits to end users as well, by providing a single mount point and global namespace so all data is easily found and shared between users and groups. No longer do users need to go hunting for data that has been moved by an administrator who is adding another silo or balancing user load across file servers. Just as the access to Scale-out NAS is unified for users of the storage system it is also unified for the administrator. This allows a single storage administrator to manage more total capacity - mundane tasks such as capacity management are minimized and mount table administration is completely eliminated. Storage administrators can redirect their efforts to more strategic efforts such as namespace planning and storage architecture, which can help contribute more directly to the goals of the organization. An example of a Scale-out NAS system from Panasas is depicted below. Technical Computing 8

9 Panasas Scale-out NAS Solution Integrates the Technical Enterprise. A great Scale-out NAS infrastructure can drive improvements in both the top and bottom lines. It improves returns by accelerating the research and product development process and lowering investment by eliminating duplication and administrative overhead. Panasas Scale-out NAS storage systems are the only storage systems designed exclusively for the Technical Enterprise, exceeding the performance, reliability and ease of management requirements and scaling from Terabytes to Petabytes in a single system. When reviewing choices for Scale-out NAS products for the Technical Enterprise, the first attribute to assess is performance. Performance is critical for both HPC and the technical applications used by the scientists and engineers. The HPC cluster is often an investment of hundreds of thousands or even millions of dollars, and it can easily be rendered ineffective or inefficient by a poor choice of storage system. If the Cluster must wait on I/O to storage, it results in an unbalanced system that does not deliver full return on the large investment made. It becomes an expensive room heater rather than a compute facility. In addition, great performance is necessary to deliver peak efficiency to users applications. Speeding the product design or research through the workflow is the key for technical organizations to improve time to market and increase revenue. Panasas offers a complete range of storage systems that can be combined into a single solution to meet any need in the Technical Enterprise. (See diagram below.) 9

10 In order to consolidate Technical storage into a single system, it is critical that storage be tailored to meet the price and performance requirements for each area and/or stage in the workflow. To support the Cluster for modeling, simulation, or analysis applications, it is necessary for storage to support many concurrent servers and high bandwidth access to data. Compromise in this area causes I/O bottlenecks and slow results. Panasas offers the PAS 8 which delivers up to 6GB/s per rack of storage at a cost effective price point. Similarly, in order to support end-users working with design applications as input to the cluster analysis or visualization of the resulting data, the system must support very scalable access with NFS for Linux and Unix users and CIFS for Windows users. The storage system must be able to deliver high levels of I/O to storage to be responsive to users and accelerate the workflow to a successful conclusion. To deliver very high performance NFS, Panasas offers the PAS 9. The vast amounts of data accumulated in the Technical Enterprise must be available to be leveraged when needed. Sending the data of prior experiments or designs to tape makes them inaccessible and useless when the team needs them to derive insight for the current problem. To store all of the data cost effectively, Panasas offers the PAS HC which can store a petabyte of data in a single rack and up to 5 GB/s of throughput when data needs to be accessed or retrieved. The PanFS storage operating system enables Panasas to bring together multiple types of storage with different price and performance attributes into a single system. The system is easy to deploy in under 30 minutes, and simple to scale. The different types of PAS storage are separate pools that can be easily combined into a single system with a single mount point tailored to the needs of each type of user or application in the Technical Enterprise. Performance is isolated between the storage pools so that a big run on the cluster has no impact on the performance of the end user s storage. Administrators have a single point of access to manage the entire storage system, vastly simplifying the task of monitoring, administering and troubleshooting the system versus trying to keep track of multiple traditional file servers. Storage is thin provisioned with quotas that are managed across all storage pools at either the volume, user or group level so capacity can be provisioned as needed and allocated as required. Snapshots are provided to allow easy retrieval of deleted data or on-line backups. Snapshots can also be used to quickly replicate data to a mirror system, transporting just the changed byte-level data for disaster recover purposes. Users also benefit from Panasas Scale-out NAS with a single mount point, single sign on and the ability to use any protocol to see any data in the system for which they have the appropriate permissions. One major East Coast University found that by creating a single Scalable NAS system instead of the traditional Filer per Department model, they were able to achieve both improved performance and lower cost. As the Director of IT stated: With a unified Panasas, we gained an integrated solution, which significantly reduced our cost of ownership and avoided the support headaches of having different storage systems in each research group. Having a single solution for providing network-attached storage to our many different research 10

11 schools saves us time and money while providing a scalable way to grow without reaching thresholds that could limit our scientific research. Our researchers are some of the best in their respective fields. It is critical that we keep their focus on research giving them as much resource as their grant money will buy by being as efficient as possible within the IT organization. To learn how Panasas can accelerate your Technical Enterprise, contact us at. About Panasas Panasas, Inc., the leader in high-performance scale-out NAS storage solutions, enables enterprise customers to rapidly solve complex computing problems, speed innovation and bring new products to market faster. All Panasas solutions leverage the patented PanFS storage operating system to deliver exceptional performance, scalability and manageability. Phone: PANASAS 2010 Panasas Incorporated. All rights reserved. Panasas is a trademark of Panasas, Inc. in the United States and other countries. 11 PW