EXECUTIVE GUIDE WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM

Transcription

1 EXECUTIVE GUIDE WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM

2 TABLE OF CONTENTS Why your modern data center needs an operating system What is a data center operating system and how does it work? The data center as a computer Data center abstraction and benefits for different teams from using a data center operating system Solving problems instead of managing them: The advantages of the data center operating system Conclusion: A flexible data center OS for the age of distributed applications

3 WHY YOUR MODERN DATA CENTER NEEDS AN OPERATING SYSTEM

4 For many organizations, the data center has evolved from a collection of siloed machines to a massive pool of resources supporting various compute and storage needs. Consider the rise of organizations that use hyperscale data centers, such as Amazon and Google, over the past decade plus 2. To support around-the-clock operations, these companies invested heavily in new infrastructure design and software implementations that, between them, have enabled profound leaps in agility and adaptability. If these organizations had stuck to the traditional approach to data center infrastructure, they would be overseeing potentially thousands of servers, as individual entities, managing them in isolation. Instead, they worked toward a hyper-converged commodity hardware infrastructure to obtain flexibility and agility above and beyond above and beyond what could be obtained with traditional siloed infrastructure. IF THESE ORGANIZATIONS HAD STUCK TO THE TRADITIONAL APPROACH TO DATA CENTER INFRASTRUCTURE, THEY WOULD BE OVERSEEING POTENTIALLY THOUSANDS OF SERVERS AS INDIVIDUAL ENTITIES, MANAGING THEM IN ISOLATION WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 3

5 The demands of both accounting for a diverse user base and meeting the the flexibility and agility needs have required an operating system that abstracts the underlying resources and makes them into something easier for for infrastructure specialists and other technical users alike to use. By breaking down software silos and enabling a pool of data center resources that would perform like PC components governed by an OS, companies can lower their total cost of ownership, operate easily, and provide greater flexibility for all teams than isolated data center infrastructure would ever allow. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 4

6 WHAT IS A DATA CENTER OPERATING SYSTEM AND HOW DOES IT WORK?

7 A data center operating system is a platform that spans all machines in a data center and aggregates them into a single pool of resources. In effect, it replaces the machine-by-machine abstractions traditionally in place in data centers, so that these machines can be utilized as needed to support many different changing workloads, rather than tied to particular applications. A data center OS provides the process isolation and resource management capabilities that are already foundational to server OSes but the data center OS now provides the OS functionality at the cluster level. The technical, time-consuming processes of updating each machine s installation one by one is replaced by centralized controls. The result is data center infrastructure that is highly flexible, programmable and easily usable by all teams within an organization. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 6

8 THE DATA CENTER AS A COMPUTER CONTACT US

9 Today s data centers run applications that are essentially distributed systems. They run the gamut from distributed processing algorithms like Apache Hadoop, to consumer-facing services such as Netflix on virtual machines. In addition, there are various DevOps technologies such as Docker Swarm, Kubernetes and Mesos that are now being utilized within the Data Center. Just like PCs, facilities have moved beyond the handful of expertcreated applications that dominated early data center operations, and with this shift comes the need for a new type of abstraction, beyond than the traditional one that was based on individual machines 1. Moving toward data center abstraction, with pooling rather than isolated resources, ensures that infrastructure is more accessible by all teams. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 8

10 More specifically, consider what it would be like to run applications on any modern PC in the same way they are handled in most data centers today. CPU, cache and memory modules would have to be assessed and specified for each program, since the OS would not provide the necessary abstraction. Specific hardware components would be tied to particular applications. This approach would be inefficient and cumbersome even on a single laptop or desktop, much less in a data center supporting Web-scale services. Instead of having the freedom to intelligently distribute workloads across all available system resources, users would have to focus on individual machines during design, deployment and maintenance. INSTEAD OF HAVING THE FREEDOM TO INTELLIGENTLY DISTRIBUTE WORKLOADS ACROSS ALL AVAILABLE SYSTEM RESOURCES, USERS WOULD HAVE TO FOCUS ON INDIVIDUAL MACHINES WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 9

11 The data center OS makes it easier to go from bare metal servers to the required service without bottlenecks getting in the way. Ideally, it would start from the boot of cold bare-metal servers and include boot code, the host operating system, resource management, and service management. It ensures that there is no management of individual servers (e.g., software installation on each machine) and that teams can avoid the divergence of patches and system versions across machines in the cluster. It also complements the presence of hyper-converged commodity infrastructure by further simplifying how resources are accessed, ultimately eliminating previous hardware and software silos. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 10

12 DATA CENTER ABSTRACTION AND BENEFITS FOR DIFFERENT TEAMS FROM USING A DATA CENTER OPERATING SYSTEM

13 Reducing the longstanding reliance on fragmented, isolated hardware as well as machine-based abstraction is essential to building data center infrastructure capable of supporting Web-scale services. Moreover, data center abstraction allows organizations to move away from manual resource management and instead treat the entire data center as a unified, cohesive utility. This shift in abstraction layers from machines to the data center at large, via the data center OS running on hyperconverged commodity infrastructure makes the data center much more accessible to non-infrastructure teams. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 12

14 DevOps DevOps is the movement that emphasizes collaboration and cross-training to bridge the divide between the traditional dev/ test and operations silos. It is designed for environments in which team members do not have to fixate on hardware concerns. With a data center OS effectively transforming individual servers into a single pool of resources, DevOps teams can focus more on accelerating development and less on wrangling with infrastructure and outdated abstraction models. DEVOPS TEAMS CAN FOCUS MORE ON ACCELERATING DEVELOPMENT AND LESS ON WRANGLING WITH INFRASTRUCTURE WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 13

15 Also, with rapid innovation in DevOps technologies and the sudden proliferation of container technologies such a Kubernetes, Docker Swarm and Mesos is a great example DevOps teams need a unified platform on which these modern technologies can be easily and quickly deployed without going back and re-engineering the underlying infrastructure. Such kind of future proofing is important because there might be a new technology they might need to deploy in six months that they aren t even aware today. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 14

16 Data science Data science teams also benefit from not having to concern themselves as much with infrastructure. Data scientists are typically statistical analysts first and foremost, rather than infrastructure or network engineers, but the analytics applications that they oversee still run at great scale and are very resourceintensive. Even the Data Engineers benefit from not having to concern themselves with infrastructure and rather focus their time and expertise on running Big Data platforms such as Hadoop and Spark and using components such as MongoDB and Cassandra. Both of them are looking for an easy way to quickly deploy the big data technologies on bare metal infrastructure. DATA ENGINEERS BENEFIT FROM NOT HAVING TO CONCERN THEMSELVES WITH INFRASTRUCTURE AND RATHER FOCUS THEIR TIME EXPERTISE ON RUNNING BIG DATA PLATFORMS WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 15

17 The data center OS provides the software infrastructure for streamlining data science, simplifying how resources are accessed, shared and distributed. Some platforms such as Hadoop have attempted to address this issue, but resource management is handled at a high level, which adds greater complexity to the matter 1. It is best to have an operating system that takes a holistic approach and starts from the boot process of the cluster and ends with the deployment of the Big Data platforms. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 16

18 Resource management and monitoring Resource management and monitoring, along with process isolation, in data centers have long struggled against the tide of IT siloing its infrastructure and projects for use only by specific teams. The usage of virtual machines as well as recent innovations such as containers have only exacerbated this tendency by encouraging the proliferation of separate stacks. Silos also set up difficulties down the road by making it hard to reallocate or repurpose hardware as workloads inevitably change. SILOS ALSO SET UP DIFFICULTIES DOWN THE ROAD BY MAKING IT HARD TO REALLOCATE OR REPURPOSE HARDWARE WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 17

19 With a data center operating system, organizations can overcome such limitations of the siloed data center and more effectively run their key services at scale, with straightforward access to a pool of resources throughout the data center. Regardless of what equipment vendor IT has chosen, data center abstraction allows for hardware-agnostic performance across infrastructure for optimal density and efficiency. Linear scalability also provides flexibility whether a service requires just a few machines or many, saving teams time and money as they adjust infrastructure for their own applications and needs. The diverse composition of teams, ranging from programmers to data scientists and DevOps personnel, becomes an asset for solving problems rather than a bottleneck related to rigid infrastructure. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 18

20 Making the data center into an aggregated pool of resources rather than a bunch of islands ultimately allows for better centralization and control of services, while reducing overhead. The implementation of a data center OS takes this treatment of the data center as a giant pool of resources to the next level, by enabling the same level of control over resources and processes that has been pivotal to the utility of traditional computers. Shared environments can support users executing many different applications, with proper isolation. Plus, infrastructure can be made readily accessible to many different user populations such as data scientists, not just infrastructure specialists. MAKING THE DATA CENTER INTO AN AGGREGATED POOL OF RESOURCES RATHER THAN A BUNCH OF ISLANDS ULTIMATELY ALLOWS FOR BETTER CENTRALIZATION AND CONTROL WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 19

21 SOLVING PROBLEMS INSTEAD OF MANAGING THEM: THE ADVANTAGES OF THE DATA CENTER OPERATING SYSTEM

22 The transformation of the data center into a Web-scale computer, replete with an OS capable of governing a pool of resources across a wide range of infrastructure, can seem daunting. Many current solutions in this space merely treat the symptoms of what holds the data center back e.g., managing individual servers instead of addressing the cause, namely the lack of an underlying OS. As long as teams have to deal with managing host OSes on specific machines, many costly and time-consuming issues will continue to emerge. A myriad of patches, system versions, etc. will require attention and encourage unneeded divergence across the entirety of the data center infrastructure. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 21

23 A modern data center OS goes beyond placing a band-aid on these problems: It makes it possible to cure them entirely. Managing individual machine installs becomes a thing of the past and teams can instead work with a turnkey solution that is much more straightforward to manage than traditional onpremises architectures. Nothing has to be installed and managed on the hard disk on bare metal when new capacity is required, saving time and simplifying day-to-day operations compared to having to keep up with individual installs within traditional data center infrastructure. A MODERN DATA CENTER OS GOES BEYOND PLACING A BAND-AID ON THESE PROBLEMS: IT MAKES IT POSSIBLE TO CURE THEM ENTIRELY. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 22

24 The data center OS is at the heart of this new paradigm for data center design and functionality. What does the OS provide that is so essential? It ensures proper resource and data sharing, as well as comprehensive monitoring and debugging. Perhaps most importantly, the data center OS creates, in effect, a broad data center abstraction, whereby various services such as Hadoop, Spark, OpenStack, Mesos, Docker Swarm, Kubernetes etc can be easily deployed and managed. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 23

25 Examples from organizations with hyperscale data centers For context, cluster applications have progressed rapidly in both their sophistication and number of users, in turn requiring a new approach to infrastructure. Consider the aforementioned advances made by Google et al in the utilization of cluster programming frameworks. CLUSTER APPLICATIONS HAVE PROGRESSED RAPIDLY IN BOTH THEIR SOPHISTICATION AND NUMBER OF USERS WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 24

26 Google s software system, Borg, controls the distribution of work across vast numbers of servers, saving Google the expense of constructing at least an entire additional data center 3. Its power derives from its ability to handle many different types of tasks on the same infrastructure instead of using a specific set of servers for each system. Wherever currently free capacity exists, those resources can be tapped for storage or processing power. Twitter has done something similar with its usage of Mesos, which, like Borg, enables Twitter to use the data center as if it were a single machine rather than sprawling facility filled with equipment. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 25

27 The data center OS provides similar benefits to anyone no matter what the scale. Moreover, a data center OS allows for a large degree of flexibility in configuring frameworks and technologies to enable cross-application data sharing. The result is the elimination of silos and the implementation of elastic, scalable computing resources readily available through APIs and self-service interfaces. A DATA CENTER OS ALLOWS FOR A LARGE DEGREE OF FLEXIBILITY IN CONFIGURING FRAMEWORKS AND TECHNOLOGIES WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 26

28 Hyper-converged commodity infrastructure and the data center operating system In eliminating silos, the first step is often to turn to hyperconverged commodity infrastructure as a replacement for islands of discrete propriety appliances. This move can reduce costs while simplifying hardware management and eliminate capacity planning. Still, this step does not solve problems such as constantly having to ensure that all server OSes are up to date. THE FIRST STEP IS OFTEN TO TURN TO HYPER-CONVERGED COMMODITY INFRASTRUCTURE AS A REPLACEMENT FOR ISLANDS OF DISCRETE PROPRIETY APPLIANCES WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 27

29 There needs to be a shift from this machine-by-machine approach to leveraging the entire data center as a single resource, through data center abstraction. Microsoft Excel sheets and scripts have been the glue holding together server management and automation at many data centers, but something more comprehensive and sustainable is required. The data center OS enables a fresh approach to resource management and monitoring, one that is in line with what is expected from data centers now that they have become the backbones of massive distributed application and services. Making the data center accessible to all teams and having it behave predictably and in unison are table stakes for running modern infrastructure. WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 28

30 CONCLUSION: A FLEXIBLE DATA CENTER OS FOR THE AGE OF DISTRIBUTED APPLICATIONS

31 Overall, a data center operating system delivers the essential benefits of managing clusters of servers from an OS-centric data center at a time when organizations must develop, test, debug and deploy a many distributed applications across their growing data center footprints. Organizations such as Google, Twitter and Facebook have exemplified this new approach of turning the data center into a gigantic computer that treats all infrastructure as resources for performing essential tasks. A DATA CENTER OPERATING SYSTEM DELIVERS THE ESSENTIAL BENEFITS OF MANAGING CLUSTERS OF SERVERS FROM AN OS-CENTRIC DATA CENTER WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 30

32 Traditional approaches to data center resource sharing and management leave much to be desired in terms of TCO and scalability for today s applications. DevOps, data science initiatives and smart utilization of resources requires the elimination of hardware and software silos so that infrastructure is addressable not just by infrastructure specialist but also by DevOps personnel, data scientists and other teams. Using a data center OS ultimately leads to better abstraction of resources, treating them as parts of the same whole rather than as discrete machines. The result is this greater accessibility for non-infrastructure personnel. USING A DATA CENTER OS ULTIMATELY LEADS TO BETTER ABSTRACTION OF RESOURCES WHY THE MODERN DATA CENTER NEEDS AN OPERATING SYSTEM EXECUTIVE GUIDE 31

33 Piston CloudOS is a new type of operating system that manages clusters of commodity servers in your datacenter as a single pool of highly-available resources. These resources may be shared by any service or framework, including OpenStack, Hadoop, Spark, Kubernetes, Docker Swarm, or Mesos for better resource efficiency, and streamlined operations. Perfected through years of OpenStack deployments at hundreds of companies, Piston CloudOS makes bare metal orchestration and cluster management an afterthought, empowering IT to deploy new services in minutes, reduce operational and management costs by 85%, and provide visibility into critical resources. To learn more about CloudOS, please visit: cloudos-compendium/ 126 Post Street, 5th Floor San Francisco, CA (800) info@pistoncloud.com