Storm Clouds? Cloud Computing in a Regulated Environment

Storm Clouds? Cloud Computing in a Regulated Environment Robert H. Smith Computer Systems Quality and Compliance discusses the quality and compliance aspects of computer systems and aims to be useful to practitioners in these areas. Reader comments, questions, and suggestions are needed to help us fulfill our objective for this column. Please send your comments and suggestions to column coordinator Barbara Nollau at barbara.nollau@av.abbott.com or journal managing editor Susan Haigney at shaigney@advanstar.com. RUPERT KING/GETTY IMAGES KEY POINTS The following key points are discussed: Cloud computing is an approach to computer services with significant potential advantages such as on-demand self-service, resource pooling, rapid elasticity, and utility billing. It is truly a paradigm shift in computer services. Cloud computing services may become available through several models such as public services, private services, contract services, and hybrid arrangements. Cloud computing is new and evolving. Its advantages may be prone to hype and its unknowns and problems overlooked. Regulated industries have unique requirements that Cloud computing must address. Access, security, backup, validation, and audits are critical areas for technical regulated industries. Organizations must carefully develop a Cloud strategy before committing to this paradigm shift. INTRODUCTION It is hard to look at anything computer related today and not see bold headlines proclaiming the power of Cloud computing. One sees all kinds of messages that suggest we should be moving to the Cloud. Journey to the Cloud, Why you should have a Cloud computing strategy, and Majority of CTOs have a Cloud strategy are but a few recent banners. But, in an US Food and Drug Administration- Autumn 2011 Volume 15 Number 4 71

regulated environment, should we be moving to the Cloud? What actually is Cloud computing? Is it easy? Should it be? What is FDA s opinion? What would a journey to the Cloud look like in a regulated company? CLOUD COMPUTING DEFINED Precisely defined, Cloud computing possesses all of the following elements: On-demand self service fully automated acquisition and productivity Resource pooling share computing resources Rapid elasticity ability to add or remove resources seamlessly Utility billing pay-per-use or metered consumption. On-Demand Self Service The central idea here is that if a user wants a computing environment, they can get one and be quickly, if not immediately, productive. For example, a project manager on a compliance project decides that their team needs a collaboration tool (like a Wiki) or an issue-tracking tool. It s on its way with a few clicks. You can imagine this would be similar to the shopping cart on a popular website for an electronic application a few clicks and your stuff is on its electronic way and ready to be used. Resource Pooling Today s computers are massively powerful and typically under-used. Resource pooling builds on the idea that powerful physical computers are under utilized, enabling them to run hundreds of virtual computers. One set of physical hardware is used to run many virtual computers. This is called multi-tenancy one computer with many distinct compute workloads running. You can think of this like adding an operation to a station on the assembly line. If a station on the line completes an operation and then waits three minutes for the next tray, then you can add three more minutes of work to that station without impacting performance that station now performs two operations, but at no additional cost. Rapid Elasticity If you need more computing resources, you get it. If the application needs additional CPU (central processing unit) power, additional memory, or additional storage, the Cloud environment can provide it. This can often take place without the user needing to do anything. The application and storage can move in the Cloud to a new location where the resources are available. Conversely, computing resources can be turned in when not needed. So to sum it up, Cloud computing offers users a way to add or release computing resources based on need. One can think of this as a perfectly elastic labor pool so each morning one could walk out front of their plant and get exactly the number of resources one needs, even if that was triple or half of yesterday. Utility Billing The electricity service at our homes is an excellent example of a model that we understand that applies to Cloud billing. There is usually some base fee or minimum to use the service, then after that you can use all you want, you just have to pay for it. The one main difference is that for electricity and water we often pay more per unit the more we use and in Cloud computing one will often pay less for additional compute units (usually things like the amount of storage or number of bytes transferred). Generally, the more one uses the more one pays in total, and the less one uses the less one pays in total. CLOUD COMPUTING MODELS Before we explore these points from a regulated company s point of view, let s look at how Cloud services can be made available. There are four basic models, as follows: Public Cloud. In the public Cloud, services are available to anyone one who wants them and can pay for them. A growing number of organizations offer these services. Private Cloud. In this model, an internal IT organization builds and manages the infrastructure. The internal organization builds 72 Journal of GXP Compliance

Robert H. Smith and creates an internal version of the public Cloud. Out-Sourced Private Cloud. In this model, the organization contracts a third-party organization that specializes in providing the Private Cloud capability. Hybrid Cloud. In this model, two of the models above are combined. Done correctly, Cloud computing is cost effective. The cost savings come on many fronts including staff, hardware, software, supporting infrastructure, and efficient utilization of those elements by sharing them across a pool of users. For common application types, one can have a highly functioning environment in less than 30 minutes. In many organizations, a similar environment could take weeks to months to procure and the cost could easily be ten-fold. The compelling case for use results from this agility and low cost. Also, to many technical pundits, Cloud computing represents a change in the computing environment. Unisys recently gave a webcast presentation where they gave the analogy that in the 1700s, a mill needed to be located near a river or stream so that it could use the flowing water to generate mechanical power. This defined early logging and mineral operations well into the 1900s. Steam engines and other advances improved the geographical options, but the model was still one of selfgenerated power. In the 1900s, the rapid growth and acceptance of off-premises generated energy dominated the landscape. Today, organizations maintain only small emergency power generating capabilities, if any at all. Unisys and others predict that a similar transformation is rapidly occurring in the computing space. The vision for the future is that much of the computing we rely on today will be done off-premises by Cloud providers. Cloud computing is certainly a compelling and interesting paradigm change. The simple fact that a user can get a powerful and capable computing platform or application in a few minutes, cheaply, and pay a low fee, makes the timeline for this paradigm to take root a question of when, not if. A Word of Caution Readers must be cautious. It is early in the Cloud marketplace and there are healthy amounts of hype. Many vendors are just putting Cloud in front of what they were already doing, or they are rushing immature offerings into the market. Buyer beware is prudent advice today. Forrester Research calls this Cloud Washing (1). There is a similar paradigm, but very different offering called software-as-a-service (SaaS). SaaS may or may not be implemented using true Cloud technology. It is thus important to peel off the marketing hype. A SaaS company may host all the computing power it needs in a traditional data center, have perfect controls, and a solid contract that could make it an excellent candidate for a regulated company. At the same time, the same software hosted in a public cloud would not be a good candidate for a regulated company. SaaS is a similar idea you can buy software on a service basis as you need it and pay for what you use. But, SaaS can be implemented in many ways. It is essential to understand the vendor s offering in the context of the other points this column discusses. In environments regulated by such agencies or rules as the Credit Card Act, FDA, The Health Insurance Portability and Accountability Act (HIPAA) and Sarbanes Oxley Act of 2002 (SOX), not all the implications of Cloud computing are well understood or even defined. Regulators may or may not align with some of the basic premises of Cloud computing. Cloud computing is a promising option because many vendors offer basic applications and computing platforms. Many are easy to use and compelling; however, in the GXP landscape there is more to consider. Promise is not enough. For example, Amazon s S3 (probably the most popular Cloud service today Simple Storage Service) can be mis-configured and inadvertently expose data. There are widespread reports that tools are being developed to exploit these mis-configurations to gain access to data. For most organizations, GXP or not, that kind of business risk would be unacceptable. Autumn 2011 Volume 15 Number 4 73

THE REGULATED ENVIRONMENT If one takes the time to read FDA warning letters, it is clear that for both small and large organizations it is easy to stray outside the intent of the regulations and good practices. We do not have guidance from FDA on Cloud computing; however, some GXP pundits and even FDA representatives have expressed concern. The concern of governing bodies with the Cloud concepts makes sense. For example, governing bodies want access to supporting electronic data and systems. What if the data are located in a country where it does not have jurisdiction? To have jurisdiction, the data must reside where FDA, through the US Marshal Service, has jurisdiction. This is a complex matter. The idea is that this raises a complex issue that some part or the data in context may reside in a place where there is no direct jurisdiction. From a practical matter, FDA has the power to take the product off the market, but that may not be enough if there is a concern about patient safety or some level of misconduct. Related to this, lets say FDA would like to inspect back-up tapes. That is a reasonable request if data integrity is in question for any reason. But let s say that Company A s data is at a Cloud provider. Because the Cloud is multi-tenant, those back-up tapes may contain data from 5 or 50 other companies. If the agency is investigating data integrity, it probably is not interested in a lot of pre-processing before they get the data. So, what are they going to get? They are probably going to get all the data. If you have nothing at all to do with Company A, do you want FDA s investigation to include your data? It is possible that user data are segregated. That can solve part, but not the entire problem. FDA is in the process of becoming more thorough and has started to look at audit and event logs. The review of the data in context requires access to application, system, and potentially other tools that are not likely to be segregated. They also may be missing other key information or not retained for sufficiently long periods. Next is the question of validation for intended use. At some level, in a Cloud multi-tenant offering, all companies would be using the same features and functions in the same way. But it is likely that at the detailed level, the configurations, settings, and options will be different or the user may require different interfaces to other systems. Who controls the validated state? Who is the subject matter expert during an inspection? Who defines the retention and archive policies? These questions have to be answered and understood within the constraints of the regulations and each company s tolerance for risk. In the regulated landscape, we can t assume, and we certainly have learned we can t rely on vendor claims or marketing hype. Related to understanding risk, the legal landscape is evolving. There are many complex, and at times conflicting, requirements that companies face. In a Cloud environment the location of data and processing are mobile they can move from system to system, and geography to geography. The mobility of data and computing can put companies at risk or clearly place them in violation of 21 CFR Part 11 and other regulations. Data controls that do not violate one region s laws can violate another s. An analog industry, the Payment Card Industry (PCI), is consistently improving its security posture for both brand protection and economic reasons. We can compare the pharmaceutical industry to the PCI industry, as both have self-governance and government regulation similar to GXP companies. Even though it is constantly improving, the PCI Security Standards Council is warning merchants about the complexities of protecting credit card data running in virtualized systems and cautioning that some configurations may make it nearly impossible for organizations to achieve compliance. Over time, these issues will be addressed, but it provides a reference example supporting the complexities of meeting regulations using this emerging paradigm. The PCI Security Council warned organizations against mixing virtual machines of different security levels to protect credit cards. They suggested that isolating systems containing cardholder data might be impossible if the in-scope and out-ofscope software components are hosted on the same hypervisor (i.e., the software layer that allows many 74 Journal of GXP Compliance

Robert H. Smith virtualized systems to run on one physical system). The PCI guidance prohibits different security levels from co-existing on the same server. While one can argue this point technically, it still shows how industry guidance can conflict with Cloud offerings. Companies planning for the use of these technologies need to understand how regulators interpret the technology. JOURNEY TO THE CLOUD The Cloud has great promise. The following are some things to consider before you start the journey. First, according to Unisys and Forrester, most organizations take three to five years to make the transition (2). That timeline is for non-regulated companies. A regulated company may take longer. In the GXP arena, companies need to consider quite carefully the lack of clear guidance and be aware that some regulators have expressed reservations. These reservations have sound rationale based on related findings in non-cloud environments that may be exacerbated by some vectors of Cloud offerings (e.g., resource pooling and rapid elasticity). These need to be well understood before using Cloud offerings. A plan or strategy to take advantage of the benefit of Cloud should include the following: Server consolidation. Companies should have a plan to move from physical devices to internal virtualized computing. This is a first step and can take one to three years. This is also a good time to perform application consolidation. Automated administration and deployment of systems. The focus should be on defining the smallest number of systems and related system policies. Generally, fewer system policies lead to better policies that are applied more consistently. Also keeping the number low helps prevent mix-ups and reduces the opportunity to select improperly. Development of standard IT policies (that address governing body requirements) will be done during this phase. Build an internal Private Cloud operated by your IT department. This is important to ensure that core assumptions have been tested and processes are documented. Smaller companies should get thorough outside review and audit of the Private Cloud. Larger organizations that have internal audit departments should likewise practice inspections and ensure that all relevant data and evidence are available, sufficient, and robust to demonstrate adherence to GXP regulations and company policy. Specific operational policy adjustment and supporting document adjustments are almost certainly required. Personnel skills will need augmenting. This can include retraining, new additions, or outside assistance. This is an essential part of the process. Cloud computing is a shift in the paradigm and will require a change management program for the IT team. It also introduces new roles related to architecture and security. Define your business goals and optimize accordingly. Different Cloud offerings are targeted at different users some technical and some non-technical. Vendors should be reviewed carefully. The contract and legal team must be involved. The regulatory implications of Public Cloud computing may be insurmountable in the short term. Private Cloud providers may be able to meet the requirements your company has, but the contract and data protection elements must be well thought out and tested. If the agreement ends, how do you get the data back with supporting audit trails? Can you show both in context? Will you be able to do so in 5, 10, 50, or 100 years? What is the disaster recovery plan? How is disaster recovery practiced? Does the disaster recovery plan make assumptions that align with your company s risk model? Does the Cloud provider have a law enforcement notification policy? Do they have a customer notification policy in the event of a security event? What are the controls and guarantees? Make sure to verify that legal provisions can be translated into technical milestones. For example, can you get a backup tape back on your site and the data extracted in context to give to an Autumn 2011 Volume 15 Number 4 75

agency that requests it? Do they have robust change control and do you have access to the records? Make sure your risk management, compliance, and regulatory affairs teams understand the new risks introduced. For some applications, anything beyond a Private Cloud offering may be too extreme. It is important to not assume that there is something about using Cloud computing that automatically means a more secure and compliance system. An example of how it could be worse is that one mistake on the part of the provider could affect tens or hundreds of systems because of the inherent re-use found in a Cloud offering. FINAL THOUGHTS Secure Cloud computing is evolving. There is a working group, Cloud Security Alliance, doing excellent work in this area and making tremendous progress quickly (https://cloudsecurityalliance. org/). It is interesting to note that their security guidance document is already at version three. It shows their agility and commitment, but also the velocity of change as Cloud computing marches forward (3). Cloud computing and Private Cloud computing offer society tremendous benefits. There are legitimate examples in non-regulated companies of return on investments (ROIs) ranging from 300% and up, making this a attractive way to get cost out of the business and improve service. Public Cloud offerings are probably too risky for regulated applications until the industry matures. Private Cloud strategies offer promise and significant ROIs through better computer resource utilization, lower power consumption, and better staff utilization. However, a prudent plan will be to view this as a new paradigm. The new paradigm requires new roles, new processes, and a refined risk model so that leaders in GXP environments do not find themselves in a bad storm. REFERENCES 1. James Staten, Cloud Is Defined, Now Stop the Cloudwashing, Forrester Blogs, October, 14 2009. 2. John Brand, apping Out The Journey To Private Cloud Enablement, Forrester Research, December 1, 2010. 3. Cloud Security Alliance, Press Release, Cloud Security Alliance unveils 2011 initiatives at CSA Summit at RSA, https://cloudsecurityalliance.org/csa-news/cloud-securityalliance-unveils-2011-initiatives-at-csa-summit-at-rsa/, February 15, 2011. GXP ABOUT THE AUTHOR Robert Smith is Director Technology Services at University California Riverside. Robert has 25 years of software and systems experience including start-up, FDA/GXP regulated, internal use and commercial systems. He holds CISSP and PMP credentials. Robert can be reached at robertsmithxtr@hotmail.com. 76 Journal of GXP Compliance