White paper. Real-world solutions for growth, scalability, management and disaster recovery. By Kyle Ohme IT Director Freeze.com

Transcription

1 White paper Real-world solutions for growth, scalability, management and disaster recovery By Kyle Ohme IT Director Freeze.com February 2008

2 Table of Contents Executive summary Scaling on demand Server management High availability and storage considerations Backup and disaster recovery High availability between two physical locations QA testing and patch management Data recovery Solution integration Other considerations Conclusion Kyle Ohme

3 Executive summary In the past, the Information Technology (IT) department was viewed as a liability, a drain on resources whose singular purpose was to deliver uptime to the organization. Today, the IT Department is a key asset, expected not only to meet uptime needs but also assure the business has the computing horsepower and infrastructure flexibility to adjust to constant market shifts. All on a tight budget, of course. Collaborating with key players in the IT channels and utilizing Citrix Provisioning Server for Datacenters software as the underpinning, Freeze.com has developed a highly scalable server provisioning, server-management and disaster- recovery model. The innovative Freeze.com approach allows IT to focus on key areas of growth and assist the business in making decisions, not based merely on budgets, but based on what IT is good at trending. The new model enables the IT Department to be proactive in identifying trouble spots and also be ready for 20x growth in a matter of minutes. Importantly, the time savings realized in the server-provisioning process alone was sufficient to pay for initiative in one year. In terms of productivity, a system administrator can be twice as productive running up to 40 servers. The Freeze.com solution was built for an environment that constantly shifts and needs to maintain agility. It was built from the ground up using best practices from key areas in the IT community. Freeze.com strives to be on the cutting edge of digital marketing in all forms of mass media and with millions of people accessing its network each day, it was an easy fit to use for testing an innovative concept. The solution detailed in this paper offers a true computing-ondemand model for both the small business and large Fortune 500 companies. Using the new model, Freeze.com is able to: provision servers from bare metal, and scale-out servers in a load- balanced solution in a matter of minutes provision a server to work in multiple-server environments and in several configurations by simply rebooting the server and pointing it at a different image to boot from achieve nearly 100% uptime for end users and allow the IT staff to be proactive in both understanding traffic shifts and in building models that will not fail under heavy load simply point and click to boot different operating systems and server configurations, or to bring up another server into a cluster for peak traffic times provide a fully redundant, high-availability configuration, allowing for a boot server to pick up sessions from any blade in the environment if a server should fail handle backups in process across several servers. This ultimately means that with the ability to back up open file systems, it is also possible to back up endless server configurations replicate the storage volume to a second location for a best-ofclass disaster recovery plan roll out mirrored and unique servers quickly. During bug testing, load testing new environments, and testing in-house workloads, it became evident that this solution can solve almost all issues, bridging the gap between test labs and realworld environments perform server restores and leave a central storage solution to handle disk scrubbing and administration distribute server images world wide without the need to be on site. Simply create the image on like systems and transfer the OS images Scaling on demand With the industry becoming more focused on Service Level Agreements (SLAs) and on constant regulation changes, it s vital to build an infrastructure that is able to expand and react as needed. Qualified IT staff is hard to find and it always seems that as more hands touch the equipment, the probability of problems cropping up increases. And providing end users with rock-solid solutions becomes even more challenging as security threats and a demand for more analytical computing power increase. In the past, the only option was to scale hardware and have an abundance of underutilized computing power just in case. Meanwhile, IT departments were struggling to find the right balance in achieving company goals while working within budget constraints. Those days are long gone. With the right solution in place IT can utilize its staff to achieve goals that make the company money. Time is no longer needed to patch Windows updates one by one; build servers; troubleshoot hardware and maintain disaster recovery models that span across several versions of hardware and software. This can all be centralized in one service model. By utilizing Citrix Provisioning Server for Datacenters, IT can now provision servers from bare metal and scale-out servers in a load balanced solution in a matter of minutes. 3

4 The capability of Citrix software in this regard surpasses anything on the market today. It facilitates the streaming of an operating system to a server without the need for a hard drive. Many companies consider computing on demand the ability to virtualize OS images over the top of either large boxes or clusters of computing power. Citrix improves on this process significantly by streaming the sectors of the OS that are required for the server to run to the actual server itself, eliminating the need for multiple layers. This is unheard of in the Windows environment where the best option has been to use products such as VMware or Virtual PC. With the Provisioning Server for Datacenters-driven model, you are easily able to provision a server to work in multiple server environments and several configurations by simply rebooting the server and pointing it at a different image to boot from. Because this is not relying on iscsi or fiber channel to boot, you are not adding an extra layer of management for administration. Two options are available from Citrix to provide an image to a server: private-image mode and standard-image mode. Private-image mode is simply moving the local storage of the server to a back-end database that can be easily managed. With private-image mode all servers have unique information and are configured separately. Think of this model as a typical server setup where each server has a unique personality. Using private-image mode to create a bare metal OS install is easy. Simply copy the pre-built image and within a matter of minutes you can roll out a countless number of images to your servers. When you are done with the servers, you can simply reboot to another image or delete the image used for testing. It is also possible to scale out servers in private-image mode simply by copying the image that you want to scale out and booting that image on a new server. It actually is that simple. Standard-image mode is used to allow one OS to boot from several servers meaning a central golden image is used to provide several servers with an operating system and all share the same content. Standard-image model allows for the administrator to update one image instead of managing each individual server. This can be useful if you have several web farms that all run IIS. The servers are almost identical, so it s easy to see the advantages of maintaining one image that several servers use to boot. Standard-image mode also assists in re-provisioning servers to handle critical load, allowing you to take a server from one server farm and re-provision it in a matter of minutes to bring up in the farm needing the added power. When the extra power is no longer needed, simply re-provision the servers back to the configuration. Scripts can be used to create personality, allowing the server to interact differently in the solution if needed. As an example of how this benefits a working organization, Freeze.com has a server farm that hosts and currently uses four IBM blades to handle the load for millions of users. When a marketing initiative causes traffic to spike by a few million people a week, Freeze.com is able to utilize idle servers and re-provision them to handle the added load of web traffic. Then at night when that server is needed to handle number-crunching functions again, it s moved from a web farm to an application cluster. With the ability to scale the servers in a mix of private and shared images, the scale out takes only about five minutes about the time it takes to reboot the server. This new model achieves almost 100% uptime for the end users and allows the IT staff to watch server performance and become more proactive in understanding traffic shifts and building models that will not fail under heavy load. Server management Citrix Provisioning Server for Datacenters is designed and built for the network administrator. Citrix eliminates ordering pieces of hardware, waiting for hardware to arrive, installing the operating system, updating the operating system with patches and antivirus software and any other routines that go along with a new server install. There is just one install. After that just copy the image and it is put in production. Installing a new server goes from days to minutes. Citrix Provisioning Server for Datacenters is also ideal for companies that do in-house developing. The administrator can hand over a new server to the developing team and if the developing group requires a re-image of the server, they can have another server back up again in little time. With Citrix it is a point and click operation to boot different operating systems, server configurations and even to bring up another server into a cluster for peak traffic times. Companies also reap productivity benefits from Citrix because system administrators can be twice as productive, able to handle 40+ servers easily. 4

5 High availability and storage considerations Storage has been growing by leaps and bounds over the last few years, so it s more critical than ever to fully understand what your vendor can provide. Understanding the storage requirements of a true fiber channel environment over a NAS and IP is critical to feeling confident that your project will succeed. The Freeze.com project included a fully redundant NAS head that was scalable on demand. Great care was taken to provide a fully redundant solution in terms of hardware and physical locations of the servers to assure full uptime. Four Terabytes (4TB) of usable space were provided for this project and were used to provide full redundancy. Citrix Provisioning Server s High Availability capability requires sharing a file system across multiple boot servers allowing for a boot server to pick up sessions from any blade in the environment if a server should fail. Sanbolic software was used to assist in this model and allow for multiple servers to have read /write access to the same volume the central database resides on. With this boot-on-demand model, hard drives are not used and local storage is not available for retaining files so it is necessary to provide storage to all servers. As the goal for this project was to provide a complete scalable model for every aspect in the data center, it was decided that it must also allow for NAS type storage. Traditional IP storage would not allow for this project to scale, as it would be necessary for the administrator to provision each server with a unique volume of storage. In a server farm this breaks the point-and-click model of this solution. To be successful, a scalable NAS solution needed to be associated with this project. Windows Server 2003 Enterprise was used to cluster two blades on separate chassis using Citrix Provisioning Server for Datacenters on the backend to assure scalability across several blade centers. As additional NAS heads are needed, it becomes as simple to add storage horsepower as it is to add servers. If a sweet spot is missed in the configuration of files being served, within five minutes an additional NAS head can become part of the group and automatically scales out the storage. The storage was designed to recover from a major additional load to the environment as well as sustain a power failure or network outage. Dual-storage controllers, switches and blades assure that it would take a major impact to disrupt the service. 5

6 Backup and disaster recovery Backups are critical to the success of any organization and with more stress on regulations and a narrowed backup window, it becomes even harder to assure they are completed in compliance with policy. A fortunate byproduct of the Freeze.com project was that backups across several servers can be handled in process. The Citrix Provisioning Server for Datacenters software uses one central database to hold all data critical to every bootable server. This means that with the ability to back up open file systems, it is also possible to back up endless server configurations. It is easy to see the hard and soft cost savings achievable with this solution less backup software needed and fewer man hours required to oversee all backups. Simply backing up one volume assures you an up-to-date restore across all servers in the event of a disaster. Snap-shotting volumes internally in storage also allows for a quick recovery if a human error or scripted error occurs. Simply restore the snapshot and you can roll back in time prior to any changes on the application layer of your new project without the added time it takes to restore from tape or disk. With Citrix powering this solution its easy to see that simply replicating the storage volume to a second location would assist in developing a best-of-class disaster recovery plan. High availability between two physical locations By replicating volumes in real time and taking advantage of network load balancing and other key software applications, it s conceivable to build a fully redundant solution that can shift resources from one data center to another based on sever loads and other metrics. QA testing and patch management Citrix Provisioning Server for Datacenters allows for an extremely rapid roll out of mirrored and unique servers. During the bug testing and load testing of new environments and testing in-house workloads, it becomes evident that this solution can solve almost all issues, bridging the gap between test labs and real world environments. Building and destroying server images becomes an overlooked key to this solution. A simple example of this would be pulling server hardware from a working server farm with little traffic and re-provisioning it to pre-test if a new MS patch will break any elements in your workload. When testing is complete, simply reprovision that hardware to pick up the same image as it booted in the past and within 5-10 minutes the patch management team can be assured success when rolled out wide scale. Testing workloads and scale-out issues also becomes a moot point. Stress testing workloads and configurations prior to release is as simple as adding a few servers and running the tests. When complete, simply return the servers to idle or back to the appropriate task. In the past, this process could take weeks ordering hardware, provisioning and testing. With Citrix Provisioning Server for Datacenters, your test labs and actual production environment can remain on the same platform. Data recovery The ability to access and recover data in a matter of minutes is essential to a successful redundant system and solution. By moving storage away from individual servers to a central location dedicated to storage performance, bottlenecks and mundane obstacles are easily dealt with. It is very easy for the IT department to perform server restores and a central storage solution handles disk scrubbing and administration. Growing volumes and restoring backups 6

7 and snapshots can be performed effortlessly, allowing IT to concentrate once again on what is best for the company and not using time normally associated with server restoration/ administration. Using Citrix Provisioning Server for Datacenters to power the server environment also means the ability to distribute server images world wide without the need to be on site. Simply create the image on like systems and transfer the OS images to any location. Mark the revision of the image in the Citrix GUI and schedule a reboot for off-peak hours. Using this model, organizations have the ability to shift network load between geographic locations in a matter of seconds and assure they meet proper SLA requirements no matter the environment variables in each location. Solution integration Solutions used in this project are: IBM Bladecenters with HS20 Blades; Citrix Provisioning Server for Datacenters; Sanbolic to allow shared storage on backend devices. To assure a full fail-over model throughout, power was built in the data center to allow a small power grid from redundant power feeds on rack transfer switches between individual circuits. Network was assembled with a full fail-over model and multiple switches to assure constant connections at all times. Storage sits in a fully redundant configuration between switches and storage controllers. NAS heads are split in a Microsoft cluster between bladecenters to assure traffic even if an entire bladecenter needed to be taken offline for maintenance. Boot servers are arranged per server in a high availability manner with two local dedicated SCSI drives in each server on a mirror RAID configuration to assure uptime per bladecenter if maintenance should be required per boot server. Core load balancers assure traffic is split between blade centers and server farms. Blade Server Configuration While building this concept, the main focus was on assembling the best-of class hardware and software to assure SLA requirements across several geographic locations and keep return on investment in line both hard and soft costs. Storage Network Simply put, this configuration is safe to human error, network error and environmental issues within the facility. 7

8 Other considerations Time savings alone in provisioning servers paid for the project in one year. Previously, provisioning of servers took four to six hours per server including OS hardening and burn in. Focused on a build-out to the lowest common denominator in staffing this model can be administrated by simply watching server sweet spots. If a sweet spot is missed it s a simple matter to shift horsepower from one unutilized set of hardware to another. This shift in horsepower means it s simple to redistribute resources without the need for multiple layers of operating systems and vendors. This model strips down the current virtualization model of computing on-demand by streamlining the concept from an OS base (normally Linux or Unix) and virtualization software (VMware and the like) with a base OS on the top and shifts it to a full OS stream to the individual servers. This drastic change means the IT staff has many fewer layers to build and maintain, allowing the operating system to be streamed to the server without the staff required to maintain a base OS, virtualization software and system administrators. This solution offers a true computing-on-demand model from a small business to large Fortune 500 companies. Kyle Ohme Kyle Ohme is currently serving as the Director of Information Technology at Freeze.com. Freeze is a leading international online promotions company, providing direct marketing solutions with a high volume of customers. June 2004, Entrepreneur magazine ranked Freeze.com as the 31st fastest-growing new business in the U.S. Kyle has been instrumental in the technological foundation of Freeze. Amongst many of his achievements, he has played a crucial role in providing cutting-edge innovations in scaleable high performance computing farms, to accommodate the needs of 85 million registered users of Freeze.com and the daily online traffic of over 1.5 million users. Kyle s expertise lies in designing and building scaleable and redundant technology solutions. He commands great experience in systems, storage, backup/ recovery and disaster recovery, with a strong focus on architecting and implementing highly available/scaleable enterprise solutions. Conclusion Shifting resources from individual servers to small commodity blade servers powered by a true solution solves almost every open issue in the IT industry an amazing feat. By taking advantage of a true computing-on demand model, it s easy to see the benefits this approach can offer to IT departments in any industry. This solution is truly revolutionary in the way it solves daunting tasks and can allow the IT department to be proactive on potential issues and be ready for 20x growth in a matter of minutes. Ultimately this allows your staff to spend time on areas that make your organization money. 8

9 Citrix Worldwide Worldwide headquarters Citrix Systems, Inc. 851 West Cypress Creek Road Fort Lauderdale, FL USA T T Regional headquarters Americas Citrix Silicon Valley 4988 Great America Parkway Santa Clara, CA USA T Europe Citrix Systems International GmbH Rheinweg Schaffhausen Switzerland T Asia Pacific Citrix Systems Hong Kong Ltd. Suite 3201, 32nd Floor One International Finance Centre 1 Harbour View Street Central Hong Kong T Citrix Online division 5385 Hollister Avenue Santa Barbara, CA USA T Notice The information in this publication is subject to change without notice. THIS PUBLICATION IS PROVIDED AS IS WITHOUT WARRANTIES OF ANY KIND, EXPRESSED OR IMPLIED, INCLUDING ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT. CITRIX SYSTEMS, INC. ( CITRIX ), SHALL NOT BE LIABLE FOR TECHNICAL OR EDITORIAL ERRORS OR OMISSIONS CONTAINED HEREIN, NOR FOR DIRECT, INCIDENTAL, CONSEQUENTIAL OR ANY OTHER DAMAGES RESULTING FROM THE FURNISHING, PERFORMANCE, OR USE OF THIS PUBLICATION, EVEN IF CITRIX HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES IN ADVANCE. THE USE CASES IN THIS PAPER ARE PROVIDED ONLY AS POTENTIAL EXAMPLES AND YOUR ACTUAL COSTS AND RESULTS MAY VARY. About Citrix Citrix Systems, Inc. (Nasdaq:CTXS) is the global leader and the most trusted name in application delivery infrastructure. More than 200,000 organizations worldwide rely on Citrix to deliver any application to users anywhere with the best performance, highest security, and lowest cost. Citrix customers include 100% of the Fortune 100 companies and 98% of the Fortune Global 500, as well as hundreds of thousands of small businesses and prosumers. Citrix has approximately 6,200 channel and alliance partners in more than 100 countries. Annual revenue in 2006 was $1.1 billion Citrix Systems, Inc. All rights reserved. Citrix, NetScaler, Citrix Presentation Server, Citrix Password Manager, Citrix Access Gateway, GoToMeeting and SmartAccess are trademarks or registered trademarks of Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the United States Patent and Trademark Office and in other countries. Microsoft and Windows are registered trademarks of Microsoft Corporation in the U.S. and/or other countries. UNIX is a registered trademark of The Open Group in the U.S. and other countries. All other trademarks and registered trademarks are property of their respective owners. PDF