AOL CUSTOMER SUCCESS STORY

Size: px

Start display at page:

Download "AOL CUSTOMER SUCCESS STORY"

Louisa Grant
8 years ago
Views:

1 AOL CUSTOMER SUCCESS STORY AOL serves over 65 million web pages a day. With the Clustrix deployment, each data center now has a fault tolerant, scalable database that can grow with the data set and increasing load. All servers attach to the same databases and always see a consistent view of the data. With the Clustrix architecture, AOL can greatly expand the HSS deployment throughout the company, consolidate servers, and save money and time.

2 ABOUT AOL America Online is a leading-edge web services company, offering premium and niche content sites, world-class tools and platforms. The AOL HSS storage system enables simple web-scale storage across a wide range of internal applications. It provides a fault tolerant, high performance, highly available, and inexpensive storage infrastructure for AOL s web properties. HSS base operations are post file and get file. HSS stores multiple copies of each file automatically for fault tolerance and performance. Files are spread across multiple data centers for site level redundancy. HSS scales to any number of objects and any aggregate performance target. PRE-CLUSTRIX The old HSS database infrastructure consisted of 4 shards spread across 10 machines. Each shard consisted of a production master, a backup master, a set of read slaves, and an admin database. Implemented as separate MySQL instances, the master received all updates from operations such as adding files, deleting files, changing file protection, and cleanup operations. The read slaves handled lookups for file get operations in production and the admin database handled housekeeping operations such as scrubbing the storage system for improperly protected and orphaned files. This architecture caused some interesting corner cases. As a robustness measure, apps which wrote files into HSS would check for accessibility before assuming success. If the app did not find the file, it would ingest again. Replication lag would delay this process. When ingesting a lot of files, the servers would get stuck in a loop uploading the same files over and over again, with the read slaves falling farther and farther behind. By eliminating read slaves, Clustrix eliminates this replication lag. Failures were particularly labor intensive to deal with in the pre-clustrix architecture. A hardware failure in one of the masters would result in hours of juggling machines and replication configurations to fully recover. The Clustrix built-in fault tolerance eliminates this concern as well. We don t think we can go to production with this [non-clustrix] setup with 3B rows in the database. Dan Pollack, Senior Operations Architect, AOL 2

3 CONSOLIDATING TO A SINGLE CLUSTRIX DATABASE Clustrix was first applied to the Admin replicas. AOL was able to coalesce all the data from all the shards into a single database using the Clustrix multi-master replication. Taking replication feeds from each of the shards, AOL was able to recombine all the data into one place for the administrative queries. Clustrix dramatically sped up the slowest queries from over one hour on MySQL to less than one second on Clustrix. Using Clustrix, they were able to scrub the entire HSS data set in 1.5 days, which was not previously possible. The second stage of the deployment was to replace the master and read slaves with Clustrix. With the no limits scalability of the Clustrix database, there is no longer any need to maintain read slaves in the architecture. All accesses can go to the same cluster and replication lag is no longer an issue. For site-level redundancy, AOL deployed two clusters in two data centers. Half the data is writable in each site. With the Clustrix deployment, each data center now has a fault tolerant, scalable database that can grow with the data set and increasing load. All servers attach to the same databases and always see a consistent view of the data. With the Clustrix architecture, AOL can greatly expand the HSS deployment throughout the company, consolidate servers, and save money and time. With Clustrix, the infrastructure can easily scale by over an order of magnitude to incorporate new data sets such as the entire MapQuest tile set. Rather than experience the pain [of expanding a sharded architecture], we would rather just go where we know scalability is better. Dan Pollack, Senior Operations Architect, AOL

4 ClustrixDB is a real-time scale-out SQL database engineered for the cloud. With ClustrixDB you can build innovative applications that deliver real-time analytics on live operational data at the same time as massive transactional volume. Our customer service is backed by amazing engineers that support over 1 trillion transactions per month in production in e-commerce, real-time bidding, and social & mobile applications. ClustrixDB is available as a free community edition for developers, a software download that runs on any cloud, and on the AWS marketplace. To learn more about Clustrix,visit

5 WORKING WITH CLUSTRIX

MakeMyTrip CUSTOMER SUCCESS STORY

MakeMyTrip CUSTOMER SUCCESS STORY MakeMyTrip is the leading travel site in India that is running two ClustrixDB clusters as multi-master in two regions. It removed single point of failure. MakeMyTrip frequently