The Use of Open Source Is Growing. So Why Do Organizations Still Turn to SAS?

Transcription

1 Conclusions Paper The Use of Open Source Is Growing. So Why Do Organizations Still Turn to SAS? Insights from a presentation at the 2014 Hadoop Summit Featuring Brian Garrett, Principal Solutions Architect at SAS

2 Contents Commercial Analytics Software and/or Open Source?... 1 SAS Offers Unique Value to Open Source Users... 1 Experience and Expertise...1 Proven Value to Customers...1 Innovation and Leadership in Analytics...2 SAS Brings Value to Open Source Solutions... 3 SAS/IML Software Integration...3 SAS Enterprise Miner...3 SAS Supports the Entire Analytics Life Cycle... 4 Preparing Data Using SAS...5 Building and Validating Models Using SAS...5 Deploying and Monitoring Models Using SAS...5 The Facts on Total Cost of Ownership... 5 SAS Analytical Innovations for Hadoop... 6 SAS High-Performance Analytics...6 SAS Visual Analytics at Scale...7 SAS Visual Statistics at Scale...7 SAS In-Memory Statistics...8 Making SAS Accessible to Professors, Students, Researchers and Independent Learners... 8 Learn More... 8

3 1 Commercial Analytics Software and/or Open Source? It s a hot topic today. As customers debate which is the best way to go, recent findings by Nucleus Research suggest that many organizations have realized that they can meet both internal and external stakeholder requirements by finding the right balance of SAS enterprise-class analytics solutions and open source solutions. Why? Because SAS is optimized for operational and production analysis and includes integrated capabilities for data management and more, while open source quickly brings new analytic algorithms to market. 1 But how can these technologies coexist in the real world to meet different business needs? Do they play together well? How do they work together to leverage Hadoop? These and other questions were answered in a recent presentation at the 2014 Hadoop Summit by Brian Garrett, Principal Solutions Architect at SAS. His presentation With the Rise of Open Source, Why Organizations Still Turn to SAS highlighted little-known facts about SAS investments in software enhancements that allow analysts to incorporate R algorithms into analytic processes as part of a comprehensive, enterprise-class SAS analytics platform. SAS Offers Unique Value to Open Source Users Everyone knows SAS. What they may not know is what SAS offers to users of open source analytics and especially to those users storing data in Hadoop clusters. Garrett explained, From my perspective, SAS offers four key strengths: experience and expertise, value to customers, innovation, and leadership in analytics. Experience and Expertise SAS the world s largest, privately held software company was founded in 1976 and currently employs more than 13,000 people worldwide across 400 office locations. Our employees are one of our biggest assets because of the depth and breadth of their experience, said Garrett. On average, the average tenure at SAS is over 10 years, which is important because 1) usually, original code developers are just down the hall and available for consultation, and 2) SAS employees have a tremendous amount of intellectual property in their heads IP that s readily available to peers and customers. In addition, SAS employee turnover rate averages 3 percent annually compared to the industry average of 22 percent, noted Garrett. Eighty-five percent of our statistics, statistical product and testing teams have advanced degrees and fully half of them have PhDs in math, statistics or operations research. So we manage to attract and retain some of the best and brightest analytical talent in the industry. For example, several years ago, SAS created an advanced analytics lab to tackle some of the newest emerging technologies and 98 percent of employees have advanced degrees and one-third have PhDs. It s this level of expertise that separates SAS development and testing processes. They can and do perform rigorous testing and validation of all algorithms used in products so that they are proven reliable, accurate and ready for enterprise use, continued Garrett. } } About 25 percent of our revenues are put back into research and development. That s huge for a software company. But we believe it s critical to driving innovation and leadership in this space. Brian Garrett, Principal Solutions Architect, SAS Proven Value to Customers SAS software is currently in use across approximately 70,000 customer sites in 135 countries including 90 of the top 100 companies in the Fortune 500. One-fourth of our revenue comes from the financial services industries, and three-quarters comes from all other major segments, noted Garrett. So we re delivering value across a wide range of industries. Figure 1 details these industries. 1 SAS Analytics and Open Source. Nucleus Research, April 2014.

4 2 Figure 1: Percentage of SAS business by industry. Equally important, SAS Business Solutions go far beyond analytic algorithms. They encompass solutions for data management, analytics, business intelligence, and high-performance analytics. Under this broad umbrella, SAS offers many horizontal and vertical solutions that help organizations use their data to solve contemporary and industry-specific business problems. Our analytic solutions all use the same underlying core technology, meaning they use the same algorithms and statistical methods in very specific ways to address such issues as fraud, credit risk, merchandizing effectiveness, customer intelligence and more, noted Garrett. So while Hadoop enables the persistence of large amounts of data in a relatively cheap fashion, SAS focuses on what you can do with data, what it can tell you. work with leading academic researchers for example, to learn about new methodologies and algorithms, as well as to evaluate the solutions robustness and effectiveness for use by customers. By bridging these two worlds academia and business SAS delivers innovative methods that matter to our customers, drawing on rapidly expanding analytical disciplines. At the same time, through continual and longstanding engagement with universities, SAS shares its own best practices with professors and students, broadening the practices application to other disciplines and industries. SAS is constantly including extensive customer input when building and enhancing products, ensuring close alignment between software functionality and business need. At the same time, SAS provides comprehensive training and documentation for all products, which is critical to ramping up and supporting users so customers quickly realize value from their SAS investments. Innovation and Leadership in Analytics Customers, industries and research organizations seeking innovative solutions to new analytic problems rely on SAS. To develop these solutions, SAS developers engage in conversations with customers across many industries to learn about their most pressing business problems. At the same time, SAS experts are actively involved in professional conferences and Advanced analytics developers at SAS learn about the broad range of problems that customers have and at the same time, they keep up with new developments in our respective disciplines. This combination enables SAS to develop new methods and algorithms that make a real difference in practice.

5 3 SAS Brings Value to Open Source Solutions The open source community brings a tremendous amount of value to the analytics community, explained Garrett. It brings together people from very different backgrounds and experiences to solve complex problems. And at SAS, we believe there s a lot of really good, collaborative work that s been done identifying new problems and finding new ways to solve them. And that s why SAS has opened up key parts of its software to integrate with certain open source software. SAS/IML Software Integration For example, SAS/IML a matrix manipulation product that supports matrix-vector computation gives users the flexibility to create custom functions. It also: Takes advantage of built-in functions, subroutines and SAS procedures. Enables interfaces with R language so users can submit R code within SAS. Moves between SAS and R data structures. Enables R functions and packages. With SAS/IML, users can submit R code within SAS, stated Garrett. So you can write R code and be running SAS/IML software. Say you want to do some matrix manipulations and some functions within R. Now you can do that. To learn more, see: SAS Enterprise Miner SAS Enterprise Miner a data mining product also integrates with open source solutions. SAS Enterprise Miner has an easyto-use, drag-and-drop interface that allows people to do data mining with ease and to create models. Integration with open source provides access to R modeling packages. In addition, if a model supports PMML (Predictive Modeling Markup Language), SAS Enterprise Miner can convert the PMML to SAS score code for conversion to production. (If a model does not generate PMML, users can still assess the model against other SAS models.) You can leverage R modeling packages from a SAS environment and generate corresponding PMML models, compare R and SAS models in one interface, and create ensemble models, stated Garrett. As shown in Figure 2, using SAS Enterprise Miner, users can simply drag and drop particular blocks and connect them together to create a process flow. In PMML output mode, the Open Source Integration node translates the R model into SAS DATA step code using PMML. The node then scores all imported data partitions with the generated SAS score code so users can easily compare R and SAS models. The node automatically runs standard SAS Enterprise Miner assessments for supervised predictive models. But the real value of the integration is that people can also create ensemble models using R and SAS, explained Garrett. SAS Enterprise Miner can import R models and be used for model transformations, imputations and more. People can then Figure 2: Using SAS Enterprise Miner and R.

6 4 Figure 3: Creating ensemble models. build R models with ease, as it wraps a framework around the R model. They can also take the two models, put them together, and make an even better model (for instance, by better targeting populations that impact critical business decisions). Figure 3 illustrates how a blended model can combine the best of both an open source and SAS model to achieve the greatest lift, and thus the greatest improvement in overall performance. SAS Supports the Entire Analytics Life Cycle As discussed previously, SAS integration with open source optimizes how data scientists can explore data and develop models. But as shown in Figure 4, SAS also fills in the gaps across the entire analytics life cycle. For example, many open source analytics software products do not support data management and preparation or model deployment and monitoring or if they do, it s in a way that is too difficult or cumbersome. In contrast, SAS solutions allow you to have an end-to-end solution across the entire analytics life cycle. Figure 4 illustrates how SAS innovates across the entire analytics life cycle not just algorithms or a new DATA step. So if you start at the top of the diagram, you can identify a specific problem that needs to be solved, explained Garrett. Our solutions then support the data preparation phase. Once data is prepared, people want to explore their data and SAS data exploration solutions make that easy to do. And so on, across the entire life cycle. We ve built our success on the fact that we can offer solutions for every step in the analytics life cycle even the building, care and management of data models and the scoring. Figure 4: SAS solutions support the entire analytics life cycle.

7 5 Preparing Data Using SAS Preparing data for analytics is different than preparing it for traditional IT purposes. For data to be analytics-ready, all data preparation steps must be completed, including data aggregation, data transformation (for distribution transformations), data enrichment (for deriving new variables), and analytical data cleaning (for missing values). Completing these steps takes a great deal of time and effort; in fact, as much as percent of time spent on an open source analytics project is on data preparation. R packages are not very good when they run into different types of data or different formats, explained Garrett. Nor does R allow for automatic treatment of measurement levels, character variables and missing values. Building and Validating Models Using SAS Once the hard work of data preparation is done, the fun begins. SAS gives users lots of algorithms and methods to solve a given problem. With SAS, users can create better-performing models using innovative algorithms and industry-specific methods, as well as verify results with visual assessment and validation metrics. SAS software also helps users easily compare predictions and assessment statistics from models built using different approaches, since they can be viewed side by side. Deploying and Monitoring Models Using SAS Once a model is finalized, it needs to be put into production. SAS has several technologies for model deployment and monitoring, including score code that can deploy in traditional relational databases and in Hadoop, commented Garrett. Our SAS Scoring Accelerator for Hadoop actually allows you to pull in a particular model, build in the parameters, build equations into calculations, and then push it into Hadoop. It s a one-click conversion process that saves you weeks and even months of time if you are trying to convert to SQL or JAVA, so you can build your model quickly and run it in parallel. At the same time, SAS Model Manager streamlines the steps of creating, managing, deploying and operationalizing analytic models. Users can import R models into SAS Model Manager and then transform the scored output into a SAS data set for reporting. And by using the SAS Scoring Accelerator for Hadoop, they can push score coding directly into Hadoop, significantly reducing data movements. The solution s performance monitoring and retraining capabilities help users take quick actions if model performance starts to degrade. The Facts on Total Cost of Ownership However, some nagging questions persist: From a financial perspective, does it make sense to run open source and SAS solutions concurrently? Isn t open source significantly less expensive because the software is free? Garrett addressed these concerns head-on. People often assume that open source is less costly because there s no software to license, he explained. But the total cost of a solution encompasses much more than just license fees. In fact, it actually comprises four variables: Hardware. Software. Human capital for lines of business (HC LOB). Human capital for IT (HC IT). Figure 5: Modernizing the analytic ecosystem leads to lower IT costs.

8 6 Figure 6: Open source solutions increase human capital costs (IT and line of business). As shown in Figure 5, as organizations modernize their analytic ecosystems over time, their total cost of ownership should decrease dramatically. For example, when companies move from legacy platforms and warehouses to grid computing plus Hadoop and a comprehensive suite of high-performance, extremely scalable algorithms for distributed computing that uses the latest analytical innovations and all this computing happens in memory they see dramatic reductions in TCO, noted Garrett. Stated Garrett: People assume that by building their own Hadoop distribution using free or much lower-cost open source software, they can get to the same place or close to it even cheaper. But this kind of thinking ignores the other cost variables. Because if I overlay total open source costs on this same chart, the costs from a human capital perspective actually grow. Figure 6 illustrates the unexpected cost increases. Why? Because organizations have to either hire or reallocate staff to do considerable extra work to get all of the technology built, integrated, tested and running. People are busy coding, integrating, caring for systems and models, and more, explained Garrett. At this conference, I spoke to a gentleman who has been building his own Hadoop distribution for over five years! All this work the human resources being consumed in both IT and the associated lines of business costs money. These are just some of the costs that are often overlooked when people consider using R or other open source software. SAS Analytical Innovations for Hadoop Garrett ended his presentation by sharing some recent analytical innovations developed by SAS that leverage Hadoop. These include in-memory analytics, visual analytics, and visual statistics. SAS High-Performance Analytics SAS offers high-performance analytics that run in memory for lightning-fast processing even for the largest data sets on Hadoop. We had a customer who was building risk models that were taking longer and longer to run, explained Garrett. The customer needed to do weekly or daily models, but they were approaching the limits of their time frame to do it. So they asked us to help them speed things up. But the amount of data they needed to process and the computational intensity required to do it was quickly outpacing Moore s Law. So we took the algorithms they needed and rebuilt them to run them in parallel across a set of machines. SAS then worked with the appliance vendors to build databases that can run the calculations in a massively parallel fashion. As Hadoop became more popular, SAS re-architected about 50 algorithms and individual statistical procedures so they can run inside Hadoop at high speeds. These procedures are used by a large number of SAS products and solutions.

9 7 Not every algorithm is parallelizable but we ve taken the ones most frequently used by customers and built them out to run in parallel, noted Garrett. Whether it s a logistic regression or neural networks or SVM (Support Vector Machines), or if you have millions or hundreds of millions of records that you re trying to do these sorts of calculations on, you can use a tool like SAS Enterprise Miner to leverage modern machine learning algorithms to run in memory and in a highly distributed fashion. SAS Visual Analytics at Scale Business analysts need tools to help them play with large volumes of data and reveal hidden patterns. But traditional business intelligence tools begin to fall over as you get into billions of records, commented Garrett. And today, many customers are way beyond the hundreds of millions of records. So our challenge was to build in-memory tools that would allow for high-speed data visualization, visual analytics, and more when people are dealing with massive data volumes. SAS is proud to have released a number of products like this. The first one is SAS Visual Analytics a drag-and-drop tool that goes against millions, hundreds of millions, even billions of records kept in a distributed, in-memory, analytical store. When I say distributed, I mean taking the data that you ve splayed out across your HDFS across all of your particular nodes, noted Garrett. And enabling each of those particular machines to play their part to lift that data in parallel, up into RAM. And maybe you do it directly on your new cluster, where you take some of the RAM off of each of your Hadoop nodes. Or you build up a rack of machines nearby for that special purpose a math rack, if you will. Instead of lifting directly up into memory, you lift in parallel up into that particular math rack that can perform the specific analysis you want. These visualizations are accomplished using the SAS LASR Analytic Server an in-memory, distributed, stateless system. Explained Garrett: It s very different than an in-memory database, because you don t ask for rows and columns out of this. You ask for analysis, such as a forecast, a decision tree, a correlation matrix, and so on. And you can do all this in a massively parallel fashion and get results at incredibly high speeds. SAS Visual Statistics at Scale SAS hasn t forgotten about the data miners, modelers, and data scientists who need to perform exploratory modeling, such as: Supervised learning (for example, logistic regression, linear regression and generalized linear modeling). Unsupervised learning (using decision trees and clustering). Model assessments and comparisons (such as lift, ROC, and classification rate). Group BY processing. Discovery at the observational level (for instance, to identify outliers and influence points). To support exploratory modeling, SAS has developed a new offering called SAS Visual Statistics, Garrett remarked. SAS Visual Statistics allows multiple users to quickly and interactively customize their models. They can add or change variables, remove outliers, etc., and instantaneously see how those changes affect model outcomes. And they can look at multiple models to determine which one provides the most predictive power. SAS Visual Statistics at a Glance Interactive and exploratory predictive modeling in a superior visual environment. Seamless integration of data exploration and model development. Concurrent access to data loaded in memory in a multiuser environment. Support for predictive modeling techniques such as clustering, linear and logistic regression, interactive decision trees, and general linear models.

10 8 SAS In-Memory Statistics For people who prefer to write code rather than use a GUI, SAS offers SAS In-Memory Statistics. This solution does everything SAS Visual Statistics can do but in a concurrent, multiuser environment within a programming environment, added Garrett. For example, users can perform descriptive statistics, regression (both linear and logistic), decision trees, random forests, generalized linear models, text mining, forecasting, and clustering. And because data persists in memory, organizations benefit from faster computation time. With SAS In-Memory Statistics, users can also work with raw data and then program as needed to generate a wide variety of advanced analytical methods and machine learning algorithms. It also includes a recommendation engine that generates both explicit and implicit recommendations. Making SAS Accessible to Professors, Students, Researchers and Independent Learners Analytic skills are highly sought by today s employers. SAS understands this and it s why we make SAS software available and free for people who want to learn it. Our goal is to seed the market with analytical talent. To this end, SAS has created the SAS Analytics U program, which makes SAS software readily available to professors, instructors, students and researchers in an academic setting, as well as to independent learners seeking to learn SAS to attain skills required for a current or future job. SAS Analytics U is a comprehensive global program that offers professors, students, academic researchers and independent learners access to: Free SAS software. Helpful resources to install, learn and use SAS. Free online classes. Interactive, online SAS Analytics U Community. Learn More To learn more about analytic solutions for Hadoop users, please see the research brief Eyes Wide Open: Open Source Analytics Software from the International Institute for Analytics, which is available at sas.com/openeyes. There has been a strong adoption of SAS as a result of this new program. In fact, more than 100,000 people have taken advantage of this program in the first six months of its availability. SAS has made its software available for free to people who want to learn it with its SAS Analytics U program, concluded Garrett. They also make videos, training, full documentation, and technical support available. Even if you are an independent learner you can get a copy of SAS today and play with the product in a noncommercial environment.

11 To contact your local SAS office, please visit: sas.com/offices SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies. Copyright 2014, SAS Institute Inc. All rights reserved _S