Addressing Fraud and Privacy Issues in Hadoop

Transcription

1 White Paper, August 2014 Addressing Fraud and Privacy Issues in Hadoop

2 MapR Technologies, Inc. White Paper, August 2014 Executive Summary Harnessing big data has become a top priority in the financial services industry. Financial services firms are trying to figure out how to leverage the benefits of big data while continuing to address regulatory compliance issues to protect sensitive data, and maintain performance, scalability, and high availability. This white paper will cover: Best practices for achieving performance, reliability, and protection of sensitive data in Hadoop deployments Innovative capabilities and solutions to help you meet regulatory compliance and risk management requirements Use cases to illustrate how big data can maximize returns on your investment, while minimizing risks The 5 action items you should take to get started Hadoop in Financial Services The business of financial services is centered on transactions and data. And the scale of structured and unstructured data generated in the financial services industry continues to explode. In order to thrive, financial services firms need to efficiently manage and generate insights from this data. They need to leverage this data to improve operational efficiencies, detect fraud quicker and more accurately, model and manage their risk, and reduce customer churn. Hadoop has emerged as a technology system of the future in financial services. According to Gartner, big data is the number one search query into Gartner Research today. Hadoop addresses multiple financial services priorities such as scale, decreasing fraud and risk through anomaly detection, data archiving for compliance, faster time to insights, cost saving through architecture consolidation, 360-degree customer view, and data protection. This white paper will examine some of the security considerations inside Hadoop, such as how to shift to real-time processing, as well as leveraging automation and on-demand applications. Key Challenges of Big Data Every industry is facing the challenge and opportunity of unprecedented data growth. One of the challenges in financial services is how to fully leverage big data without exposing or risking sensitive information. There has been a major increase in the proportion of unstructured data in big data architectures. This poses a security challenge for financial institutions. Financial services companies have done a good job identifying where sensitive data exists in structured fields. However in unstructured data like PDFs or text files, sensitive data such as credit card numbers and transaction amounts are not easy to track. This drives a much higher need for data discovery to identify and classify sensitive data. Another security challenge for financial services entails compliance. In the 1990s, the priority in IT was security making sure data didn t get stolen or illegitimately modified. Today the first priority is making sure you are compliant with the regulatory rules. To deal with compliance, you need to be mindful of data lineage, monitoring, reporting, and auditing. You need to demonstrate that you know where your sensitive data is and show what you ve done to protect it.

3 2 MapR Technologies, Inc. Solution Priorities There are consistent priorities across industries for Hadoop solutions including performance, scale, reliability, and security. These are some best practices on how to achieve each of these priorities. Performance If you seek high performance, you must consider all the factors in your system: software, disk, compute, memory and network I/O. If you compromise on one element, it will likely impact performance significantly. Security also impacts performance, in which you can expect about 5 percent overhead. With Hadoop, you can alleviate some of the impact associated with data protection by leveraging MapReduce, so encryption and decryption can be done in a distributed process. Scale Plan for growth beyond your initial projections. Scaling your data and users is not only about extrapolating out current usage into the future it s also thinking about what other tasks you ll take on. These other tasks arise as a result of the positive impact of initial successes as you prove the value of your infrastructure, more users will want to gain similar benefits. Some other ways your data load will grow dramatically include: Extending the time windows for your data so you can create more accurate analytical models. Adding new data sources that will enrich the data source and expand the data load. Adding new use cases for your data, which will increase the user load. Be prepared to see your successful production deployment of Hadoop grow a lot faster than you expected. Reliability Reliability and business continuity are positive sounding terms but actually involve the grim topic of damage control after a failure. If avoiding a disastrous failure is important to you, as it should be in any enterprise architecture, be sure you get all the right people involved up front in the technology selection process including systems administrators, application developers, and DevOps. Don t spend a lot of time on clean up after a system failure because you didn t plan for business continuity.

4 3 MapR Technologies, Inc. Solution Priorities continued Security There are the four areas of consideration when addressing security within Hadoop: Access Perimeter Visibility Data Defining what users and applications can do with data Guarding access to the cluster itself Reporting on where data came from and how it s being used Protecting data in the cluster from unauthorized visibility Techical Concepts Permissions Authorization Authentication Network isolation Auditing Lineage Encryption Tokenization Data masking MapR lets you have multiple, segregated user groups, applications, and data sets in a single cluster. MapR offers true multi-tenant Hadoop, along with authentication and authorization controls. Dataguise discovers and protects sensitive data at the data (cell) layer and provides visibility for audit reporting and data lineage. 5 Action Items to Get Started When planning out your Hadoop deployment, start with these tips based on best practices from our customers: 1. Strategize develop concrete goals Get prepared for production. If you have large volumes of data that you don t want to copy to a remote cloud environment, or you have secure data that can t be stored at a third-party site due to regulations, then you want to stick with on-premises deployments. You ll be managing the system for yourself and you have to plan out that growth. Don t limit your possibilities. Be open to additional use cases. Hadoop deployments typically handle multiple business problems. We believe that expanding your Hadoop deployment is a matter of when, not if. Example Telecommunications Company Plans for Multiple Use Cases. One MapR telecommunications customer has built out an enterprise data hub and has already been able to identify millions of dollars in revenue. They realized they could open up the system for additional data and end users. They are planning ahead with the expectation that they can create other revenue streams by having a central repository for analyzing data.

5 4 MapR Technologies, Inc. 5 Action Items to Get Started continued Prevent the Aha vs. Gotcha moment. As we see our financial services customers move through the evolution from testing to pilot into production, we try to help them avoid surprises. As companies move into production, they can get into a phase where the governance, risk management or security group says: you can t do this, you are exposing sensitive data in ways that are outside of policy. Anticipate and get in front of those security reviews. 2. Automate, automate, automate Your data will grow 6000 percent in six years. Your IT staff headcount will grow 50 percent. Your security technology should not require security staff to manage it. Dataguise uses automation to deploy security in a way that is seamless and transparent to business users in Hadoop. Automation helps you to keep up with data growth, simplify critical tasks and reduce the risk of error with manual effort. Example Our analytics business draws on the purchasing data of its 90 million credit card holders across 127 countries. Major credit card company Major Credit Card Company and Payment Risk Management. This credit card company has multiple business applications such as fraud detection, risk analytics, and cross-selling data to third party suppliers. They were able to deploy multiple ]use cases in a single cluster, which is a significant trend in Hadoop deployments. They use a high degree of automation running discovery, scans, and protection of sensitive data natively inside the cluster on a five-minute basis. There are different credit card values attached to different sets of customers. They want to create tiers of protection inside Hadoop so people can run different levels of analytics against the millions of cards. The ability to tier, separate, and automate was critical to success there. The customer starts with source data from credit card transaction files and Omniture files moved into Hadoop via NFS or SFTP. They use Dataguise technology to automatically detect, mask, and encrypt different fields of sensitive information based on policies. 3. Foresee high rates of innovation There are frequent announcements of new technologies coming out in the Hadoop ecosystem. Take advantage of a wealth of new innovations in the Hadoop ecosystem such as Falcon, Storm, Impala, Spark, and the upcoming Apache Drill project. These new projects often add powerful capabilities that make Hadoop a more comprehensive platform for managing data. Drill, for example, lets users query a wide variety of data types without database administrator intervention. It runs SQL queries across different repositories of unstructured and structured data and ties it all together. When planning out your deployment, think about the areas with the highest rate of change, and what you can do from an architectural perspective to create non-blocking design choices. Make sure your security assumptions aren t tied to specific formats or file sizes.

6 5 MapR Technologies, Inc. 5 Action Items to Get Started continued 4. Expand your community Expand your definition of what community is for the sake of security and the common good. Share ideas and best practices with different divisions at your firm, professionals in your network, and even with competitors. This is similar to a neighborhood watch program, where protecting your neighborhood is equivalent to protecting yourself, and the collective/ collaborative effort makes everyone better off. Example We initially got into centralizing all of our data from an information security perspective. We then saw that we could use this same environment to help with fraud detection. Michael Fowks, SVP Fraud Operations and Security Analytics, Zions Bank Zions Bank: Fraud Detection and Security Analytics. MapR customer Zions Bank is a full-service US bank based in Utah with offices in several western states. Zions Bank realized that they were able to build multiple use cases in the same cluster, handling both security analytics as well as fraud detection. They built statistical models to detect fraud, and mine data to evaluate suspicious activities. Their huge volumes of collected data to identify fraud patterns required massive scalability. Their security team meets regularly with members of other banks to share best practices because they know they all benefit from this. They share their experiences with Hadoop. They don t compete with each other by being more secure, they compete in the ways that banks should, with great customer service, great rates, relevant products, etc. Both of these use cases are based on using MapR for anomaly detection. 5. Explore the technology landscape Do your research and due diligence to select the right technology for your needs. Be sure to talk to vendors, analysts and the broader community. Go to conferences and attend meet-ups to talk to other firms. And be sure to validate any advice you get with proof points. It s much harder to undo a bad decision than to take the time to ensure you re making the right decision.

7 6 MapR Technologies, Inc. Summary Big data offers significant benefits to financial services firms including decreasing fraud and risk through anomaly detection, data archiving for compliance, faster time to insights, cost saving through architecture consolidation, 360-degree customer view, and data protection. Financial services firms are trying to leverage the benefits including Apache big data while continuing to address regulatory compliance issues to protect sensitive data, and maintain performance, scalability, and high availability. The MapR Distribution including Hadoop along with the Dataguise security solutions for Hadoop are a great combination for meeting the needs of financial firms. MapR was rated the top-ranked distribution for Hadoop in a recent Forrester report on Hadoop big data solutions. And MapR specializes in bringing Hadoop customers into production. With all the enterprise-grade features, the performance, scalability, security, and administrative advantages, customers rely on MapR to get business value from their big data. We ve collected these action items from our customers to successfully address fraud and privacy issues in Hadoop. Strategize/develop concrete goals, automate wherever possible, foresee the rapid rate of innovation in the Hadoop ecosystem, expand your idea of community to share security best practices, and explore the technology landscape thoroughly before making your final technology decision. MapR delivers on the promise of Hadoop with a proven, enterprise-grade platform that supports a broad set of mission-critical and real-time production uses. MapR brings unprecedented dependability, ease-of-use and world-record speed to Hadoop, NoSQL, database and streaming applications in one unified distribution for Hadoop. MapR is used by more than 500 customers across financial services, government, healthcare, manufacturing, media, retail and telecommunications as well as by leading Global 2000 and Web 2.0 companies. Investors include Google Capital, Lightspeed Venture Partners, Mayfield Fund, NEA, Qualcomm Ventures and Redpoint Ventures MapR Technologies, Inc.