E-Guide BIG DATA ANALYTICS

Transcription

1 E-Guide BIG DATA ANALYTICS

2 W e are a big data analytics, mobile app, realtime. Gartner predicts that by 2015, 4.4 million IT jobs will be generated to support big data, generating 1.9 million jobs in the United States. With that point, CIO s must change their thought process when it comes to big data. This e-guide, from SearchCIO.com, explains how big data is impacting today s IT industry and what CIO s must do to incorporate big data analytics in their overall business. PAGE 2 OF 13

3 'BIG DATA' ANALYTICS, NUMBER-CRUNCHING NATION Not all "big data" analytics are created equal. Not all real-time data is real or really matters. Not all killer mobile apps kill. That much can be surmised from even a cursory review of the digital weapons deployed in this presidential election, already dubbed the nerdiest in American history. But there can be no doubt that politics, like everything else, is entering the big data analytics world. Whether the much-touted mobile app used by 34,000 Romney volunteers to relay real-time voter counts at the polls turned out to be "nothing short of a failure," as has been reported and denied, or was a match for the Mobile Pollwatcher app used by the Democratic party to rustle up voters to the polls, I'll leave to the politico-technocrats to judge. The Election Day mobile apps were designed to interact with the massive databases and big data analytics tools relied on by the dueling campaigns. And there's no doubt the data mining tools and predictive analytics used by Project Orca, the Romney campaign's effort to turn big data into meaningful action, will be compared to death with those used by Narwhal, the Obama campaign's massive IT system used to segment and target voters. PAGE 3 OF 13

4 On the face of it, Orca didn't get its Narwhal. But technology? Technology, its history-making promise and terrible limitations on full display, loomed larger than ever on the political battlefields, big data in particular. The power of leveraging big data -- which requires being able to collect it and analyze it with the right data model -- was breathtaking in this election, as demonstrated by the polling done by such pollster nerds as Nate Silver, author of the FiveThirtyEight blog in The New York Times, and Princeton neuroscientist Sam Wang. Aggregating all the polls' results (which were calculated by collecting and analyzing varying amounts of big data), Silver and Wang and their ilk predicted the outcome of a political process nearly perfectly. Gut feelings, political persuasion, the timbre of one's voice -- those had nothing to do with the predictions. The pundit class ignored these big data conclusions at their peril, as one after another confessed how wrong they were -- a day late. "Politics as usual" is done with. Politics, of course, is just realizing what savvy merchants have known for a long time: Aggregating statistics from many people makes human behavior predictable. Big data analytics overcomes the uncertainty of the variations among individuals. As the power of big data to make accurate political predictions sunk in, I PAGE 4 OF 13

5 was reminded of nothing so much as Glik's, a family-owned business I covered as a retail reporter in St. Louis years ago. The Gliks have been merchants since In the modern era, their business had become a chain of 50-some fashion stores that offered al name-brand clothes in small towns in the Midwest, some with populations as small as 7,000. Come time for the obligatory story on Christmas sales and whether merchants would make their numbers that year, the big retailers would put out their hot picks and predictions for the season. This was the heyday of the "genius merchant": legends -- like media darling Millard "Mickey" Drexler, then of Gap -- who operated on instinct, who knew in their gut the right cut of coat or color for the season. When I needed a bead on the season, I made sure to call Jeff Glik, the son who was running the Glik's chain at the time. Between trips to New York, Jeff would be poring over weather data, surveying customers, shopping the competition, on the phone with managers from Missouri to Michigan, crunching the numbers to get just the right selection for each of the chain's stores. Way before the rise of the big data-driven retailer, he was doing his own version of Moneyball. Politics -- along with everything else -- is catching up with retailers. We are a big data analytics, mobile app, real-time. Don't take my word for it. Gartner Inc. stirred headlines last month with its prediction that by 2015, PAGE 5 OF 13

6 4.4 million IT jobs will be generated to support big data, generating 1.9 million jobs in the United States. Every big data-related job here will spawn three jobs outside of IT, for a total of 6 million U.S. jobs over the next four years. That's how much stock employers put in big data and in the analytics required for making information into something that matters. It's a job that will lash together mobile technology, social media and cloud computing. Gartner analyst Peter Sondergaard tried out the slogan "Nexus of Forces" to describe the "next age of computing." Everything that rises must converge. The computers that caught your attention many years ago are having profound impacts on civilization, let alone on your role as CIO in this tornado of big-data mobile computing. If the presidential election proved nothing else (and for the record, I believe it proved a great deal), it drives home the point to CIOs that their job can't be about "technology as usual." PAGE 6 OF 13

7 BIG DATA ERA REQUIRES NEW APPROACH TO INFORMA- TION GOVERNANCE STRATEGY Many organizations have struggled with managing unstructured data -- the often text-heavy, unorganized information that, left unattended, can cause huge risks and unnecessary storage costs. As the big data era continues, companies will have to reexamine and adapt their. By 2018, 25% of progressive organizations will manage all their unstructured data using and storage management policies, up from less than 1% today, predicts Stamford, Conn.-based consultancy Gartner Inc. "Once it's created, it's around forever," Gartner Research Director Alan Dayley said of company data at the Gartner Security and Risk Management Summit in National Harbor, Md., in June. "We need to do something with it, and we need to start governing it." The trouble is, many modern organizations struggle with data governance. The amount of data floating around the average organization -- much of it trivial -- makes determining who owns specific data, how long to keep that PAGE 7 OF 13

8 data and who is responsible for managing it a difficult proposition. First and foremost, organizations must understand exactly what data they have, and the value of it, Dayley said. Another cause is data confusion, especially from a regulatory compliance standpoint. "We're not clear on regulatory and compliance issues," Dayley said. "We don't understand what we're supposed to keep, so we keep everything." Information governance implementation and deployment requires input from across the organization, Dayley said during his Gartner Summit presentation: Compliance officers should be consulted to interpret regulatory compliance requirements and how long information must be retained according to these regulations. They can also help determine audit schedules. The legal team has responsibility for assessing information risk and determining a defensible deletion policy. Business users must understand the current and historical value of data PAGE 8 OF 13

9 and need to be included during the information management policy development. The primary goal of an is to make sure data supports business priorities effectively and efficiently. Any data that presents very little value, such as transient user communications, early working copies of files and old data from legacy applications, should be deleted, Dayley added. "Over time, the value goes way down, whereas the cost to continue to manage it goes way up," he said. "You just can't keep keeping everything forever. It's costly on storage; it's costly just trying to filter through all of it and understand it." INCORPORATE BIG DATA ANALYTICS If used strategically, analyzing this big data produces huge benefits, said Vice President and Gartner Fellow Neil MacDonald during the summit. MacDonald said that when it comes to information security, organizations often establish baselines of "normal" data behavior and look for meaningful deviations. Giving information more context through big data analytics allows organizations to establish a better understanding of this "normal" behavior and PAGE 9 OF 13

10 determine meaningful deviations from the baselines. "You have to have a very good idea what normal looks like, then look for meaningful variations from that to infer malicious intent," MacDonald said. "How do you find a needle in a haystack if you don't know what the needle looks like?" Difficulties arise when organizations are required to govern content that they did not create and do not own but may be responsible for -- or find value in. For example, employee-generated social media data creates potential privacy risks but can also be very useful to the business from a marketing standpoint. The cloud is another concern, because it's part of a trend wherein IT has less and less direct control of the organizational infrastructure, MacDonald said. As more elements of IT infrastructure go mobile, the tech department must offset these security concerns with detailed auditing, logging and monitoring of big data activities. "You want visibility to compensate for the lack of direct controls," he said. If properly managed, breaking down and analyzing big data provides huge business benefits, he added. "You've got the data; why not leverage it?" MacDonald said. "Focus on your objective, which is risk prioritized, actionable insight telling you what to do PAGE 10 OF 13

11 and what to focus on so that you can have the most impact on protecting the assets of your company." This won't be easy. The volume of information in the big data era, combined with new file system technologies, repository formats and nascent programming interfaces, mean that more sophisticated and mature archiving, e-discovery and compliance technologies are not yet available, Dayley said. As a result, organizations will be forced to manage and govern some of this content using manual policies and practices versus automated software while they wait for vendors to catch up, he added. Dayley predicts progressive companies will incorporate policies and products to assist them with automatically governing their unstructured data. "What these tools do is give you a good visualization of the data and help companies understand what it is," Dayley said. PAGE 11 OF 13

12 FREE RESOURCES FOR TECHNOLOGY PROFESSIONALS TechTarget publishes targeted technology media that address your need for information and resources for researching products, developing and making cost-effective purchase decisions. Our network of technology-specific Web sites gives you access to industry experts, independent content and analysis and the Web s largest library of vendor-provided white papers, webcasts, podcasts, videos, virtual trade shows, research reports and more drawing on the rich R&D resources of technology providers to address market trends, challenges and solutions. Our live events and virtual seminars give you access to vendor neutral, expert commentary and advice on the issues and challenges you face daily. Our social community IT Knowledge Exchange allows you to share real world information in real time with peers and experts. WHAT MAKES TECHTARGET UNIQUE? TechTarget is squarely focused on the enterprise IT space. Our team of editors and network of industry experts provide the richest, most relevant content to IT professionals and management. We leverage the immediacy of the Web, the networking and face-to-face opportunities of events and virtual events, and the ability to interact with peers all to create compelling and actionable information for enterprise IT professionals across all industries and markets. PAGE 12 OF 13

13 RELATED TECHTARGET WEBSITES PAGE 13 OF 13