Big Data: Dangers, Rewards... with HR in the middle Behind shop fronts, under office buildings, within borders of countries, a silent war is being waged. Troops are virtually surrounding our borders and protecting us from attack. In government buildings, unseen technocrats protect Government secrets. Merchants guard their markets from scammers and thieves. Health agencies keep an eagle eye out for any hint of biological threat. But you will not see these people on the streets. And whether you know it or not, you re one of the people feeding intelligence to them, all the time. Welcome to the world of Big Data, where knowing where to align your business along online advances and having staff who can utilise new tools sensibly is becoming increasingly important. What is it? Big Data is a term that has started to make its way into everyday conversation, but like all technical terms, it is going through its honeymoon period where some are treating it as the next big thing that will help solve everyone s problems. In reality, its meaning has yet to become clear to the general public. In the broadest terms, Big Data is about the huge amount of information now being collected as a by-product of everyone s online lives. For the technically-minded, big data can be thought of as, even conservatively, databases of terabytes or larger. To put that in perspective, one terabyte is around a quarter of a million digital photos. If you wanted to transport it, you would need around 1,600 CDs. The US Library of Congress printed collection could fit into two terabytes. Big Data stores include all sorts of information from where telephone calls are made to how people pay their bills, and correlations are flagged by sophisticated computer programs for further study, such as whether Tweets complaining about sickness correlate to regional disease outbreaks or if employee leave patterns correlate to stress leave claims. 1
Consider the case study of the analysis of Twitter streams at the launch of a product: when Apple releases a product, it monitors Twitter for market intelligence on how the product is being received by the market as well as any problems being encountered. The amount of data being generated was not merely huge, it was growing all the time and had to be analysed in real-time. In these cases, data acts like a flowing stream of indeterminate size. Automated analyses allow the company to focus on exploring potential issues flagged by the automation, and release solutions directly to the market as things are happening. Compared to traditional market research, there are several key differentiating elements: first, the data is not a fixed quantity and is constantly being added to; it is not pre-sorted or standardized; analysis has to be done in real-time; and the sheer volume of data requires much of the analysis to be automated. What it means for Human Resources For Human Resource practitioners with access to Big Data or in companies that amass it, such a database represents a range of opportunities and challenges that were not present even a few years ago. Finding people who can work with big data is proving challenging and has raised the rates of such people by 17 per cent in the last year. The need extends beyond the IT department however: issues such as statistical literacy to understand the data, the ability to critically analyse information, balance experience with statistical insights, and to apply findings is paramount. Yet research suggests less than four in ten employees have these skills. From a hiring perspective, corporations large enough to have Big Data need to consider how new employees will be able to engage with this new source of insight, and use it to effectively maintain a system of solid decision-making. Big Data also provides large or multinational Human Resources departments with opportunities to monitor the workforce in a way not possible previously. Conglomerates like McDonald s are using Big Data techniques to sort through applications and inform recruitment processes in a way that was traditionally the role of a Human Resources assistant. Big Data can also be used to monitor employee behaviour to assist in turnover and retention strategies. Like all technologies before it however, Big Data can be used to 2
undertake the routines, but people are still needed to make the decisions. The lesson; being open to this new technology will help define where you sit within the changes and understand the transition of the below: Table: Transition from Traditional Research to Big Data Step Traditional Big Data Formulation of question First step, the question is carefully formulated to answer a gap in knowledge A response to a flagged correlation, the researcher comes into the picture after the data is already through a first pass Identifying a sample Second step, the sample is identified as a group of people or data that will specifically provide responses to tools created to answer the research question All the data needs to be checked to see whether it is useful; a sample is more about the data that helps provide insight into the flagged correlation Collecting data Cautious, limited to the sample, directly based on the question Constantly happening, the data is chaotic and automated systems identify patterns Analysing data Using the specific information collected Using the sample of data, which is cleaned up and made into usable 3
information Reaching conclusions The end of the research project Like accounting, the research is a snapshot or refers to a period but data collection is ongoing Why the change? In the past, such data was much less accessible: routinely collected data on for example, purchasing behaviour was hidden away in physical files; while information on someone s lifestyle or other such indirect data needed to be collected specially in surveys or other data collection projects. Data from one organisation or department was seldom easily or cheaply accessible by another. Even with the introduction of computers, the lack of computer memory together with legal requirements to keep information such as bills and invoices physically meant that it was not until recently that general information was kept in digital form. But now it is: everything from payment information to telephone calls to Tweets is available for analysis. But with all this information comes the inability for it to be sifted through and understood quickly or easily by humans. Coupled with the pool of information growing incessantly, it provides different challenges to business intelligence analysts and executives. Big Data: Big Risks, Big Rewards Big Data can provide great success if used well. But if blindly followed or assumed to be high quality purely because there s so much of it could lead to danger. A big risk in big data is that the automation can end up providing less-experienced or less-savvy users a sense of false security; wasting effort and leading to poorly supported decisions. Commentators are talking about how not all Big Data is high-quality but this misses two- 4
thirds of the equation: it s not just the data that makes up a good analysis, but the question and the conclusion as well. This is particularly troublesome in an era overflowing with information; where staying on a relevant, result driven path can become clouded. People were previously critically engaging with limited information, but are now acting as passive consumers of an overflow of information. Added to that, traditional barriers such as research departments or professional editors simply don t exist to stop poor quality or even outright wrong information from gaining public exposure. Passively accepting as true whatever the computer tells you, and assuming it is true because it comes from the latest faddish term, Big Data could be disastrous in the wrong hands. Big Data means that Human Resources and Information Technology departments must work more closely together to ensure that this potentially valuable new tool is successful for business. 5
References Bischoff, G. big data is a big tool for the medical community in Urgent Communication online: 31 January 2013 http://urgentcomm.com/data/big-data-big-tool-medicalcommunity Healy, M. Lies, damn lies and big data in Information Week, 5 November 2012 Kubick, W.R. Big data, information and meaning, vendors have developed systems for massive databases, but are we data ready? in Applied Clinical Trials, February 2012 Newman, P. Poor data quality leaves companies vulnerable, without insight in Experian QAS, online: 30 January 2013 http://www.qas.com/company/data-qualitynews/poor_data_quality_leaves_companies_vulnerable,_without_insight_9386.htm Sanderson, M. Maximise performance with big data in Health Management Technology, January 2013 Schectman, J. Big data pay premium highest it will ever be in Wall Street Journal, online: 31 January 2013 http://blogs.wsj.com/cio/2013/01/31/big-data-pay-premiumhighest-it-will-ever-be/shah, S. Why big data doesn t always equal big insight in Information Week, 7 November 2011 Sorensen, C. The new boss: big data: companies are turning to computer programs to decide who to hire, fire and promote in Maclean s, 29 October 2012 6