BIG DATA FOR YOUR DC AK Schultz Nov 2014
What is Big Data? It is data. And it is BIG. Page 2
BUT WHY IS IT CALLED BIG DATA? Big Data Is: These are data sets so large and complex That it becomes difficult to process Using traditional data processing applications Page 3
But How Big? 90% of ALLthe data in the world has been generated over the last 2 years! *ScienceDaily, May 2013 Page 4 Swisslog Confidential
DATA TODAY One TB was created in during while you have been viewing this slide! Page 5 Swisslog Confidential
Let s Give You A Sense of Scale 5 EB = 5 Billion GB Page 6 Swisslog Confidential
Then from 2003 to 2012 2.7 ZB = 2.7 Trillion GB Page 7 Swisslog Confidential
And now 8 ZB = 8 Trillion GB 1140 GB for every man, woman and child on earth! Page 8 Swisslog Confidential
Why Should You Care? Page 9
DATA-WISDOM CONTINUUM DATA INFORMATION KNOWLEDGE WISDOM DECISIONS Facts Gathering Research Presentation Organization Gives Meaning Application Synthesized Learning Understanding Interpretation Actionable Change Movement Optimization THE PAST THE FUTURE WHY? Reveals Patterns WHAT? Reveals Relationships WHAT IS BEST? Reveals principles WHAT ACTION? Reveals direction Page 10
In supply chain, what do we care about? Enterprise Application Data Automation Sensor Data, aka, the Industrial Internet Page 11
Valuable Data is Thrown Away Page 12
Examples from Warehousing Compared to ecommerce, this is Small Data. A typical warehouse storing pallets will generate 1 to 2 GB per mos in its WMS. A typical retail distribution ASRS dealing primarily in case handling will generate ~25 times the data for the same case volume (25 to 50 GB per mos in its WMS). 5 to 10 GB per mos of industrial internet data. 3 to 5 GB of data in a data warehouse. This means we are likely not leveraging 300 to 660 GBs annually per site. This is not so much data. Page 13
COST OF STORING BIG DATA 1 TB (Solid State) is < $500 Less if you buy lots of servers What is there to discuss? 1980 $190,000 1990 $9,000 2000 $15 2010 $0.07 Page 14 Log first. Ask questions later.
Valuable Data is Thrown Away Waterfall versus AGILE Just tell me what you want to keep and give us the business case to justify the server space. I am not sure what I want to keep, but I will know when I see it. We can t get help to build reports, but they we send us a flat file. So we will just build spreadsheets. I spend more time building these spreadsheets and charts that no one seems to care about. Page 15
Just Say No! These documents are a dead end of knowledge Flat Files Excel Expensive to create You are not saving hard drive space but simply sprinkling to data around Can be created by few. Page 16 Macros Access Distributed in a clumsy way Accessible by few, and not usually by those who can take action Viewable to the who can take action, most likely on print outs in break rooms
Why Today is a New Day Industrial Ethernet Internet of Things ecommerce Large Scale Enterpise Systems Low Cost Bandwidth Processing Power & Memory Cloud Computing Unstructured data storage Social Media High leverage, data driven, actionable insights Page 17
ENTER THE DATA LAKE Data lake for structured and unstructured data Traditional Structured Data Raw Unstructured Data DATA LAKE SQL NoSQL!!! Allows for the distributed processing of large data sets across clusters of computers Page It 18 is designed to scale up from single servers Swisslog to Confidential thousands of machines
Data Lake vs Data Warehouse Benefits Generally less expensive per GB Variety better equipped to everything from RDBMS to Video Feeds Volume Traditional data warehouses can bog down. Velocity Distributed processing enables faster storage and processing Uses MapReduce to split data in chunks that can be process in an parallel manner Page 19 Swisslog Confidential
DATA SOURCES For a Warehouse Industrial Ethernet 1. Inputs 2. Outputs 3. Scanners 4.. WMS 1. Order Data 2. SKU info 3. DATA LAKE ERP 1. Financial 2. Production 3. CMMS 1. Maint Data LMS 1. Labor Standards 2. Performance 2. Parts 3. 3. Imagine this data all living in completely separate data warehouses. Page 20 Now imagine running a query against data from each.
A Way to Help Find the Unk-Unks Most of it does not matter. Some of it is crazy important. DATA LAKE Page 21 Swisslog Confidential
WE ARE SEARCHING FOR CAUSALITY TIME SPACE RLTNSHP CAUSE EFFECT BIATHALON EFFECT Page 22 Swisslog Confidential
THE DATA LAKE DOES NOT CARE Time, Geography, Org Charts, Politics TIME SPACE RLTNSHP CAUSE DATA LAKE EFFECT BIATHALON EFFECT Page 23 Swisslog Confidential
You Need to Do Something With the Data But WHO is WHO? 1 2 3 DOMAIN KNOWLEDGE DATA SCIENCE TECHNOLOGY Page 24 Swisslog Confidential
You Need to Do Something With the Data But WHO is WHO? 1 2 3 DOMAIN KNOWLEDGE DATA SCIENCE TECHNOLOGY Page 25 Swisslog Confidential
HOW SHOULD YOU DISPLAY DATA? Customer WMS / ERP / LMS SmartLIFT SmartLIFT DASHBOARD SmartLIFT DRIVER GUI Page 26
BIG DATA INTELLIGENCE SmartLift: Big Data Meets Forklift Cockpit: A web-hosted business intelligence tool Interactive and user configurable charts Threshold based email alerts WIDGET BASED! In order for Big Data to be actionable, you need to provide: Right information, to the right people, at the right time. Page 27 It is not possible to create on GUI that meets every employee s needs on a 15 monitor!
General Guidelines 1 2 Actionable information Right info, right person, right time. Predict Failure Resolve before Impact Historical Real-time Data Aggregation Line Mgr Executive Mgr 3 Forecast throughput volume Predictive Age of Data Page 28 Swisslog Confidential
Swisslog Data-Wisdom Solutions DATA INFORMATION KNOWLEDGE WISDOM DECISIONS Facts Gathering Research Presentation Organization Gives Meaning Application Synthesized Learning Understanding Interpretation Actionable Change Movement Optimization Cockpit ConditionMonitoring CrystalBall Page 29 Swisslog Confidential
CAUTION What Big Data is NOT: Big data is NOT a panacea; it will not magically make all of your business problems go away. Big data is NOT a replacement for relational database management systems at least, not today maybe in a few years when query performance radically improves (or new methods arise, e.g. Stinger). Big data solutions are NOT simple; spinning up an HDInsight cluster is a breeze, but then what? By its very definition, you are dealing with vast amounts of data, most of which is unstructured. There is a lot to wade through to find the nuggets of relevant data; there is no easy way to perform this operation. Page 30
HOW TO MOVE FORWARD Start storing more of your data! Log first, ask questions later! Begin to turn your data warehouses into data lakes. Build a Big Data Team (Domain, Data Science, Technology). Likely you will need to outsource some of it. Master Information and Knowledge before trying to get into Wisdom. Think AGILE not Waterfall. There are many Unk-Unks. A waterfall approach will likely get bogged down. Start small and grow. Page 31
Big Data will change the face of Logisitics Technology convergence makes this an exciting time. Things we wanted to do 10 years ago are now technically and financially viable. I am fully confident that if you seriously embrace Big Data, the business upside is tremendous. Page 32
THANK YOU FOR YOUR ATTENTION. www.swisslog.com Page 33