BIG DATA FOR YOUR DC

Transcription

1 BIG DATA FOR YOUR DC AK Schultz Nov 2014

2 What is Big Data? It is data. And it is BIG. Page 2

3 BUT WHY IS IT CALLED BIG DATA? Big Data Is: These are data sets so large and complex That it becomes difficult to process Using traditional data processing applications Page 3

4 But How Big? 90% of ALLthe data in the world has been generated over the last 2 years! *ScienceDaily, May 2013 Page 4 Swisslog Confidential

5 DATA TODAY One TB was created in during while you have been viewing this slide! Page 5 Swisslog Confidential

6 Let s Give You A Sense of Scale 5 EB = 5 Billion GB Page 6 Swisslog Confidential

7 Then from 2003 to ZB = 2.7 Trillion GB Page 7 Swisslog Confidential

8 And now 8 ZB = 8 Trillion GB 1140 GB for every man, woman and child on earth! Page 8 Swisslog Confidential

9 Why Should You Care? Page 9

10 DATA-WISDOM CONTINUUM DATA INFORMATION KNOWLEDGE WISDOM DECISIONS Facts Gathering Research Presentation Organization Gives Meaning Application Synthesized Learning Understanding Interpretation Actionable Change Movement Optimization THE PAST THE FUTURE WHY? Reveals Patterns WHAT? Reveals Relationships WHAT IS BEST? Reveals principles WHAT ACTION? Reveals direction Page 10

11 In supply chain, what do we care about? Enterprise Application Data Automation Sensor Data, aka, the Industrial Internet Page 11

12 Valuable Data is Thrown Away Page 12

13 Examples from Warehousing Compared to ecommerce, this is Small Data. A typical warehouse storing pallets will generate 1 to 2 GB per mos in its WMS. A typical retail distribution ASRS dealing primarily in case handling will generate ~25 times the data for the same case volume (25 to 50 GB per mos in its WMS). 5 to 10 GB per mos of industrial internet data. 3 to 5 GB of data in a data warehouse. This means we are likely not leveraging 300 to 660 GBs annually per site. This is not so much data. Page 13

14 COST OF STORING BIG DATA 1 TB (Solid State) is < $500 Less if you buy lots of servers What is there to discuss? 1980 $190, $9, $ $0.07 Page 14 Log first. Ask questions later.

15 Valuable Data is Thrown Away Waterfall versus AGILE Just tell me what you want to keep and give us the business case to justify the server space. I am not sure what I want to keep, but I will know when I see it. We can t get help to build reports, but they we send us a flat file. So we will just build spreadsheets. I spend more time building these spreadsheets and charts that no one seems to care about. Page 15

16 Just Say No! These documents are a dead end of knowledge Flat Files Excel Expensive to create You are not saving hard drive space but simply sprinkling to data around Can be created by few. Page 16 Macros Access Distributed in a clumsy way Accessible by few, and not usually by those who can take action Viewable to the who can take action, most likely on print outs in break rooms

17 Why Today is a New Day Industrial Ethernet Internet of Things ecommerce Large Scale Enterpise Systems Low Cost Bandwidth Processing Power & Memory Cloud Computing Unstructured data storage Social Media High leverage, data driven, actionable insights Page 17

18 ENTER THE DATA LAKE Data lake for structured and unstructured data Traditional Structured Data Raw Unstructured Data DATA LAKE SQL NoSQL!!! Allows for the distributed processing of large data sets across clusters of computers Page It 18 is designed to scale up from single servers Swisslog to Confidential thousands of machines

19 Data Lake vs Data Warehouse Benefits Generally less expensive per GB Variety better equipped to everything from RDBMS to Video Feeds Volume Traditional data warehouses can bog down. Velocity Distributed processing enables faster storage and processing Uses MapReduce to split data in chunks that can be process in an parallel manner Page 19 Swisslog Confidential

20 DATA SOURCES For a Warehouse Industrial Ethernet 1. Inputs 2. Outputs 3. Scanners 4.. WMS 1. Order Data 2. SKU info 3. DATA LAKE ERP 1. Financial 2. Production 3. CMMS 1. Maint Data LMS 1. Labor Standards 2. Performance 2. Parts Imagine this data all living in completely separate data warehouses. Page 20 Now imagine running a query against data from each.

21 A Way to Help Find the Unk-Unks Most of it does not matter. Some of it is crazy important. DATA LAKE Page 21 Swisslog Confidential

22 WE ARE SEARCHING FOR CAUSALITY TIME SPACE RLTNSHP CAUSE EFFECT BIATHALON EFFECT Page 22 Swisslog Confidential

23 THE DATA LAKE DOES NOT CARE Time, Geography, Org Charts, Politics TIME SPACE RLTNSHP CAUSE DATA LAKE EFFECT BIATHALON EFFECT Page 23 Swisslog Confidential

24 You Need to Do Something With the Data But WHO is WHO? DOMAIN KNOWLEDGE DATA SCIENCE TECHNOLOGY Page 24 Swisslog Confidential

25 You Need to Do Something With the Data But WHO is WHO? DOMAIN KNOWLEDGE DATA SCIENCE TECHNOLOGY Page 25 Swisslog Confidential

26 HOW SHOULD YOU DISPLAY DATA? Customer WMS / ERP / LMS SmartLIFT SmartLIFT DASHBOARD SmartLIFT DRIVER GUI Page 26

27 BIG DATA INTELLIGENCE SmartLift: Big Data Meets Forklift Cockpit: A web-hosted business intelligence tool Interactive and user configurable charts Threshold based alerts WIDGET BASED! In order for Big Data to be actionable, you need to provide: Right information, to the right people, at the right time. Page 27 It is not possible to create on GUI that meets every employee s needs on a 15 monitor!

28 General Guidelines 1 2 Actionable information Right info, right person, right time. Predict Failure Resolve before Impact Historical Real-time Data Aggregation Line Mgr Executive Mgr 3 Forecast throughput volume Predictive Age of Data Page 28 Swisslog Confidential

29 Swisslog Data-Wisdom Solutions DATA INFORMATION KNOWLEDGE WISDOM DECISIONS Facts Gathering Research Presentation Organization Gives Meaning Application Synthesized Learning Understanding Interpretation Actionable Change Movement Optimization Cockpit ConditionMonitoring CrystalBall Page 29 Swisslog Confidential

30 CAUTION What Big Data is NOT: Big data is NOT a panacea; it will not magically make all of your business problems go away. Big data is NOT a replacement for relational database management systems at least, not today maybe in a few years when query performance radically improves (or new methods arise, e.g. Stinger). Big data solutions are NOT simple; spinning up an HDInsight cluster is a breeze, but then what? By its very definition, you are dealing with vast amounts of data, most of which is unstructured. There is a lot to wade through to find the nuggets of relevant data; there is no easy way to perform this operation. Page 30

31 HOW TO MOVE FORWARD Start storing more of your data! Log first, ask questions later! Begin to turn your data warehouses into data lakes. Build a Big Data Team (Domain, Data Science, Technology). Likely you will need to outsource some of it. Master Information and Knowledge before trying to get into Wisdom. Think AGILE not Waterfall. There are many Unk-Unks. A waterfall approach will likely get bogged down. Start small and grow. Page 31

32 Big Data will change the face of Logisitics Technology convergence makes this an exciting time. Things we wanted to do 10 years ago are now technically and financially viable. I am fully confident that if you seriously embrace Big Data, the business upside is tremendous. Page 32

33 THANK YOU FOR YOUR ATTENTION. Page 33