Turning Big Data into Big Value Martin Kits van Heyningen President & CEO
My Wireless Devices Uploading to the Cloud
More Wireless Sensors on my Phone and my Watch
And my Bike all Uploaded to the Cloud
Drill Down for Analytics
Enablers of Big Data Increased connectivity Proliferation of small, cheap sensors New technologies for capturing, storing, processing, and analyzing vast amounts of data Cloud storage Cloud computing On-line analytics and visualization tools
Internet of Things Billions of connected devices in the world Sensors in everything from thermostats to toasters Are we going to run out of IP addresses? IPV6, a 128-bit value Petabytes of data M2M sensors never sleep
Big Data in Formula One Racing 240 terabytes of data is sent from vehicles to crews during every race More data than there is in the entire Library of Congress Hundreds of sensors on every vehicle Tire pressure, fuel burn efficiency, aerodynamic forces, engine temperature, etc. Data is analyzed in real-time by onsite race engineers Remote experts and at team headquarters overseas Using Big Data to both predict outcomes before the race begins Make adjustments and optimizations during the race Source: Forbes
What is Big Data?
What Big Data Isn t!
What is Big Data? Big Data is not a single technology, technique or initiative Rather, it is a trend across many areas of business and technology Big Data involves data that is too diverse, fast-changing or massive for conventional technologies V3 Big Volume, Velocity or Variety of data Unstructured data Traditional use of the relational database has become a liability for many companies
Two Types of Big Data Operational systems For real-time, interactive workloads where data is primarily captured and stored Analytical systems Capabilities for retrospective, complex analysis that may touch most or all of the data Each has their preferred tools and technology architecture Operational systems tools Support unstructured databases (document databases like MongoDB) NoSQL databases are natively able to handle load by spreading data among many servers Analytical systems tools MapReduce has emerged as the first choice for Big Data analytics Combining operational and analytical technologies using Hadoop Apache Hadoop has a storage part and a processing part
What is Big Data Good For? Enables fact-based decision making Provides opportunity to explore the bigger picture Intelligent operation of end-to-end systems Local performance monitoring and optimization Condition-based and predictive maintenance
Goal: Data Analytics to Reach Foresight Source: DNV
How Data Can Give You a Competitive Advantage Example: Progressive Insurance Shows a list of quotes from all their competitors Often, Progressive isn t the cheapest Why do that? Better data, better analytics, better models Keep profitable customers and hurt competitors Give competitors expensive claims Live telemetry from your car to the insurance company Data on speed and mileage Acceleration Cornering Further identify profitable customers and reward with lower rates
What Does One Trillion Cells Cost? A spreadsheet that has one million rows and one million columns Five years ago, this sort of programming would require over a million dollars of hardware to run So cost of computing power required to run a trillion-point dense matrix?
$10
What s Needed to Get Started Need data capture Need ability to economically transmit big data Need cloud storage/computing Need analytics and experts Need actionable definitive cost savings or profit optimization
Challenges for Big Data for the Maritime Industry No onboard IT staff Globally-dispersed collection points, always moving Tons of data to transmit Difficult to update onboard servers, applications Real-time telemetry is difficult Satellite transmissions more costly than land-based Internet
On-Vessel Solutions Needed Using CANbus, ethernet, and data capture modules Centralized repository of all key data that can be delivered off ship Onboard data pre-processing in the local cloud Aggregate/compress to reduce file sizes Use Local Virtual Machine servers VM image/software distributed via multicast standardize across fleet Always current - no ad hoc updates onboard
What Data Can You Capture Engine monitoring parameters Myriad of sensors onboard Training records by person Crew morale Maintenance records and breakdowns Sensors of opportunity Voyage Data Recorder information KVH TracPhone VSAT accelerometers and gyros can show: Roll and pitch of vessel (waves, safety) Good proxy for sea state and wave information Accelerometers, vibrations
How to Efficiently Get Data Off the Vessels Some of the most advanced ship management companies still rely on manually entered data that gets emailed back every day KVH multicasting is a great way to get data on the ship Doesn t help to get data off the vessel KVH developing hyper-efficient data upload technologies Record, compress, and analyze locally Data is aggregated and sent at low peak usage times Critical events and alerts are sent immediately 2000 sensors and 2.5 GB/day!
Analysis Onshore & Real-time Feedback Onboard Onboard Display Shoreside Analytics Source: Stage3 & GreenSteam
Future Routing in Three Dimensions? Third dimension is money! Cost-based routing Optimize for $$, not time or distance navigation in 3D space Data input is critical to new type of routing models Engine/ship monitoring data from all sensors Cost/ton and consumption rates for HFO (heavy fuel oil) and MGO (marine gas oil) Fixed running cost of ship (crew, generator consumption, etc.) Vessel charter rates High res weather data Sea state and parametric roll Variable speed optimizer Slow down and let storm pass Speed up and get in front of a weather event Avoid ECAs (Emission Control Areas) - use cheaper fuel but longer route Weather updates arrive every six hours No Captain, you can t do this in your head
Third Party Analytics IBM s Watson Won quiz show Jeopardy in 2011 No Internet access Watson Analytics: Business built around Watson to provide cloud-based big data visualization and insights Based on natural language questions
Big Data is the Next Revolution Business process optimization It s not just about fuel Started with engineers because they are technical Videotel training scores Nine million training records Training scores predict efficiency/profitability Safety KVH Crewtoo happiness survey 120,000 seafarers Strong correlation between accidents and morale Preventative maintenance records and costly repairs Sensor data can predict equipment failures Subtle vibrations or temperature changes
Remote Analysis by Experts Pre-crime unit Predict failures before they happen Delay expensive maintenance that isn t required yet Repair onboard with live remote help Onboard videos that show step-by-step instructions New business opportunity for data analysis Retired experts
How KVH is Using Big Data to Understand and Help its Customers Analyze data traffic patterns and predict congestion and satellite purchase requirements KVH Terabytes of Data by Lat/long
mini-vsat Broadband Data Usage Analytics Usage records stream from antennas Years of data, zoom in to the five-minute level or in daily, weekly, monthly, yearly views What s in that data stream? Vessel position, speed and heading Firewall logs: every TCP/IP session is identified by application signature Logs stream to Amazon Kinesis (big data log processor) Then upload to Redshift data warehouse KVH team can analyze Redshift data with tools like Tableau Owners can manage their data usage and manage costs Correlated with usage, position, and account records valuable insight into a fleet s SATCOM needs
Probably the Most Important Thing for Managers to do is Make Data a Priority Many established companies don t have Big Data programs Most start-ups and early stage companies do No legacy data Keep proprietary data proprietary Share data that isn t proprietary Radar upload build global live radar images Over the horizon radar from shared radar Live global sea state and weather measurements
But What About Privacy? Goldcorp challenge Struggling mining company CEO took their entire geological database and put it online Secrecy about mine reserves and exploration used to be paramount Staff appalled at the idea of exposing their super-secret data to the world Offered $500,000 in prizes for anybody who could locate promising seams More than 1,400 contestants identified 110 new targets; 80% of which resulted in substantial new discoveries of gold Goldcorp now a $25 billion company
Adopt a Big Data Mindset But we are not a data company!? Every company is now a data company and you d better wake up to that fact Data is a resource in its own right Used to thinking of information to solve a task Public/premium data you can buy and import from Microsoft Azure Marketplace Need to think creatively; data won t ask questions Benefit from understanding More about your customers Sales cycles and traffic patterns Demand for your product or service Business inefficiencies
Summary Growing divide between firms that can manage in a Big Data world and those that can t MIT study found data-driven firms performed 5%-6% better each year Maritime industry lagging behind others If you want to be able to compete in the future, you had better start now and save the data No one can analyze data that doesn t exist! Dramatic changes in tools and costs Hire smart analytics team to mine the data Work with IT partners that can do more than just provide connectivity get them to solve your problems