The New IoT Standard: Any App for Any Device Using Any Data Format Mike Weiner Product Manager, Omega DevCloud KORE Telematics
About KORE The world s largest M2M/IoT services provider 12 Carriers Enterprise Ready Over 110 Countries 3.5 Million KORE makes true global coverage a reality with ten cellular partners and two satellite partners, covering the entire globe With geo-redundant data centers and serving a variety of Fortune 500 leaders, KORE s team of more than 75 developers is ready to support any level of enterprise-grade project KORE provides its clients with unparalleled worldwide coverage and support with offices in 5 countries and by actively doing business in over 110 countries With more than 3.5 Million active units, KORE is the world s largest independent global provider of connectivity and application enablement As the world s largest independent global M2M/IoT services provider, we provide our customers with the most effective services for connectivity and application enablement. Doing business in over 110 countries across the globe and maintaining strong network relationships with leading carrier providers, KORE is able to provide unparalleled M2M/IoT enabling services. - Alex Brisbourne, CEO of KORE
Iot Growth This is the largest growth in the history of humans Janus Bryzek (forbes.com) Gartner claims that the Internet of Things was the most hyped technology in 2014, and for good reason 30 25 Online Machines GE estimates that the Industrial Internet has the potential to add $10 to $15 trillion to the global GDP over the next 20 years Connected Devices (billions) 20 15 10 Gartner estimates that connected things will exceed 25 billion by 2020 5 0 2010 2015 2020 Mobile Phones PCs & laptops Tablets Other "Things" Data Source: The Internet of Things, MIT Technology Review
How Connected Devices Communicate It s all ones and zeroes Manufacturers either encode their device data into a custom format, or into a human-readable encoding such as ASCII. Data transmission is typically sent via TCP or UDP protocols and facilitated via a radio or satellite signal, often through a major cellular/data carrier. GSM HSPA Satellite Wi-Fi Etc. 0110 0110 0011 1101 1110 0001 TCP/UDP 2.4m/s,1236812s,330036s,98.6f
The Problem(s) How can I develop against so many unknowns? There are no formatting standards across device manufacturers Different units Different formats Different delimiters Different encoding = The same headache There are often no persistent standards even for a single device A simple firmware update can change the structure of the message, ultimately bringing your application to a halt $ $ $ Sooner or later you will need to change or update your devices Are your existing devices subpar? Is your carrier turning down 2G or 3G service? Changing devices means you will have to redevelop parts of your application
Message Formats Differ by Manufacturer ultimately driving demand for standardization Message formats and encoding differ by device manufacturer. Consider the following GPS data: $$35890104727109 9,4003,2015/04/21, 17:42:48,39.24506, -84.38529,255.2, 0,0.0,10,0.9,11.9,16,1, 1,0,0.0,0.0,0,1,1,1## 00 00 01 4D 53 83 3F 34 61 66 38 66 34 37 62 63 41 6E 64 72 6F 69 64 20 34 2E 34 40 17 64 37 61 CD B3 DE CC 43 9F 4A 3B 8D 1A BB C3 7F 63 41 23...
ASCII vs Custom Encoding Data 1010 ASCII is a characterencoding scheme that portrays data in humanreadable text. ASCII supports 127 characters in total, the majority of which you can type via your keyboard. ASCII messages are simple to understand and can typically be parsed through the use of a delimiter (typically a comma). vs Custom Encoding enables devices to encode subsets of the message into a variety of different data types. These data types can be proprietary to the manufacturer or be standard data types such as strings, integers, floats, etc. Although more difficult to work with, custom-encoded messages typically condense the overall size of the message.
The Difficulty of Interpreting Custom Encoding E = mc, what? Lack of visualization Encoding Endianness Variety of data types Custom data types Dynamic lengths Linear parsing Rigid output Very tedious big endian unicode exponent 0xF1 bytes fixed length BCD unsigned int float packed BCD little endian 0101 bits concurrency mantissa boolean integer headers double float hexadecimal radix singed integer
Interpreting Custom Binary Encoding Step 1: Understand the device message 00 00 01 4D 53 83 3F 5F 38 65 35 64 34 37 39 34 61 66 38 66 34 37 62 63 41 6E 64 72 6F 69 64 20 34 2E 34 40 43 9F 4A 3B 8D 74 C8 C0 55 18 A0 BA F6 0A B4 3F 72 76 50 40 02 D2 09 00 00 00 00 00 00 00 00 BD 5C 2F 66 BD 20 22 79 What you see above is a 600-bit message sent from an Android device. This binary message is portrayed in hexadecimal notation in order to save space. Remember that all devices communicate with ones and zeroes. Let s begin deciphering and parsing the message
Interpreting Custom Binary Encoding Step 2: Take a chunk of data, if you can find it 00 00 01 4D 53 83 3F 5F 38 65 35 64 34 37 39 34 61 66 38 66 34 37 62 63 41 6E 64 72 6F 69 64 20 34 2E 34 40 43 9F 4A 3B 8D 74 C8 C0 55 18 A0 BA F6 0A B4 3F 72 76 50 40 02 D2 09 00 00 00 00 00 00 00 00 BD 5C 2F 66 BD 20 22 79 According to the documentation of the device, bits 345-409 are the longitude location of the device The hexadecimal value of the device s longitude in this case is: C0 55 18 A0 BA F6 0A B4 The binary equivalent of this value is: 11000000 01010101 00011000 10100000 10111010 11110110 00001010 10110100
Interpreting Custom Binary Encoding Step 3: Determine the endianness of the value 00 00 01 4D 53 83 3F 5F 38 65 35 64 34 37 39 34 61 66 38 66 34 37 62 63 41 6E 64 72 6F 69 64 20 34 2E 34 40 43 9F 4A 3B 8D 74 C8 C0 55 18 A0 BA F6 0A B4 3F 72 76 50 40 02 D2 09 00 00 00 00 00 00 00 00 BD 5C 2F 66 BD 20 22 79 To further complicate matters, some devices generate data left-to-right, while others do it right-to-left. This is referred to as endianness. Big Endian conveys the most significant byte in the smallest memory address. Little Endian conveys the least significant byte in the smallest memory address. You ll need to consult the device s documentation to determine which endianness your device uses.
Interpreting Custom Binary Encoding Step 4: Determine the data type and convert the value 00 00 01 4D 53 83 3F 5F 38 65 35 64 34 37 39 34 61 66 38 66 34 37 62 63 41 6E 64 72 6F 69 64 20 34 2E 34 40 43 9F 4A 3B 8D 74 C8 C0 55 18 A0 BA F6 0A B4 3F 72 76 50 40 02 D2 09 00 00 00 00 00 00 00 00 BD 5C 2F 66 BD 20 22 79 Following the device s documentation, we know that this value is a double precision (64-bit) float Floats are positive & negative numbers that support decimals to a varying degree of precision Typically your programming language will provide an easy conversion class or method (e.g., C# s BitConverter) to derive the value: -84.3848102
Interpreting Custom Binary Encoding Step 5: Repeat the process for each value 00 00 01 4D 53 83 3F 5F 38 65 35 64 34 37 39 34 61 66 38 66 34 37 62 63 41 6E 64 72 6F 69 64 20 34 2E 34 40 43 9F 4A 3B 8D 74 C8 C0 55 18 A0 BA F6 0A B4 3F 72 76 50 40 02 D2 09 00 00 00 00 00 00 00 00 BD 5C 2F 66 BD 20 22 79 Repeat the process for each pertinent value and ignore the values you don t need. Note that not all values will be 64-bit floats. You will need to understand how to convert multiple data types and perhaps even custom data types. Be careful, if you make a mistake early all trailing data will also be incorrect.
So Why Custom Encoding? Smaller data size = lower cost Most data service providers charge by data usage Smaller data size = lower cost Consider the longitude value: -84.3848519 ASCII 8 bits per character -84.3848519 = 11 characters 8 x 11 = 88 bits (11 bytes of data) Custom Encoding -84.3848519 is a float data type Single Float = 32 bits (4 bytes of data) Double Float = 64 bits (8 bytes of data)
IoT Development Made Easy via Omega DevCloud Let us show you how we do it
The Benefits of Middleware Focus on building your application, not integration Modularizing development by leveraging middleware will help protect you from needing to redevelop your user-facing application when devices or their message structures change. It will also alleviate the time, cost, and overhead of continually needing to redeploy your application as you onboard new devices.
Questions?