beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice
Overview Businesses have long managed and analyzed data and have earned benefits. We come across a lot of complexity when it comes to data. Traditional datawarehousing and business intelligence deal with data that is structured and stored in a traditional relational database while other data, including documents, customer service records, pictures, videos and sensor data from machines and new information sources such as data from social media and click stream data form the new and ubiquitous source of data which is semi-structured and unstructured. The challenge is how to make sense of the intersection of these different types of data. We always had a lot of data, the difference today is that significant more of it exists and is growing and it varies in type and timeliness than ever before. Hence data needs to be managed differently. Therein lies the opportunity and challenge of Big. The emphasis is on highly advanced analytical (predictive modeling) and visualization tools (Geospatial mapping, heatmap) thereby greatly increasing the value derived. Our Big practice offers consulting, project management and systems integration services across the Big stack just described. As new and ubiquitous sources of data such as documents, customer service records, pictures, videos and sensor data that are either unstructured or semi-structured are being thrown out with ever growing volumes and velocity, never greater has been the challenge in both managing it as well as making sense of it. BIGDATA Rx is Technosoft s Big practice that is committed to working with our customers in this new and exciting field. It s your prescription for your Big headaches.
IT Services Consulting, Project management and System Integration Big Use End-user applications Big Analytics Visualization tools Analytical tools Big management systems Traditional BI suites and online analytical processing Basic visualization applications Traditional analytics (BI, data mining) Traditional relational database Conventional file system BI/Traditional Analytics - Big Analytics Advanced analytics (Predictive modeling, modeling & optimization) NoSQL database e-commerce, search and social gaming Advanced visualization applications - Geospatial Mapping, Dendograms, Graphs Heatmaps, Parallel Coordinates etc. In-memory computing modeling, integration and processing, storage and management Hadoop Hadoop Distributed File System (HDFS) Telematics, Predictive Asset Maintenance MapReduce programs Stream mining Complex event processing Parallel relational database Streaming data management (Storm / Apache S4) Difference between traditional and Big technology stack Traditional applications were primarily limited to data visualization, while insights from Big analytics are inputs for data-driven end-user applications. Big analytics requires advanced analytic tools on statistics, algorithms, etc. Visualization tools required are beyond charts and graphs like clustergram, dashboards, etc. Big storage and database management systems needs to be highly scalable and connected to low latency access mechanisms. Environment suitable to handle unstructured data from multiple sources. Requires Massively Parallel Processing (MPP) and NoSQL querying tools against SQL for traditional database. Our View of the Big Stack
Big Services and Capabilities Need Services Description Application areas* Our Practice Store large quantities of unstructured data Architecture Big architecture, RDBMS. Website click streams Tweets and Facebook likes Sensor data Emails In setting up BIGDATA Rx, the Big practice at Our 200 strong team consists of scientists with Technosoft, we were cognizant of the 4 main questions that repeatedly cropped up in our discussions with customers and executives alike 1. How to store large quantities of unstructured data? 2. How to achieve faster data access, storage and analysis? 3. How to accomplish real-time analysis of high volumes of data? deep background in machine learning applications across various sectors, skilled programmers in languages like R, Python and Java and business analysts with deep domain knowledge across multiple verticals. Our diverse capability is spread across Mining (40%), Integration (30%), Visualization (20%) and Architecture (10%). This ensures that the team is best equipped to help Faster data access, storage and analysis Real-time analysis of high volumes of data Integration Mining Big integration services, migration and management. Text mining, Taxonomy, Classification, Dynamic Neural Networks, Time Series Modeling, Event Sequence Analysis etc. Real-time embedded systems Algorithmic trading E-commerce Social networking Risk management Customer intelligence Revenue optimization Assortment Merchandise planning 4. How to respond to issues instantly through the power of actionable insights from analytics? you overcome these and other challenges through the application of Big concepts and technologies for a variety of industry applications. Gain actionable insights from analytics and respond to issues instantly Visualization QlikView, Tableau, SpotFire, Shiny R, D3 JS etc. Energy management SEO optimization Real-time traffic congestion detection using GPS data *Relevant industry use cases
Our Big Solution Architecture The following visual is a representation of how Technosoft believes Big systems can and should be architected in a complex network where multiple connected devices and sub-systems (power systems equipment, vehicle information systems and geographical information systems to name a few and which are either mechanical, electronic or computer) are emitting high volumes of unstructured data at very high velocities and are of wide varieties. The characteristics of complex networks that lend itself well to Big Analytics are: 1. Multiple subsystems, for example, routers, switches, cables, software 2. Each sub-system having characteristic nonlinear dynamical behavior 3. Sub-systems interact nonlinearly leading to intractable emergent phenomena 4. Dynamic Neural Networks can be used to predict behaviors of such networks Complex networks show similar properties and common behavior independent of technology and include Smart Grids, Internet, Computer Networks, Telecom Networks, Transportation Systems, Oil and Gas pipelines and Storage Area Networks (SAN) to name a few. 7
BATCH PROCESSING ENGINE Complex networks of connected Devices emit high-velocity streams Network Device Usage Service Universal Format XML CSV EDI LOG Objects SQL Text JSON Binary bases File systems Create Map Reduce Environment HADOOP CLUSTER EVENT PROCESSING STREAM PROCESSING ENGINE Warehouse Predictive analytics Business intelligence Rule Base Visualization Layer The data generated typically is network data, device data, usage data and service data. is also captured from secondary sources like social media (Twitter for instance). This streaming and non-streaming data is then processed into three different types of storage mechanism: that requires batch processing is moved into the Hadoop cluster that is of high velocity and generated rapidly is moved into the stream processing engine where for example Storm is used to take action And the traditional data warehouse is used to store data that is neither high volume nor high velocity Subsequently analytics is performed on the stored data. Rule based analytics is performed on high velocity data in which the data streams are compared with the rule base and action taken based on the program logic. On the other hand predictive analytics is performed on high volume data wherein algorithms are applied to discover patterns and take resulting action. Offline, batch & stream Processing must work together Lastly the data is then visualized in the forms of bars, charts or dashboards for actionable insights. On diverse data in motion Social Media STORM TOPOLOGY
Analyzing in Motion We can help A Predictive Asset Maintenance framework that minimizes equipment downtime, improves productivity, reduces service costs and enhances customer satisfaction Complex System A Big solution architecture for the era of complex networks of connected devices. Helps analyze data-in-motion and delivers actionable insights Predictive Modeling on streaming, unstructured data in motion from complex network of connected devices Scalable stream processing system Actionable visualizations and dashboards Multiple sub-systems Sub-system has characteristic nonlinear dynamical behaviour Sub-systems interact nonlinearly leading to intractable emergent phenomena Dynamic Neural Networks can be used to predict behaviours of such networks In summary our Big Solution Architecture helps in from Complex Networks of Connected Devices Analysis of Unstructured in Event Streams Near-Real-Time Analysis of -in-motion Visualization of Analytical Results in Dashboard Why Technosoft Investments in service products like Predictive Asset Maintenance framework A unique Big solution architecture Cross-functional teams that comprise scientists, Visualization experts, Technology & tool experts and domain specialists Partner ecosystem Spotfire, Microstrategy, Mirror 42, Tableau and the Open Source World
beyond possible Technosoft Corporation is an IT and BPO services provider with headquarters in Southfield, MI, USA and delivery centers in India. We provide information technology, business process outsourcing and consulting services to companies in North America, Australia and New Zealand and Asia-Pacific Regions. As a privately owned company we answer to only two constituencies - our customers and our employees. Our customers rely on us to provide services and solutions that leverage our industry and domain expertise combined with our technology prowess, delivery focus and quality. Our collaborative culture and work environment helps attract and retain exceptional talent which is a key ingredient of our sustained growth. See how Technosoft can go Beyond Possible for your organizational needs. wecanhelp@technosoftcorp.com www.technosoftcorp.com Your Prescription for Big Headaches BIGDATA Rx is Technosoft s Big practice consisting of scientists and skilled programmers in lanugages like R, Python and Java offering 4-pillar services: data mining, integration, visualization and architecure. They are specialists in helping companies manage both the challenges and opportunities associated with unstructured data - volumes, velocities and variety. BIGDATA Rx is Technosoft s prescription for your Big headaches. Copyright 2013, Technosoft. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Technosoft. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.