Embedded BI: Agents Unleashed Rajat Ghosh Philipose Mathew A Patni White Paper
COPYRIGHT Copyright Patni Computer Systems Ltd. All Rights Reserved. October 2004 Restricted Rights This document may not, in whole or in part, be copied photocopied, reproduced, translated, or reduced to any electronic medium or machine readable form without prior consent, in writing, from Patni Computer Systems Ltd. Information in this document is subject to change without notice and does not represent a commitment on the part of Patni. This document is provided "as is" without warranty of any kind including without limitation, any warranty of merchantability or fitness for a particular purpose. Further, Patni does not warrant, guarantee, or make any representations regarding the use, or the results of the use, of the written material in terms of correctness, accuracy, reliability, or otherwise. All other brand and product names are trademarks of their respective companies. Patni Computer Systems Limited India North America UK & Europe Japan Akruti, MIDC Cross Road No.21 Andheri (E), Mumbai 400 093 Tel: +91 22 5693 0205 Fax: +91 22 5693 0211 238 Main Street Cambridge MA 02142 Tel: +1 617-354-7424 Fax: +1 617-876-4711 Vistacentre, 50 Salisbury Road Hounslow, Middlesex, UK. TW4 6JQ Tel: +44 20 8538 0120 Fax: +44 20 8538 0276 4 th floor, Aoyagi Building, Chuo 5-39-11, Nakano-ku, Tokyo 164-0011 Tel: +81 3 53281952 Fax: +81 3 53281951
Table of Contents Overview...4 What is Embedded BI?...4 What are Agents?...5 Features...6 Inside the Agent world...6 Classification of Agents...7 Agent Standards...7 Agent Systems...8 Usage of Agents...8 Business Application of Agents...8 Business Process Management...8 Supply Chain Management...9 Manufacturing...9 Agent Impact on BI...9 Real-Time BI...10 Process Management...10 Data Integration...11 OLAP & Reporting...11 Data Mining...11 Future of EBI...12 References...12 About the Authors...13 About Patni...13 Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 3
OVERVIEW BI needs of many sectors are not completely addressed by current BI vendors, as they do not provide the information to act on the changing business needs in real time. The progress made in Agent technology has the potential to take BI considerably away from the data warehouse centric models and drive it towards a more real time and responsive Embedded Business Intelligence (EBI) model. Embedded BI provides businesses with the intelligence to make right decisions at the right time; not later to act upon. Agent Technology, which inspires EBI, is a new generation software, which is able to manage in real time, the complexities of modern businesses. These autonomous intelligent agents are distributed right throughout the business processes, acting both proactively and responsively to business situations. Agents have the intelligence to gather information, reason it out, and evaluate it and the authority and autonomy to act independently. Thus it can watch systems, consider new information, monitor changes, learn from previous results and trigger actions that will bring it closer to achieving its objectives. If EBI, becomes a reality, it would mean lesser number of business portals, as the BI agents take decisions in real time and reduce the Data warehousing needs. EBI is at its early stages and the fast emerging Business Process Management(BPM) needs point to its fast realization. In this paper we will discuss Agent technology, its applications and its impact on BI. WHAT IS EMBEDDED BI? BI from an IT standpoint, involves providing businesses better decision making capability using historical business data spread across disparate data sources. This decision-making may happen weeks later after it has impacted the business considerably. Sectors like Supply chain and Manufacturing need instant BI rather than reacting to the events affecting their business after they have occurred. For example, delay of trucks in delivering shipments due to unexpected traffic delays may result in loss of customers. A mechanism to obtain the traffic delay information in real time, getting advised on possible alternatives and rerouting the truck definitely helps the business to retain its customer. Service oriented businesses have been clamoring for faster and possibly a real time oriented decision support system. Business process management requires systems that are intelligent and support real time decision-making. Embedded BI (EBI) has been introduced into the BI landscape as a future technological revolution, which can provide machine based intelligence and better network and data management capabilities. This could be a catalyst for the current convergence initiatives of BPM and BI. EBI is the intelligence available to an application or process for its best possible execution in complex and unpredictable environments. The process intelligence is provided by intelligent software agents that have been provided with the business process know how and rules. In the truck Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 4
example above, the agent deployed to handle the routing gets updated in real time about the traffic as it interacts with other agents on the road and also traffic-updating agents. Based on the available information, the agent suggests the best alternative route. In real life, Agents are part of our everyday life. Sales agents, insurance agents and agents of professional athletes & stars, represent people who offer us services in the best possible way. They are entrusted to work for someone as they have the expertise and intelligence. Likewise software agents have the intelligence to represent and act autonomously across systems to achieve defined objectives. Off late, BI vendors have been marketing their product APIs as embedded BI where the BI reports can be invoked from applications using the JAVA or.net interfaces. The EBI we discuss here is agent based BI and not about embedding callable BI reports/services onto operational applications. WHAT ARE AGENTS? As per Professor Nicholas Jennings who helped pioneer the use of agent-based techniques for realworld applications, a Software Agent is a computer system situated in some environment capable of autonomous action to meet its design objectives in this environment. An agent has the capability to analyse a situation, make decisions, communicate and also negotiate with other agents and then convey outcomes to the system and its users. The central idea underlying agents is that of delegation. The owner or user of an agent delegates a task to the agent and the agent autonomously performs the task on behalf of the user. Shopping bots like MySimon.com, which allow users to compare the latest prices of products from Web Retailers, use agents to collect information. Individual Mobile Agents are simultaneously sent from the Shopping Bot Agent host server to interact with the Merchant Agents or directly with the merchant databases to obtain the latest price, discounts and quantity available. Instead of querying multiple merchant databases over the network, agents go over to the merchant side, get information, return and consolidate the information from all agents and provide the best deals to the shopper. Figure 1. Agent assisted shop bot Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 5
FEATURES An Agent has Autonomy, the ability to act without human intervention. It has the ability to halt itself, ship itself to another computer on the network, and continue execution in the new computer. In the new computer the agent does not restart its execution from the beginning whereas it continues from where it left off. Agents can communicate and interact with each other through communication networks. Mobile agents may also travel from one place to another place to collect and process information and send the results back. These mobile agents provide an innovative concept for creating distributed systems -- they bring the computation to the data rather than the data to the computation. They can reduce network load, overcome network latency, encapsulate protocols, work autonomously and asynchronously and adapt dynamically. The most intelligent agents will be able to learn, and will be able to adapt to their environment, in terms of user requests and the resources available to the agent. The key aspects of intelligent agents are their autonomy, their ability to perceive, reason and act in their surrounding environments, as well as the capability to cooperate with other agents to solve complex problems. INSIDE THE AGENT WORLD Work on Distributed Artificial Intelligence in the 1970s pioneered the work on software agent based multi agent systems. A basic agent structure includes a life-cycle model to create, destroy, start, suspend and stop agents, a computational model that provides the agent the computational capabilities, a security model to access other systems and also to interact with other agents securely and a communication model to interact with applications & other agents. A common standardized infrastructure is required for agents to interact with host systems and other agents and also for safeguarding the agents. The heart of the agent infrastructure is an agent server that will run on all computers that hosts agents. The server hosts agents, manages the communication among agents, authenticates and supervise agents and also transports agents to other systems. Agent servers share information among each other and provide information to agents regarding other agent servers. A dedicated runtime environment interfaces between the agent and its host and provides the agent with the required resources from the host system. Applications or users interact with the agent framework using a Client. Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 6
Figure 2. Agent Architecture Agent Aglet An Aglet is a Javabased autonomous mobile agent that can move from one host on the network to another. An aglet requires a host Java application, running on the host computer before it can run on the host computer. When aglets travel across a network, they migrate from one aglet host to another. Each aglet host installs a security manager to enforce restrictions on the activities of non trusted aglets. These agents are able to perform in this way owing to their ability to communicate with a database based on ontology. Ontology is a computer-readable description of knowledge about the resources in an enterprise's network. It relates attributes to different classes of objects, such as available business resources, projects, customer orders or plans, to enable knowledge of available resources to be built up in ontology. The software agents become intelligent because they can make use of the knowledge contained in ontology to use in the process of negotiation and decision-making. CLASSIFICATION OF AGENTS Agents are mainly classified as static or mobile agents based on their mobility, deliberative or reactive agents based on their response to events and autonomous, learning and collaborative agents based on their characteristics. Autonomous Mobile agents are software programs that can be dispatched from one computer and transported to a remote computer for execution. Arriving at the remote computer, they present their credentials and obtain access to local services and data. Mobile agents in addition to the basic agent model have a navigation model. The principle of the mobile agent approach is that a local method call is faster than a remote procedure call. This is due to the fact that remote procedure calls have to handle the arguments passed to and the value returned from the remote procedure. Additionally, each remote procedure call suffers from the latency of the underlying network. AGENT STANDARDS Agent standards and specifications are evolving. The Foundation for Intelligent Physical Agents (FIPA) develops the specifications for interoperation of heterogeneous software agents. Specifications on Agent Applications, Architecture, Communication, Agent management and Agent Message transport are being developed. Research is also being done on suitable Agent programming and communication languages. Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 7
AGENT SYSTEMS Agent systems like Aglets by IBM, Grasshopper by IKV, and Concordia by Mitsubishi have been developed in the last few years. An Agent System is a platform that can create, interpret, execute, transfer, and terminate agents. When a mobile agent transfers itself, the agent travels between execution environments called places. A place is a context within an agent system that provides a uniform environment in which an agent can execute. It provides the means for managing mobile agents, enforcing security policies and accessing local resources. USAGE OF AGENTS User Assistant agents: They provide information or advice to users e.g. Microsoft Office tools provide animated agents to help users. System Management agents: They assist in system and network management tasks like load balancing, failure anticipation, problem analysis and information synthesis. Decision support agents: Decision support agents are mostly deployed in closed environments, utility companies and military organizations. These agents are used for information synthesis and decision support. These systems may alert an operator of a potential problem in the system, provide information in support of a complex decision. Interest matching agents: These are probably the most extensively used agents types. They are used on the internet sites to recommend similar products of interest to those purchased by consumers. These agents observe patterns of interest and usage in order to make recommendations. Organizational structure agents: These agents are structured to operate in a similar manner as human organizations. For example, multi agent supply chain systems would have agents playing the roles of buyers, suppliers, brokers, stock, orders, line items and manufacturing cells. Operations systems would have resource agents, material agents, process agents and so on. BUSINESS APPLICATION OF AGENTS Agents can streamline business processes in a more flexible and robust manner and agents representing entities can also negotiate for services from each other. Agent applications are more pronounced in business process management landscape where each agent represents a process and interacts among other process agents based on business goals. Supply Chain Management (SCM) operations, Manufacturing and Network Management domains also have wide agent applications. Business Process Management There are a huge number of processes that are executed in an enterprise and managing them efficiently is a priority. Project ADEPT (Jennings et al., 1996) views a business process as a community of negotiating and service providing agents. Each agent represents a distinct role or department in the enterprise and is capable of providing one or more services. For example, a design department may provide the service of designing a telecom network, a legal department may offer the service of checking that the design is legal, and the marketing department may provide the service of pricing the design. Agents who require a service from another agent enter into a negotiation for that service to obtain a mutually acceptable price, time, and degree of quality. Successful negotiations result in binding agreements between agents. The proactive nature of the Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 8
agent means services can be scheduled in a just-in-time fashion (rather than pre-specified from the beginning) and the responsive nature of the agents means that service exceptions can be detected and handled in a flexible manner. Supply Chain Management In Supply chains, agents are modeled to have their own attributes. A warehouse with its constraints like capacity, product level, out of stock units etc could be an agent in the supply chain network. In a multi-agent system, each agent communicates with the network of agents, negotiating on such constraints as quality, price and time and then making decisions for committing resources to match demand. In such situations, agent technology tries to model the real world, including conflicting requirements and constraints, to provide the best possible match of resources with demand. Logistics technology is now at a stage when it can provide tangible benefits in this kind of scenario, helping drivers find alternative routes, suggesting the most cost-effective times of day to travel and allocating resources to where they are most needed. Manufacturing AGENT IMPACT ON BI In manufacturing, the enterprise is structured as a hierarchy of work units. For example, There will be units for milling, lathing, grinding, painting, and so on. These work units will be further grouped into flexible manufacturing systems, each of which will provide a functionality such as assembly, paint spraying, buffering of products, and so on. A collection of such systems is grouped into a factory. A single company or organization may have many different factories, though these factories may duplicate functionality and capabilities. The goal of an agent system is to efficiently manage the production process at these plants. This process is defined by some constantly changing parameters, such as the products to be manufactured, available resources, time constraints, and so on. In order to achieve this enormously complex task, a multi-agent approach is adopted, where each factory and factory component is represented as an agent. Each agent has a collection of plans, representing its capabilities. What happens when EBI is enabled extensively in manufacturing operations, SCM, network services and Customer Management? When BI needs become less? Would data warehouses be required? Also how will agent technology in turn affect the traditional BI software processes like ETL, OLAP etc.? Agent programs representing the components of a process, act upon situations in the best possible way based on business rules thereby executing processes first time right itself. Defects and inefficiencies will reduce to lower levels. Agents ensure that processes execute within the SLAs. Processes and processes, which cannot execute within the SLAs are escalated to the business in real time as compared to the current escalation method. Current BI framework caters to the strategic and operational BI needs of the business. Data from multiple operational sources are integrated into the data warehouse and various analysis and reporting tools work on this data warehouse to provide the strategic and operational BI. Operational BI provides businesses the intelligence to carry out effective process management, identify defects earlier and improve upon product quality, ensure machine efficiency and timely maintenance and Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 9
Application Area Futuristic agent enabled shop floors will have better real time intelligence, be proactive in nature as they ensure that the machines and their components remain healthy and efficient. Each machine would have multiple agents responsible for monitoring the process efficiency, diagnosing components, alerting about part replacements in advance and also interacting with the agent at the supply store ordering the replacement part. also better resource management. Strategic BI includes predicting future sales or comparing revenues of current year with the past years in different sales regions. We do see EBI making a major impact in the operational BI space but quite less on Strategic BI. As Agents with business rules are deployed to carry out the processes, operations in an enterprise will see more efficiency and improved quality thereby reducing the need for operational BI. Data warehouses and reporting portals deployed for operational BI will become less relevant. Real-Time BI Smart agent programs are embedded within the components of the business. Agents that are part of the operations ensure that the processes run within defined goals. When the processes are not efficient, the concerned agent communicates with the report agent resulting in an instant alert and reports the user about the issue along with recommended actions. Agent would interact with operational data stores and data warehouses effectively performing real time predictive analysis and business activity monitoring i. The reporting agents will invoke the Web Services of Reporting and Analysis tools to provide detailed graphical reports. Process Management Agent based modeling of business applications will be able to isolate inefficient processes. In this method, an ideal model of the business is developed and monitored and then constraints are introduced as encountered in day-to-day business. Agents represent physical components of the system. In the case of a supply chain it could be trucks, consumers, suppliers, etc. Behavior of the agent system is initially monitored under ideal conditions and thereafter behavior when business constraints are introduced are also studied. Events responsible for inefficiencies are singled out for improvement. Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 10
Data Integration A body shop passes the invoice details of a vehicle repair using its agent system to the Fleet Company s agent rather than sending across a weekly invoice file. Once the body shop system does the invoice, its agent interacts with the Fleet agent, which calls a web service to update the database. Thus, the agents will perform the data integration between interfacing processes on the fly. Data warehouses are currently developed across organizations to analyse integrated operational data for better decision-making. The intelligence that the agents gather in course of time helps in taking quick and accurate decisions in real time reducing the need for warehousing the data and acting on it later. This would mean a shift towards real time network based BI. Sectors implementing EBI will have reduced demand for data integration. Extensive adoption of EBI within a business, its partners and suppliers would reduce the need for traditional data warehouses as agents take the right decisions based on their intelligence. OLAP & Reporting EBI enabled OLAP tools would help users spend less time analyzing the data. Once the user feeds the interface agent with the business rules and goals, the agent can interact with the data warehouse agent, which can query the Data warehouse and provide the information. The reporting agent picks up the data from the Data warehouse agent and provides the user with the most detailed and accurate business picture. This would also mean that future BI reporting and analysis vendors might have to introduce agents as primary interface to users. The analysis tools could be of less relevance to users. They may need agents to do the analysis and provide the findings to users. Reporting agents of users can interact with each other and share already created reports rather than query the data warehouse for the same report. Agent metrics dashboards will be a new addition to the reporting portals to analyse the performance of agents. Data Mining Data mining uncovers hidden trends and patterns from data and caters to the BI for organizations to uncover potential risks and opportunities. Gathering credit risk patterns by datamining credit history information spanning across multiple databases is a technological challenge today. EBI provides distributed datamining the much needed processing power courtesy, the Mobile Datamining Agents. Datamining Agents can traverse multiple databases, query and return with identified potential risks. Network and processing power requirements are considerably less as the mining happens on individual databases. In addition to this agents can also assist in text and unstructured data mining. Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 11
FUTURE OF EBI EBI definitely has the features to provide a new dimension to BI in specific domains. Emergence of EBI as a top line solution will take considerable time as agent standards and specifications are still evolving. Agent frameworks across systems should be interoperable and secure. For businesses to invest in agent infrastructure the ROI and data security would be the key factors. Agents move across networks with business information and businesses would need to see secure agent frameworks. Security is a major concern in the agent framework and is one of the areas where lot of work is being carried out. How agents will guard privacy as they hop from system to system is being worked upon. Research is still on to develop full proof agent systems that will protect agents from being tampered by hackers and affected by viruses. Making each and every component of a business process, part of an agent system is easy to talk about. But real life agentification of processes is laborious, expensive and complex and businesses are a long way from it. The first step in going about EBI would be prototyping a slice of the business process to understand the intricacies involved in the agentification. Business processes need to be modeled using standard modeling language, suitable agent platform identified, meta models needed to support the agent specification designed and interoperability among the agents ensured. Visualization, debugging, tool based simulation, testing and validation should also be carried out. As per Forrester research, Store level maintenance and predictive maintenance application domains could see agent-based automation by 2008. Also real-world-aware-agent applications by 2009 are possible that will help automate business processes. Advances in technologies like web services, wireless networks, RFID and bluetooth would further assist in the realization EBI framework. REFERENCES ADEPT: Managing Business Processes using Intelligent Agents Jennings et al., 1996 Application of Intelligent Agents Jennings et al., 1998 Software Agents in Business: STEADY adoption Curve Forrester Research Material available on FIPA site www.fipa.org Agent Technology: Enabling Next Generation Computing AgentLink Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 12
ABOUT THE AUTHORS Rajat Ghosh is a data warehouse architect with over six years experience on variety of business intelligence applications on wide-ranging hardware / software platforms across industry verticals. He has delivered innovative BI solutions to Patni s global customers and been appreciated for technical leadership at numerous instances. His technical expertise goes beyond traditional BI areas including Enterprise Application Integration. Philipose Mathew is a BI technical specialist who has been deploying BI solutions to customers for nearly three years. He is part of the Technology Focus Group of Patni Business Intelligence COE and evaluates new BI technologies and products. Philip brings with him his prior experiences in developing and maintaining business applications in Leasing, Quality and Manufacturing verticals over multiple technologies. ABOUT PATNI Patni Computer Systems Limited is a global IT Services provider servicing Global 2000 clients in the Manufacturing, Insurance, Banking & Financial Services, Telecom, Retail, and Energy & Utilities, Logistics & Transportation, Media & Entertainment industries. With an employee strength of over 9,000; multiple offshore development facilities across eight cities; and 24 international offices across the Americas, Europe and Asia-Pacific; Patni has registered revenues in excess of US $250 million for the year 2003. Patni's technology focus spans e-business solutions, enterprise applications, embedded technology solutions and enterprise systems management. Its service offerings include application development and reengineering, application management and business process outsourcing. Committed to quality, Patni adds value to its client's businesses through wellestablished and structured methodologies, tools and techniques. Patni is an ISO 9001:2000 certified and SEI-CMMi Level 5 organization, assessed enterprise wide at P-CMM Level 3. In keeping with its focus on continuous process improvements, Patni adopts Six Sigma practices as an integral part of its quality and process frameworks. Patni is one of the Top 6 Indian-based companies who operate optimized global delivery models for customers, fronted by a strong local consulting capability. As industry leaders, Patni introduced Offshore Development Centers, and pioneered 'follow the sun' development and support frameworks. Patni s Business Intelligence Center of Excellence Patni has been delivering Business Intelligence (BI) and Data Warehousing (DW) solutions to Fortune 1000 global organizations over the last 6 years. Our service offerings range from Consulting Services, Customized BI/DW Development to Maintenance. Our solution has a foundation of deep domain knowledge, technological expertise, proven implementation frameworks and effective knowledge management. We have formed alliances to enable faster implementations and fully exploit product features. Our comprehensive solutions have delivered tangible benefits to our customers across industry verticals. Copyright Patni Computer Systems Ltd., 2004. All rights reserved. 13