PIE. Internal Structure

Transcription

1 PIE Internal Structure

2 PIE Composition PIE (Processware Integration Environment) is a set of programs for integration of heterogeneous applications. The final set depends on the purposes of a solution under consideration. PIE may have two basic sets: standalone full featured application server and built-in component into another application. The second one has some advantages and some limitations in comparison with the first one and it depends on core implementation. PIE as an Application Server PIE consists of: PIE Server application server PIE Studio editor for graphic description of business processes PIE Adapters a set of intermediary programs for applications connection PIE Control a set of PIE programs running on well-known operating systems: Windows, HP UX, AIX, Sun Solaris PIE Docs set of documents for development of integration solutions on PIE platform. Adapters for widespread systems interconnection are supplied together with applications server: FileAdapter provides communication of external systems with PIE using file exchange ORAAdapter provides PIE access to Oracle DBMS ODBCAdapter provides PIE access to any database using ODBC drivers FTPAdapter provides PIE access to data via FTP protocol NamedPipeAdapter provides PIE data exchange using Named Pipes XOWALAdapter provides PIE data exchange using XOWAL communication libraries of the CMA company CORBAAdapter provides data exchange with PIE using CORBA functional call protocol that provides possibilities of connection third party adapters to PIE. In addition to above mentioned adapters CMA Company has developed XAdapters adapters for interaction with external systems such as WebSphere MQ Series, Microsoft MQ, SWIFT, MICEX, RTS, Reuters, Bloomberg, etc. PIE has its own proprietary protocol of guaranteed message delivery implemented using the XOWAL program components for internal and external adapters communication. It s worth mention that PIE tools are mainly designed for software engineers. To create such PIE components as business tasks, data adapters and other internal components one should know at least С/C++ and one of Java, JavaScript or both languages. Python, Perl, Ruby languages are to be considered as extensions for the future. The choice of programming languages for development of program components in PIE environment is determined by the fact that PIE is based on XPCOM component object model which provides cross-platform for applications development. PIE Service-Oriented Architecture PIE uses a set of the best practices and methods known as Service Oriented Architecture (SOA) and Service Oriented Business Applications (SOBA) for developing integration solutions. External applications are presented in PIE as a set of services which communicate with PIE Application Server via Enterprise Service Bus (ESB). PIE ESB is a program infrastructure representing a message handler and internal adapters. The adapters solve the problems of external applications connection, routing, format conversion and thus conceive all the difficulties of external application adaptation by presenting it as a service with standard set of properties. ESB provides such important functions for the system activity as message correlation, service of guaranteed message delivery, data transmission security etc. Picture 5 presents PIE architecture for integration solutions. The core of the integrated systems on the basis of PIE is the application server which consists of: Dispatcher Internal Adapters for communication with external applications Business Rules Router (BRR). The Dispatcher provides ESB functionality and does message exchange via internal adapters both between services and the BRR as well as between the application services directly. Dispatcher also handles execution of PIE procedures for initialization and shut-down. BRR creates, executes and deletes business processes instances called tokens. It provides a separate execution thread for each token and does the business process execution which includes task invocation of the business process.

3 Pic. 5. Service-Oriented Architecture of integration solution on PIE platform Token a business process instance which is created on the basis of the existing XML-descriptions from the repository. Tokens are executed in BRR s addressing space.tokens are data structures containing information related to the execution of the corresponding business processes. Task a program component that performs the business logic of a particular step of a business process. Tasks are the basic elements from which business process description is made of. Task may contain any program code which is usually small and can be limited to an activity with memory variables. But mainly tasks are intended for sending message to invoke external applications (external services) or internal functional components (internal services). Components with expanded functionality allow PIE application server to connect modules with additional functionality (internal services). For example the activity with global business processes variables environment variables is done in this way. The methods of the components can be invoked from the tasks and internal adapters. External applications interact with each other through PIE by means of message exchange. The order of interaction for applications and internal components is determined by so called frames. Frame is a structure of a business process which description is stored in the repository as an XML subtree. The methods of the components can be invoked from the tasks and internal adapters. External applications interact with each other through PIE by means of message exchange. The order of interaction for applications and internal components is determined by so called frames. Frame is a structure of a business process which description is stored in the repository as an XML subtree. When PIE server receives a message that requests some business process execution it searches the repository for a corresponding business process description and downloads it to the RAM. As a result a token of the process is created and then executed. During this execution the tasks contained in the frame process are invoked according to the UML-diagram structure. The tasks in their turn can invoke either internal program components performing certain business logic or sending/receiving messages via ESB for activation of application services.

4 External Application Adapters Essentially the integrated applications can communicate with PIE directly. This is the case for new applications. In real life for communication with PIE you will often require intermediary software known as adapters. The main task of the adapters is to abstract as much as possible the PIE core from the peculiarities of external applications interfaces implementation. Adapters provide reconciliation of the information formats transferred between external applications and the PIE server. Adapters are intended to initiate applications activities on the basis of the messages received via PIE and to send the results of the PIE work in the form of messages. In general the adapter together with the integrated application is defined as service according SOA concept. Adapters can be internal i.e. they work directly in the addressing space of the PIE dispatcher and external when they function as independent applications. Internal adapters communicate with PIE directly via program interfaces (API). These adapters function in the same process with PIE core cause they are PIE components. External adapters are independent applications that may function and may not function in the same process with the applications being integrated (for PIE it does not matter). External adapters communicate with external applications as well as with PIE internal adapters. Several external adapters can be connected to the same internal adapter. Internal adapters communicate on the one hand with services, i.e. applications being integrated, directly or through external adapters. On the other hand (from the PIE s dispatcher side) internal adapters are suppliers and users of the PIE messages. Applications Integration Approach The main idea of PIE is a support of business applications information communication without going deeply in applications nature as well as a separation PIE core from operating systems, hardware platforms, communication protocols and DBMS. PIE offers an approach which allows to present application interfaces in accordance with business processes tasks and to construct new logical applications on this basis. Thus on the logical level the applications may differ from the original physical applications. The basic concept for PIE application integration is a decomposition of business-processes into tasks (jobs) and integrated applications into functions (services) that perform corresponding tasks (Pic.6). PIE automates application services invocation in line with business logic that makes it possible creation of composite applications with functionality much wider than just a sum of the functionalities of the original applications. Business processes are described with the help of business rules. Business rules determine what should be done on each step of the business process. Business rules are activated according to business process states. Thus business rules in PIE correspond to tasks which are the base for business process descriptions that determine the order of services invocation by applications. It is important to note that after decomposition of business processes and business applications it is necessary to carry out the analysis and generalization of the decomposed tasks in order to find similar tasks executed in different business processes and corresponding services. When all the necessary tasks of the business processes and corresponding application services are defined then they are to be described in the repository by means of PIE Studio. The descriptions of business processes, tasks, services and means of their interaction are put into unified repository which is the only place where descriptions of business tasks and interfaces of interactions with applications are stored. Virtual Applications Development According to the PIE Technology: On the logical level company applications represent business logic of the company; The company s business logic is a set of business processes that execute this business logic; Tasks correspond to business rules. Such an approach allows to develop so called virtual applications (Pic.7), generated from the tasks implemented by real applications running on different physical servers

5 Pic.6 Business processes decomposition into tasks and business applications decomposition into service PIE s virtual application is a set of interdependent tasks performing the company s business logic. It can use tasks belonging to different applications. Thus virtual applications exist and are executed by means of interpretation of business processes descriptions from the system s repository. Such technology supports the high level of tasks reuse and even business processes. If it is necessary to add a new business process then we can write only a program code performing the tasks or invoking the services that execute new business rules. The already existing tasks can be used without change. Thus PIE does not only provide information exchange between applications but also creates a synergistic effect by providing the functioning of the virtual business application that executes the new integrated business logic. Integration of Legacy and Off the Shelf Applications PIE uses two different strategies for processing of external (off the shelf) and internal (self-developed) applications. These strategies were developed due to the fact that all details of self-developed applications are usually known including internal logic and source codes. As for external applications only the functions that they are able to execute are usually known. Hence different approaches to fish out maximum possibilities from the applications of these two types should be used. External applications are treated as encapsulated ones. Moreover old (legacy) internal applications should be also treated as encapsulated because source data may be lost and its developers (programmers who engineered these applications) may be not available any more. The only thing which is known about these applications is the set of the tasks (functionality) they perform.

6 Pic.7 Business-processes decomposition into business rules and tasks PIE makes it possible to generate new business processes from the set of business processes which includes functional possibilities of external and internal applications. Thus using the concept of the business process PIE eliminates the differences in the nature of external and internal applications in such a way that they can work together as a single mechanism. Applications Integration Scenario The scenario for developing PIE virtual application by integration of the existing applications includes several stages. 1. Decomposition of each application into services executing business tasks. This means that the functionality of the application interface that will be used to connect this application to PIE should be described according to the PIE s rules. 2. Development of UML-models of business processes. Find out what tasks should be developed and integrated into PIE. 3. Registration of each application as a service in the repository. 4. Customization of existing and development of a new adapters (if needed) for each application. 5. Description of XML-messages formats and registration them in the repository. 6. Registration of global environment variables in the repository. 7. Registration of all tasks in the repository and their programming. 8. Creation of formal schemes for business processes ( business frames ) with the help of the special graphic editor and their registration in the repository.

7 Pic.8 Activity diagram example Visual Presentation of Business Processes Business process is the PIE key element. A business process is created when dispatcher receives a request from the service that corresponds to this business process. Business process is displayed as an activity diagram (UML) where activities correspond to tasks and transitions correspond to possible results of tasks execution. Such an activity diagram provides the user with the possibility to simulate business processes with the help of a simple and intuitive graphic editor which is a part of PieStudio Workstation. An example of an activity diagram is shown on Pic.8. As we have already mentioned PIE supports component approach. Each task is implemented as a component which is used many times later during business processes execution. Tasks descriptions are put in the corresponding section of the repository. Transitions are made according to possible task output. The outputs depend both on the state of particular business process and the state of integration bus in general. Business process state is defined depending on local data stored in it. The state of the ESB depends on the values of environment parameters. Environment parameters description is stored in the particular section of the repository. Examples of environment parameters: number of active users, EOD/BOD indicator, possibility to grant a credit etc. PIE Repository Objects The repository is used in PIE as warehouse of metadata describing business processes logic, i.e. the order of tasks implemented by the applications execution, format of messages used for data exchange between applications, features and methods of services used for making integration application. Business processes descriptions are stored in the repository and are activated from there. Each time business logic requires to execute a process, a special program tool business rule router invokes it from the repository. In the repository are stored the following XML-descriptions of the basic entities of PIE: Frames description of structured business processes; Tasks description of the components functionalities implementing the business logic; Services description of information interaction with external applications interfaces; Adapters description of program components used for data exchange with external applications; Components description of additional program modules enhancing PIE basic functionality; Messages description of types of messages used by the system for applications information interaction.

8 Business processes are decomposed into tasks on the system analysis stage. From the point of view of the business logic interfaces of each external application are decomposed into a set of information services implementing separate business tasks. To integrate applications the user should point out links between the tasks which will invoke corresponding services thus forming a unified business process. It can happen that an appropriate application or service is not found for some tasks. In this case there should be developed additional program components implementing the necessary business logic. These components are intended for enhancement of functionality of the integrated external applications. Usually the functions of the task are restricted to sending a request to the external application and receiving an answer from it. Special tasks are intended for tracking of environment parameters and setting the tasks results according to environment parameters and business process local data. All tasks and services descriptions are located in a separate section of the repository. After business process is formed it is converted into business frame (business process description in XML language) and stored in the corresponding section of the repository. Business frames that have already been created can be overloaded and activated on the fly during the work of the system. In this case the existing tokens would use the old description of the business processes. For new tokens new descriptions should be done. If the user requires the execution of a particular business process, a corresponding business frame is taken from the repository and the token is generated. Token is a business frame created for this particular request. Simultaneously there can be several tokens corresponding to the same business frame. Tokens communicate with external applications by sending and receiving XML-messages. Incoming and outgoing XML-messages are generated and processed by tasks used during business frame development. In course of operation tasks also perform data exchange in the form of the XML format messages. Tokens contain XML-messages created, picked up and modified during tokens life cycle. Message type definitions are stored in a special section of the repository. Environment parameters are available for all the tokens created by the router. The repository can contain any number of environment parameters. Environment parameters values can be used as a unique global variable of PIE. Environment parameters can be defined for each service and also for the whole ESB. For each parameter there is a value by default and a collection of possible values. The default value for each environment parameter and its value for each service can be assigned independently. The router manages data input and output, creates, activates and deletes tokens. The router coordinates distribution of generated tokens by execution threads. The number of threads and distribution algorithm can be configured. For example one thread can be configured for the generation of all reports while for some other processes related to online processing (e.g. doing data input of an order for securities purchase/sale in online trade system) multiple parallel working threads are allowed. Thus on the one hand restricting the system loading during unimportant tasks execution and on the other hand giving the processor power for basic functions execution. In the router each token is assigned a set of attributes for its type like owner (as a rule, PIE user), a maximum number of simultaneously executable tokens of this type etc. The router uses intellectual algorithms of tokens distribution by threads on the basis of the tokens attributes and environment parameters and also manages the events received by it from the system management module. In course of operation the business rules router can reroute a token from one thread to another. This allows providing load balancing, high level of readiness etc. How PIE Server Works First of all the adapter connects to the external application and registers in the PIE application server, thus the logical connection of the external service is done. Adapter registration includes: For external adapter connection of external adapter to PIE application server on the transport level; For all adapters is executed the special businessprocess Logon the description of which is stored in the repository; Special systems tasks should be included into Logon business process in order to draw attention to the fact that the service is linked to PIE. Other tasks can be also added for external applications connection etc.

9 Pic.9 PIE server operational scheme As a result information about the connected service appears in the dispatcher and it can rout messages to and from it. The router receives messages from the services via the dispatcher and creates a token. The token is created according to the corresponding business frame stored in the repository. To achieve maximum performance the business logic description is placed from the frame to the RAM memory. Tokens are created on the basis of incoming messages sent by the services (external applications are treated as services). When a message is received the necessity to create a new token is estimated. The incoming message that creates a new token is called initiating message. The router creates a token on the basis of the information extracted from the input data and repository. If an incoming message is identified as initiating one the router chooses a corresponding business frame from the repository and generates a token. Each token contains: A control block which contains information needed for token execution and its routing. Part of this information is in the business frame; Local data which is stored in the form of XMLmessages, containing an initiating message, all XML-messages (sent and received) that the token exchanges with external applications during its lifecycle. Also all XML-messages that the tasks exchange within the token and if necessary the results of tasks execution. Then the router activates the token (Pic.9). In different threads tokens are activated independently from each other. Several tokens in the thread cannot be activated simultaneously. Using the information of the token s control block the router analyses the current state of the token and initializes the execution of tasks corresponding to this state. Each task has an access interface to local token data. Since local token data is represented as DOM-structures, different methods of DOM structures manipulation are used for reading/writing. Tasks are divided into two types: programmable and predefined. The users can develop new tasks or edit the programmable ones. Predefined tasks don t require programming they execute the function of interaction with services, i.e. with external applications. Predefined tasks group includes such types of tasks like «Invoke», «Receive» and «Transform». These tasks functionality definition is related to a set of parameters specification and is done without any programming. At the same time the task type Invoke is used as the only tool for business process communication with the services represented as external applications. An important feature of the task type Invoke is the possibility of fixing the mode of service interaction synchronous or asynchronous. In

10 the first case the whole process of service interaction including message sending to the correspondent external application and receiving answers from it is executed within the framework of one task type Invoke. In the second case when the asynchronous method of interaction is used a return back to the next task execution is done just after a message sent. It is necessary to indicate the place in the business frame where the reply from the server should be received. The task type Receive is used for this purpose. If the token s control block determines (e.g. by the task execution results) that the token is not ready for further execution, it marks this token as blocked and the business rules router wouldn t activate it until the event unblock. The token s control block unblocks the token if the event expected by the token occurs. It can be an answer from the external application, time-out completion or an event generated by another token etc. The task type Transform allows defining a procedure of XML-message conversion graphically, without programming. During its execution the token can form XML-messages both as a reply to a calling service and to other services. Here calling means the service has prior sent a request as incoming message. Internal data representation in PIE is based on XMLstandard. That s why any incoming messages from the service should be in XML-format. If the external application is capable to process XMLmessages then there is no need to use a special adapter for data conversion from one format to another. Technology without adapters is used for such external applications connection. Otherwise special adapters are used to convert XML-messages to the format of external application and vice versa. PIE as a built-in Component In another Application Server The second basic set of program modules that has been mentioned at the beginning of this article is so called builtin PIE component. It was designed using Java based Application Server approach. According to this conception PIE is considered as an Application Server Component and it is built into Java based Application Server. Built-in PIE Component is implemented using Java technology. The main difference between two sets is that PIE as a component of Application Server has extended and standardized facilities to interact with other applications using host Application Server ones. PIE Component inherits all advantages of the host Web and Application Server. So in many cases PIE needs not having its own adapters. PIE can get messages from host components using their interfaces and returns data same way back. Proper interfaces for JavaScript tasks and utility functions are presented and kept as helper module in PIE repository. PIE continues to fulfill its main role to keep and process business rules of integrated applications. There is a special case when we need to use one of PIE internal adapters. It is when we need to use PIE external adapters or other applications based on XOWAL network protocol. To provide compatibility with them built-in PIE core includes internal XOWAL adapter supporting the same XOWAL protocol as used in standalone PIE Application Server. Built-in PIE core uses repositories created by PieStudio. The core loads and executes business frames from the repository just the same way like standalone PIE does. In addition to pure PIE-style business processes PIE Component supports a specialized type of business processes called precompiled. CMA solutions DepoX/TradX are built on the base of precompiled processes. PIE Component uses additional specialized light core to execute such processes. Built-in PIE Component can import precompiled processes into PIE Repository. Imported precompiled processes in this case become full featured frames of PIE Repository They become visible in PieStudio, can be modified and customized using PieStudio GUI and saved in PIE Repository. The mentioned above DepoX/TradX engines allow full integration with PIE and export their processes into PIE repository. Imported frames are processed by PIE Component main core as native ones. Built-in PIE advantages Due to direct interaction between PIE core and host Application Server there is no performance losses caused by transporting messages. Precompiled host Application Server business processes become native frames of PIE repository and get access to all facilities presented by PIE including frame modifications, customizations, use of PieStudio GUI, etc. Repository may contain both pure PIE-style frames and precompiled imported frames from the host insides. PIE Component has an access to interaction facilities of the host Application Server. Built-in PIE Component is Java implemented same as host Application Server. Tasks for business frames can be developed using Java and JavaScript language facilities. It extremely widens cross platform abilities.