CTMS/CTIS INTEGRATION Contract Routing No. 04 HAA 00063 Architecture Layers Version 1.1 Approved By Robert Wycoff CDOT ITS Office Signature: Date: John Williams CDOT ITS Office Signature: Date: Prepared By:
Revision History Date Version Description Author 08-Mar-2004 0.1 First Draft Jason Westra Sachin Saindane 27-Apr-04 0.2 Scheduler and Communications are services, not layers. Added Delegate Layer. 11-May-04 0.3 Added Notifications service and Services Catalog Jason Westra Jason Westra 21-May-04 0.4 Added New Architecture Drawing Sachin Saindane 07-March-05 1.1 Added more depth around layers and guidelines Jason Westra Confidential EnRoute Traffic Systems, Inc. Page 2 of 17
Table of Contents i. Executive Summary 5 1.Introduction 6 1.1 Purpose 6 1.2 Scope 6 1.3 Audience 6 1.4 References 6 2. Logical Architecture Reference Model 7 2.1 Architectural Layers 7 3. Application Layer 9 3.1 Client Layer 10 3.2 Delegate Layer 11 3.3 Messaging Layer 11 3.4 Services Layer 12 4. Application Support Layer 13 5. External Services Layer 14 6. Platform Services Layer 15 7. Tiers 16 7.1 Client Tier 16 7.2 Presentation Tier 16 7.3 Business Tier 16 7.4 Integration Tier 17 7.5 Resource Tier 17 7.6 Achieving Deployment Agility 17 Confidential EnRoute Traffic Systems, Inc. Page 3 of 17
Table of Figures Figure 1: A Four-Layer Organizational Structure For Application Components...7 Figure 2: Conceptual Application Layer...9 Figure 3: CTMS Application Layer...10 Table 1: Application Support Layer...13 Table 2: External Services Layer...15 Table 3: Platform Services Layer...16 Confidential EnRoute Traffic Systems, Inc. Page 4 of 17
Architecture Layers i. Executive Summary This particular document describes the roles and responsibilities of the various layers that are central to the CTMS/CTIS architecture and catalogs the services available to the architecture. Confidential EnRoute Traffic Systems, Inc. Page 5 of 17
1. Introduction This document describes the application architecture of CTMS and the various roles and responsibilities of each layer that contributes to the overall CTMS/CTIS software architecture. 1.1 Purpose The purpose of this document is: To supplement the technical information provided in the Messaging Architecture Assessment Document. To capture and convey the significant messaging architectural issues and decisions which have been made during meetings and discussions. 1.2 Scope The scope of this document is the CTMS/CTIS system. 1.3 Audience This document describes the underlying messaging architecture within the core CTMS architecture from various perspectives. Since most sections are technical in nature, the document is focused towards technical managers, technical architects and the software development team. 1.4 References The following references were either used in the creation of this document or may be used to supplement the detailed descriptions of the CTMS/CTIS system: The CTMS/CTIS Architecture Assessment Document. The CTMS/CTIS Messaging Architecture Assessment Document. Design Patterns: Elements of Reusable Object-Oriented Software - Gamma, Helm, Johnson, Vlissides Core J2EE Patterns: Best Practices and Design Strategies Alur,Crupi, Malks Confidential EnRoute Traffic Systems, Inc. Page 6 of 17
2. Logical Architecture Reference Model The logical architecture makes it possible to communicate the design of the system with both technical and business audiences. It covers the structure, guidelines, and capabilities of the system without getting mired in the details of the technical implementation of the architecture such as protocols and interfaces. The logical architecture organizes broad definitions into categories and organizes them in such a way as to allow the implementation to be carried out effectively and efficiently. It provides a way to help determine build vs. buy for each category whether at a coarse-grained or granular level. For instance, it can help make decisions like whether or not to implement a whole subsystem with COTS (e.g., an ERP solution) or just a few services in the subsystem with a specific OTS (commercial off the shelf) product. 2.1 Architectural Layers Application services may be organized into layers, each representing a gross classification of the degree to which services in the layer have to do with business or technology. Services in higher layers have more to do with business and are most likely custom built while services in lower layers tend to be more technical and are often purchased rather than built. Figure 1, below, illustrates a four-layer classification scheme with Application, Application Support, External Services and Platform Services layers. The foundation of the application architecture is the Platform Services that represent the raw technology upon which a solution is built. Examples include operating systems, computer and networking hardware, operating systems, network operating systems, interface cards, and communication protocols. Provides the user interface, business logic, and process flow custom-built for the application. Application Layer Example: UI, images, and services Provides common, but custom services to the Application Layer that function as adapters/ wrappers to External Services. Application Support Layer Example: A wrapper around SNMP protocols or a J2CA Connector to a legacy system Provides third-party APIs and components that the application must interface with. Example: DB, Java Mail API, Java Comm APIs External Services Layer Network, operation systems, and hardware. Example; Routers, Unix on Solaris, firewalls Platform Services Layer Figure 1: A Four-Layer Organizational Structure For Application Components The External Services Layer represents purchased or pre-existing software with which your system must integrate. Examples include database management systems, messaging services, application monitoring services, e-mail, and legacy systems. These services are purchased or pre-existing and rely on underlying Platform Services Layer in order to operate. The Application Support Layer sits above the External Services layer. Services in this layer may include application security (e.g., authentication and authorization), object identity and naming, exception and event handling, logging, and transaction management. These services form the common, architectural plumbing of your enterprise infrastructure and are reusable across multiple applications. In addition to the services just mentioned, perhaps one of the most powerful services in this layer is the adapter service that can be used to encapsulate access to external and platform services, insulating Application Layer components from technology changes. Confidential EnRoute Traffic Systems, Inc. Page 7 of 17
To understand how each layer provides insulation for the next, consider this example. Multiple components in the Application Layer require authentication and authorization services. Users and permissions are stored in a relational database. The components requiring security services call a Service Adaptor interface that exposes methods like login(username, password), logout(user), and haspermission(permission, user). Depending on the request, the Service Adaptor might query the relational database using SQL to validate login credentials or to see if the user has the permission specified. As the company grows, an enterprise security provider is chosen and all applications must now conform to. It happens to use an LDAP Directory Server for the storage of users and permissions. The Service Adaptor simply needs to change its implementation to make LDAP directory calls now instead of SQL queries. Applications that used the Service Adaptor did not have to change. Unfortunately, applications in the company that did not use the Service Adaptor have to change each touch point in the system to conform to the new LDAP directory APIs. At the top of the architecture stack, the Application Layer represents services unique to a particular system its presentation, business logic and data storage logic. These are nearly always custom built components because they have to do with a specific business problem. Adherence to layering rules increases the technical agility of a system in that underlying technology can be replaced with minimal, if any, impact on the operation of higher-level services. The degree to which this type of layer encapsulation is required is a practical matter, depending on both the anticipated rate of change in the External and Platform Services layers and the number of Application Layer components that use a lower layer service. Confidential EnRoute Traffic Systems, Inc. Page 8 of 17
3. Application Layer The following section details the Application Layer of the CTMS in more depth. The Application Layer described above is further broken into sub-layers for purposes discussing sub-layer responsibilities as well as guidance for implementation of the Application Layer. Client Layer Supports Web, portable, and GISbased thick clients Delegate Layer Gateway to Services Layer orchestrating services into complex units of work Messaging Layer Transport bus for messages to and from clients and services Services Layer Loosely coupled services perform business logic and data access Figure 2: Conceptual Application Layer The conceptual architecture of CTMS is depicted in Figure 2 as a service-oriented architecture where services can be implemented as anything from Web Services, EJBs, Java Management Extension (JMX) MBeans or even plain old Java objects (POJOs). Requests are sent via the Messaging Layer to the remotely accessible Delegate Layer, which acts as a routing agent. The Delegate Layer routes the request to Services, which execute behind it and are only accessible from this layer. The architecture supports a Messaging Layer with any type of protocol including RMI, IIOP, JMS, SOAP, and HTTP/S and both asynchronous events as well as synchronous RPC-like calls. Because services are deployed behind the Delegate Layer, they are easily distributed and clustered. The following diagram gives a more technical look at the server-side layers and how services interact with other services using method invocations and events. Confidential EnRoute Traffic Systems, Inc. Page 9 of 17
Service Service Messaging Layer Event Delegate Layer Event Service Service Figure 3: CTMS Application Layer In Figure 3, services never communicate directly to one another. All events whether synchronous or asynchronous are sent over the Messaging Layer and routed by the Delegate Layer to the appropriate Service to handle the request. The Delegate Layer may route the request to multiple services or just a single service. The following layers are discussed in detail: 1. Client Layer 2. Delegate Layer 3. Messaging Layer 4. Services Layer 3.1 Client Layer The Client Layer will be implemented as a thick, Java client across 3 screens (a GIS map, a notifications screen, and at later iteration, a CCTV (Closed Circuit Television) screen for camera input). It is responsible for displaying the user interface to the users of the application. It must be able to send and receive both synchronous and asynchronous communications via the Messaging Layer. Examples of asynchronous notifications include alarms, errors, and replies to instructions from operators. The main responsibilities of the Client Layer are: 1. Display the GUI (graphical user interface) 2. Initiate system requests 3. Handle messages from the Messaging Layer 4. Handle errors Confidential EnRoute Traffic Systems, Inc. Page 10 of 17
3.2 Delegate Layer The Delegate Layer is the gateway to the Services Layer. More importantly however, the Delegate Layer is responsible for orchestrating individual services in the Services Layer into more complex units of work. It controls the transactional nature of a request and provides subsystem abstraction for services and legacy systems. Clients use the Messaging Layer to contact the Delegate Layer, passing it a message either synchronously or asynchronously and the Delegate Layer routes each message to the appropriate services. The main responsibilities of the Delegate Layer are: 1. Routing of requests to proper services 2. Coordinate multiple services into a coarse-grained unit of work 3. Abstract core subsystems into more manageable interfaces 4. Apply security controls to clients for services they request 5. Act as an auditing mechanism for requests in and out of the system The Delegate and Services Layers of the architecture should be consistent across a clustered deployment. This means that the services and orchestrations deployed across a clustered application are exactly the same on each node. This makes it easier to configure, deploy, and implement. As stated earlier, the Delegate Layer must provide a remote interface for clients to connect to and call services. However, the Delegate Layer must also be smart enough to know that it is already within the server space and when to use non-remote APIs, which are more scalable. An orchestrated sequence of service calls should be local to the server space. Likewise, calls from services to the Delegate Layer should be localized. Since the same services are deployed across all cluster nodes, any service is always locally accessible from the Delegate Layer once inside the server space. 3.3 Messaging Layer The Messaging Layer represents the protocol, protocol handlers and underlying infrastructure used to communicate to and from the Delegate Layer. The layer can be implemented using protocols such as SOAP, RMI, IIOP, or JMS and protocol handlers that know how to connect, make requests, and receive replies on the protocol of choice. The Messaging Layer is the bus that messages are passed on between services either synchronously or asynchronously. The implementation must be clustered for load balancing and high-availability. For instance, if Java Web Services are used, they should be servicing requests from a distributable Servlet Container. If Stateless Session Beans are used the EJBs should be clustered to load balance requests across EJB Containers. If the implementation is JMS, the JMS Provider should support fail-over using a persistent message store and it should support load-balancing messages across numerous JMS receivers. The Messaging Layer has the following major components: 1. Channel the destination upon which messages are sent and received. In a JMS implementation, this is a Queue or Topic. 2. Consumers Handle the messages on the Messaging Layer 3. Producers Send messages to the Messaging Layer The main responsibilities of the Messaging Layer are: 1. Secured, anonymous connectivity to services over an asynchronous message bus 2. Capabilities to administer channels (add, remove, query channels) 3. Allow remote notifications to groups of listeners (e.g., JMS Topics) 4. Allow remote notifications to a specific listener (e.g., JMS Queue) 5. Provide fail-over capabilities in the event of a communications failure. a. For example, a JMS Provider that stores messages persistently b. For example, a JMS Provider that provides automatic fail-over of JMS consumers to a secondary JMS Server c. For example, a JMS Provider that provides automatic fail-over of JMS producers to a secondary JMS Server Confidential EnRoute Traffic Systems, Inc. Page 11 of 17
d. For example, if a node is down, an RMI client might contact another node in the cluster and resume request processing. As an architectural guideline, the number of channels should be kept to a minimum to ease the burden of channel and message management on the provider. This generally leads to a pattern where a message consumer handles a message by passing it off to a secondary handler in the form of a Command pattern. 3.4 Services Layer A service has a well-defined interface describing the services it provides to clients. A service can be invoked remotely not assuming that the client of the service is operating in the same environment. The CTMS architecture designates the Delegate Layer as the gateway to services, so services must expose their APIs through the Delegate Layer if they are to be remotely accessible. Consequently, services exist within a computing space (e.g., a J2EE Server) and can be designated as local to this space. Services are self-contained, that is, they can be used independently of any other services. Services are very loosely coupled and do not interact with each other, but instead perform very succinct tasks and rely on the Delegate Layer to coordinate more complex, multi-step services to fulfill a request. A service is loosely coupled. This characteristic is achieved by enforcing that no service references another service directly. Instead, the services live under these guidelines: 1. Services use the Messaging Layer to make anonymous requests that are handled by the Delegate Layer as it sees fit. 2. The Delegate Layer handles coordination of independent services to achieve complex request handling. 3. References to services can be discovered rather than fixed at development or deployment time. This once again, is handled by the Delegate Layer, which has the capability to find services dynamically. Not only are services decoupled from each other, the Services Layer is decoupled from the Client Layer as well. Thus, services must never connect directly to a client to make requests or replies directly to the client. This would tightly bind clients with services and vice versa. It would also make security and auditing more difficult to provide if each service was opened up to the Client Layer. Confidential EnRoute Traffic Systems, Inc. Page 12 of 17
4. Application Support Layer Components in the Application Support layer may be common to multiple applications within an enterprise. They are considered plumbing and therefore do not offer significant business value to individual applications using their services. Services in this layer often include the following: Category Name Application Security Common Libraries Exception Handling Event Handling Logging Object Identity and Naming Service Adapters Transaction Management Description of Service These are components that have to do with security the application and typically involve user authentication, access authorization, and non-repudiation. May also include auditing and management of user accounts and account activity. This represents 3 rd party utilities that perform basic operations such as parsing XML, regular expressions, and data validations. These are used within the same process as the application; whereas, an External Services Layer component might function in a separate address space altogether. Handles errors in the application in a standard way. Usually integrates with the Logging service to provide information about the errors to system administrators and developers alike. Handling events posted from application components. Applies persistence to the events to prevent loss during application failures. A specific type of service adapter called out because of the significance that logging plays in production operations. The service provides ways to log various levels such as DEBUG, WARN, and ERROR as well as guidance to development teams on which to use. This service generates globally unique ids (GUIDS). It usually requests the service from a database or other shared resource. It can provide caching algorithms to enhance performance where the number of transactions per second is high. In simple terms, these are wrappers. Service adapters represent a interfaces to external services to isolate the implementation of a service from the protocol the application uses to access it. In J2EE, these are J2CA Connectors. Implements services to track business data changes. Represents them in a logical unit of work (LUW) to promote ACID transaction processing at the resource level. Table 1: Application Support Layer Confidential EnRoute Traffic Systems, Inc. Page 13 of 17
5. External Services Layer External Services are almost always 3 rd party products when discussing buy vs. build options. External Services generally provide the capability to run on their own and provide an API for an application component to request services from it. Often external services are grouped together to form more useful products. For instance, Application Servers bundle resource pooling, a transaction coordinator, a message bus and a web server together into one product offering. Nonetheless, there are an infinite number of combinations of external services and many are not listed here at all. However, the table below does provide some of the more common services in this layer. Category Name Application Server Application Monitoring Services Business Process Management Clustering and Replication Services Content Management Services Database Management System Directory Server Distributed Timing Service Message Bus Description of Service Generally, a class of server that bundles multiple Application Support Layer and External Services Layer categories into a single product. There are two standards available: J2EE and.net. Examples of J2EE servers include BEA WebLogic, IBM WebSphere, and JBoss. The most notable.net example is Microsoft s.net platform. Applications that monitor the health of the system and report anomalies to any number of media including emails, pagers, logs, and monitoring consoles. Generally, JMX and SNMP are used to integrate systems with console vendors. Software that allows the creation, deployments, and monitoring of business processes. Usually includes workflow, automation, and business activity monitoring capabilities. Hardware and software that enable high-availability operation of the system wherein two or more homogeneous components operate as single entities and back up each other in the event that one fails. Includes session-state replication, and database synchronization, for example. Services that provide a user-friendly interface to manage the content of a web application. Generally, used by business users to alleviate the technical users from this task. Any data storage system including relational databases and object databases. Examples: Microsoft Access and SQL Server, Oracle, and Sybase. Directory services are used in distributed applications to provide classifications/properties, lookup services, and authentication. They are generally LDAP and accessed by an API like Java Naming and Directory Interface (JNDI). A distributed timing service that synchronizes all clocks on all computing devices on the network. Servers that that enable and guarantee delivery of asynchronous communications between components. Confidential EnRoute Traffic Systems, Inc. Page 14 of 17
Category Name Network Security Personalization Services Resource Pooling Runtime Environment Transaction Coordinator Web Server Workflow Engine Description of Service Hardware and software components used to lock down the network from viruses and intruders. Generally provided by firewalls and intrusion alert software. Services that track and profile users behavior to personalize content and campaigns to individuals. Supports pooling of expensive resources such as remote connections and database connections. Provides and API to get and return the resources to and from the pool. Provides a runtime environment for application components. The best example is Java Virtual Machine, while the C# runtime is also gaining popularity. Other systems like Visual Basic require DLLs to run properly. Services that enable the two-phase commit protocol and/or compensating transactions for components whose functions span multiple resources. This type of server generally excels at serving static and dynamic content over the HTTP protocol. Examples include Apache and IIS. Coordinates manual tasks between humans. Task lists are shared and individual allowing tasks to be acknowledged and completed by a single person. Tasks are usually persistent allowing them to be tracked and performed over extended periods of time. Table 2: External Services Layer 6. Platform Services Layer The Platform Services layer is the foundation of the application architecture. It represents the raw technology upon which a solution is built. Examples include operating systems (OS), computer hardware, networking hardware such as routers, switches, and hubs, interface cards, communication protocols, mass storage, and replication technologies. This layer is 100% technology based with zero business affinity and is almost always purchased as a 3 rd party product rather than built by the development team. While this layer has zero business affinity, each layer above the Platform Services Layer must be able to co-exist and operate efficiently upon this foundation. Luckily, many standards exist today that negate the need to design solutions around proprietary hardware and networking specifications. For instance, standards exist for network protocols such as TCP/IP and wireless standards like IEEE 802.11b and IEEE 802.11g that allow network hardware to be easily swapped with little worry of interoperability. Table 3 covers the more common Platform Services in more depth. Category Name Computer Hardware Description of Service Includes components such as cables, UPSes, monitors, motherboards, AGP and PCI cards, CD/DVD-ROM drive, keyboards, memory sticks, processors, and mice. Confidential EnRoute Traffic Systems, Inc. Page 15 of 17
Category Name Communications Protocols Network Hardware Operating Systems Other devices Description of Service Describes how low level communications are transmitted over hardware and network devices including TCP/IP, 802.11b and IEEE 802 for wireless, and RS-232 for serial communications. These components include CAT5 cable, fiber, routers, hubs, switches, NIC cards, wireless cards and wireless access points. They are hardware devices designed to connect other devices over a wired or wireless network (usually Ethernet-based) and compatible with standards such as TCP/IP, IEEE 802.11b and IEEE 802 depending on whether it is for wireless networking or not. Provides insulation over the hardware to make it easier to work with hardware devices through plug-and-play. The most common are Windows and various flavors of Unix and Linux such as HP-UX, Solaris, Red Hat, and Suse. This category is broad and includes anything non-typical such as cameras, TVs, radars, etc. Table 3: Platform Services Layer 7. Tiers Layers provide the language necessary to discuss the logical structure of systems independently of their physical deployment. We use the term tiers (or optionally, partitions) to describe the run-time topology of the system; that is, which of the services are deployed on which of the network computing nodes. Node types (or tiers) common to J2EE applications are Client Tier, Presentation Tier, Business Tier, Integration Tier, and Resource Tier (Core J2EE Patterns, pg. 120-121). 7.1 Client Tier This tier represents where the presentation is rendered. Typically, a web application presentation will render in a browser. A wireless application may be rendered in a handheld device such as a PDA or phone. On the other hand, a thick client runs on a desktop computer using a graphics card to render the display. The desktop computer may have one or more monitors depending on the complexity of the application. Finally, a Client tier can also be represented as a headless client such as a batch process that renders nothing or only outputs information to logs. 7.2 Presentation Tier The Presentation tier is responsible for providing a view into the system by serving up dynamic web content or client windows. If it is a web application, it typically includes one or more computers in the network demilitarized zone (DMZ) where browsers make contact with the application and which serve static content. The Presentation tier for a Java client shows views of the application such as Java Swing windows and widgets on a desktop computer. Usually, initial authentication and authorization take place at this tier. If using OS based authentication such as an NT Domain login, this logic already took place in the Client tier. 7.3 Business Tier The Business tier represents one or more computers in the secure area of the network that provide common services and business logic to application clients. Components in this tier are EJBs and Web Services that serve the business Confidential EnRoute Traffic Systems, Inc. Page 16 of 17
logic and can be deployed across multiple nodes in a cluster ensuring load balancing and high-availability for the services. This tier is also responsible for managing transactions, which can span multiple Business and Integration tier components. For instance, a transaction in the Business tier could include calls to an EJB, a Connector, and then send a message via JMS, all of which must commit or rollback together. Transactions in the Client, Presentation, or Integration tiers are not recommended for many reasons. Some tiers are inherently unstable (e.g., a wireless PDA in the Client tier), which could leave resources locked in transactions until a timeout occurs. If the Client and Presentation tiers start transactions they will most likely be distributed across multiple nodes, which is costly as well. Finally, if the Integration tier manages transactions, then it s possible for components in the Business tier to be left out of the transaction. 7.4 Integration Tier The Integration tier communicates to external resources, which can be hosted on any number of computers on a secure network. Components in this tier include JDBC, J2CA Connectors, and JMS for messaging. They could also include proprietary software such as the case with 3 rd party vendors who provide adaptors to their services. 7.5 Resource Tier The Resource tier contains business data and resources shared across applications. This tier may include databases, legacy systems, 3 rd party services, or directory services such as LDAP servers to provide authentication and authorization capabilities to the application. Some components in this tier may contain business logic especially if they are legacy applications or ERPs. Multiple tiers generally rely on the Resource tier. For instance, the Client tier may initiate login to a workstation by verifying credentials against an LDAP server. Likewise, the Presentation tier will authenticate and authorize access to web resources and the Business tier may authenticate, authorize, or even administer the directory. Similarly, a relational database is usually the preferred data store for today s applications and a single database may be shared by many applications. 7.6 Achieving Deployment Agility As it turns out, strict adherence to architecture layers, rules and design patterns provides a great measure of deployment agility as well, making it easier to scale up to a larger number of users or processes. An initial deployment of a web application, for example, might host presentation, business and integration components all on one server or server cluster. However, as the application scales, it may be necessary to move presentation, business and integration behavior to different nodes so that each node s resources are better utilized. Without proper planning, redeployment of components may incur unnecessary communication overhead (e.g., too chatty ) and may require software modifications to handle remote communications efficiently. If the architect anticipates redeployment as a possibility, he or she may design scalable communications into the software with an eye on future deployment strategies. Confidential EnRoute Traffic Systems, Inc. Page 17 of 17