The Service Availability Forum Specification for High Availability Middleware

Transcription

1 The Availability Forum Specification for High Availability Middleware Timo Jokiaho, Fred Herrmann, Dave Penkler, Manfred Reitenspiess, Louise Moser Availability Forum Abstract The Availability Forum Application Interface Specification is an industrial standard for high availability middleware that is intended for telecommunications, data communications and networking applications that must provide continuous service to their users. The Application Interface Specification allows application developers to write application software that is portable across different vendors implementations of the specification. 1. Introduction The Availability Forum (SA Forum) is a consortium of computer and communications industry companies that are dedicated to producing standards that enable the development of carrier-grade systems from commercial-off-the-shelf (COTS) hardware platforms, operating systems and middleware. The key characteristic of a carrier-grade system is its ability to provide uninterrupted service to its users. Typical applications for which the SA Forum standards might be employed to achieve high availability are telecommunication and data communication applications, such as connection establishment, routing and billing. Value-added services, such as voice mail, instant messaging, presence and caller id differentiated response, are becoming increasingly important and will be appropriately implemented using the SA Forum standards to ensure high availability of these services. The SA Forum has developed two standards: the Hardware Platform Interface and the Application Interface Specification. The relationship between these two interfaces is shown in Figure 1. The Hardware Platform Interface (HPI) is an industrial standard for managing a carrier-grade hardware platform that is independent of any particular hardware. The HPI represents the platform-specific characteristics of the physical hardware in an abstract model, and provides standard functions for monitoring and controlling that model. A vendor s implementation of the HPI represents the physical hardware in terms of this abstract model, and translates the library function calls defined by the specification into appropriate actions of the physical hardware. The Application Interface Specification (AIS) is a standard for high availability middleware that is independent of a particular vendor s implementation. As such, it supports portability of applications across high availability middleware supplied by different vendors, just as the Hardware Platform Interface defines an interface for platform management services that is platform-independent. The AIS defines extensive Application Program Interfaces (APIs), which an application programmer can use in conjunction with a vendor s high availability middleware that implements the specification. In particular, it defines APIs for: Availability Management Framework Cluster Membership Checkpoint Event Message Lock. Applications Operating System Availability Middleware Hardware Platform Platform Drivers AIS Interface HPI Interface Figure 1. Relationship between the AIS Interface and the HPI Interface.

2 This paper presents an overview of the SA Forum Application Interface Specification. A previous version of this paper appeared in [2]. For the complete Application Interface Specification standard, please see [7] 2. Application Interface Specification Model The Application Interface Specification represents the high-availability characteristics of the system in an abstract model, and defines extensive APIs that support that model Physical and Logical Entities The AIS model defines several physical and logical entities, including nodes, components, service units, service instances, component service instances, service groups and protection groups, as shown in Figure 2. Node Group Unit active standby Protection Group Protection Group active Node Unit standby Figure 2. Entities of the Application Interface Specification Model. A cluster node is a logical entity that represents a physical node, which behaves like a computer that runs a single instance of an operating system and exports the AIS and HPI APIs. A component is a logical entity that represents a set of resources, such as processes, that are managed by the Availability Management Framework. A service unit encapsulates the components representing hardware and/or software resources into a single management unit. A service instance is a particular service (workload) that the Availability Management Framework has assigned to a service unit. A component service instance is an element of a service instance that a component within a service unit provides. A service group is a set of one or more identical service units that provide service availability to its users. A protection group for a component service instance is the set of components to which that component service instance has been assigned Unit State The AIS model defines service unit states, including the Readiness State, which can be out-of-service, inservice or stopping, and the HA State, which can be active, standby or quiesced Capability Models The AIS model defines several Capability Models, in particular, the x_active_and_y_standby, x_active_or_x_standby and x_active models. In the x_active_and_y_standby model, a component can handle x active and y standby component service instances simultaneously. In the x_active_or_x_standby model, a component can handle either x active or x standby component service instances, but it supports only all active or all standby component service instances at a time. In the x_active model, the component can handle x active component service instances at a time, but does not support any standby component service instances Group Redundancy Models The AIS model also defines Group Redundancy Models, including the 2N and N+M models, showninfigure3. In the 2N redundancy model, each of N service groups contains two service units that provide service availability to one or more service instances. One of the services units is active on behalf of one or more service instances, and the other service unit is standby on behalf of those same services instances. In the N+M redundancy model, the service group has N+M service units. Each of the N service units is active on behalf of one or more service instances, and each of the M service units is standby on behalf of one or more service instances. Active Active Active Active 2N Redundancy Model Active Active Active Active N+M Redundancy Model Figure 3. 2N and N+M Redundancy Models.

3 3. Availability Applications The Availability Forum Application Interface Specification is intended for telecommunications, data communications and networking applications. The Availability Management Framework manages these applications, which are structured into sets of components. A component is an implementationoriented software abstraction for decomposing and distributing applications for service availability. The services that the application provides are decomposed into component service instances that are assigned to components, which support them in active or standby mode. s and component service instances are the basic entities that are manipulated by the APIs of the Availability Management Framework. They are further structured into higher-level entities for the purposes of application and system management. s that are co-located on a node and that must all be in-service or out-of-service together are grouped together in a service unit. The service unit is the unit of redundancy from the management perspective, whereas the component is the unit of redundancy from a design and runtime perspective. A set of identical service units that participate in a given redundancy model are grouped together into a service group. The redundancy model and other attributes of the service group describe how the service units are instantiated on different nodes. When structuring an application for service availability, it is important to take into account the important properties of redundancy, scalability, upgradability and isolation. depending on the redundancy model associated with the service group Upgradability A service unit and its components are upgraded as a unit. To maintain operation of service, component service instances must be able to be assigned from one version of a component to another Isolation Each component is expected to monitor its own state and detect, contain and isolate faults within its scope. 4. Application Interface Specification Areas The Application Interface Specification defines APIs for an Availability Management Framework and for core high availability services, namely, the Cluster Membership, Checkpoint, Event, Message and Lock s. These services provide the basic functionality of the cluster on which the Availability Management Framework and the applications that must be highly available are deployed. These areas of the Application Interface Specification are illustrated in Figure 4, and are explained in more detail below. Application Process Message Library Checkpoint Library Availability Management Framework Library Application Process Lock Library Event Library Cluster Membership Library Availability Management Framework Library 3.1. Redundancy To provide service availability, components deployed in service units are replicated on different cluster nodes according to a redundancy model. To participate in a given redundancy model, components must support the required capabilities. For example, the components must be able to support their component service instances in both the active and standby mode for the 2N redundancy model Scalability To increase application throughput or capacity, additional service units in a service group or additional service groups in a cluster can be deployed. The level of service availability can be traded off against capacity, Message Checkpoint Availability Management Framework Cluster Membership Event Lock Figure 4. Application Interface Specification Areas Availability Management Framework The Availability Management Framework is the software entity that provides service availability by coordinating redundant resources within a cluster. It is based on a view of a logical cluster that consists of a number of cluster nodes. These nodes host various resources in a distributed computing environment.

4 The Availability Management Framework provides a set of APIs to enable highly available applications. It determines the Readiness State and the HA State of a component and checks the health of a component by invoking callback functions of the component. It also allows a component to query the Availability Management Framework for information about the component s state, using functions of the Availability Management Framework Cluster Membership The Cluster Membership provides membership information about the nodes in a cluster to the applications. A cluster is simply a set of nodes, each with a unique identifier, that are connected together by a communication network. As nodes join and leave the cluster, the membership of the cluster changes. The Cluster Membership provides functions that allow an application process to retrieve information about the cluster membership and the cluster nodes. It also allows an application process to register a callback function to receive membership change notifications automatically as those changes occur. All functions of the Cluster Membership are available on all nodes in the cluster Checkpoint The Checkpoint provides a facility for an application to preserve its state across hardware and software faults. If a process fails and is restarted, it can recover quickly by retrieving its state from a checkpoint. If the node on which a process is running fails, then a standby on another node can become active quickly by obtaining its state from the checkpoint. Checkpoints are cluster-wide entities that are designated by unique names. A copy of the data stored in a checkpoint is called a checkpoint replica, which is typically stored in main memory rather than on disk for performance reasons. A given checkpoint may have several checkpoint replicas stored on different nodes in the cluster to protect it against node faults. To avoid accumulation of unused checkpoints in the system, checkpoint replicas have a retention time. If no process has opened a checkpoint for the duration of the retention time, the Checkpoint automatically deletes the checkpoint Event The Event is a publish/subscribe multipointto-multipoint communication mechanism that is based on the concept of event channels, where a publisher communicates asynchronously with one or more subscribers over an event channel. Multiple publishers and multiple subscribers can communicate over the same event channel, and publishers can also be subscribers on the same event channel. Individual publishers and individual subscribers can communicate over multiple event channels. Events consist of a standard header and zero or more bytes of publisher event data. The Event API does not impose any specific transfer syntax for the data, although an implementation of the Event is expected to use a standard for data marshalling to support heterogeneity of data representations between publishers and subscribers Message The Message provides a multipoint-to-point communication mechanism by which processes on the same or different nodes can communicate. Messages are written to, and read from, message queues. Queues are persistent or non-persistent, and migratable or nonmigratable. If an active process fails, a standby process can process the messages from the message queues. The source of the messages continues to communicate with the same queue, and is unaware that a fault has occurred. Queues can be grouped together to form queue groups. Queue groups are identified by logical names, so that a process is unaware of the number of queues and of the physical location of the queues to which it is communicating. Queue groups can be used to distribute workload among the queues in the group. Queue groups can be used to maintain transparency of the sender process to faults in the receiver processes, represented by the queues in the queue groups. The sender process communicates with the queue group. If a receiver process fails, the sender process continues to communicate with the queue group and is unaware of the fault, because it continues to obtain service from the other receiver processes Lock The Lock is a distributed lock service, intended for use in a cluster, where processes in different nodes might compete with each other for access to a shared resource. The Lock provides entities, called locks, which are used to synchronize access to shared data among the application processes. Each lock is claimed and released individually.

5 The Lock provides a simple lock model supporting two locking modes that support exclusive access and shared access. Synchronous and asynchronous calls, lock timeout, try lock, deadlock detection, lock orphaning and lock wait notifications are provided. 5. Using the Application Interface Specification The Application Interface Specification defines APIs for the functions of the Availability Management Framework and of the Cluster Membership, Event, Message, Checkpoint and Lock s that an application can invoke. These functions are implemented within libraries that are linked into the application processes that must be highly available. The AIS also defines callback functions of the application that the Availability Management Framework and the Cluster Membership, Event, Message, Checkpoint and Lock s can invoke. The application programmer must implement these callback functions within the application program. The application program initializes itself with the Availability Management Framework or the particular service, and registers the callback functions, by invoking an initialize function. Subsequently, it closes its association with the Framework or the particular service, by using a finalize function. The Framework and each of the services also provide functions to detect pending callbacks and to specify callback execution behavior. The header files for the functions of the libraries of the Availability Management Framework and the various services are taken directly from the Application Interface Specification, and are written in the C programming language. The header files must be included in the application program modules that use those libraries. 6. Related Work Over the past several decades, there has been much work, both in industry and in academia, on building highly available and fault-tolerant distributed systems. Several excellent general reference books in this field are [1], [3], [4] and [6]. There have been many research papers written in this field, far too many to cite here, and there have been a number of commercial products related to high availability and fault tolerance. There has also been work by other standards organizations in developing standards for high availability and fault tolerance middleware. In particular, the Object Management Group has adopted a specification for Fault Tolerant CORBA [5]. The OMG specification focuses on fault tolerance for CORBA based on replication with strong replica consistency, whereas the Availability Forum AIS specification provides high availability. Fault Tolerant CORBA defines interfaces related to replication, fault detection and notification, and checkpointing and restoration. CORBA itself provides a message service via remote method invocation using the Internet Inter-ORB Protocol, and includes an Event. The OMG has also defined a specification for Real-Time CORBA that is intended for embedded systems. The OMG s specifications are written in CORBA s Interface Definition Language and, thus, are language-independent and more abstract than the Availability Forum s specifications, which are written in C. The OMG specifications are, however, specific to CORBA. 7. Conclusion The Availability Forum Application Interface Specification, discussed in this paper, is an industrial standard for high availability middleware. It represents the high availability characteristics of the system in an abstract model, and defines extensive APIs that support that model. The Application Interface Specification defines a comprehensive set of APIs for an Availability Management Framework, and various services, in particular, a Cluster Membership, Checkpoint, Event, Message and Lock. 8. References [1] Birman, K. and van Renesse, R., Reliable Distributed Computing with the Isis Toolkit, IEEE Computer Society Press, Los Alamitos, CA, [2] Jokiaho, T., Herrmann, F., Penkler, D. and Moser, L., The Availability Forum Application Interface Specification, RTC Magazine, Vol. XII, No. 6, June 2003, pp [3] Marcus, E. and Stern, H. Blueprints for High Availability: Designing Resilient Distributed Systems, John Wiley, New York, NY, [4] Mullender, S., Distributed Systems, Addison-Wesley, ACM Press, Menlo Park, CA, [5] Object Management Group, Fault Tolerant CORBA, OMG Technical Committee Document ptc/ , April 2000, [6] Powell, D., ed., Delta-4: A Generic Architecture for Dependable Distributed Computing, Research Reports ESPRIT, Springer-Verlag, Berlin, [7] The Availability Forum, Application Interface Specification, April 2003,