Maximizing Productivity through Parallel and Distributed Computing

Transcription

1 Maximizing Productivity through Parallel and Distributed Computing Philip Chan Software Technology Department College of Computer Studies De La Salle University 2401 Taft Avenue, Manila, Philippines Abstract. In recent years, networks of workstations have become economical alternatives to parallel computers. We propose a project that aims to transform a network of computers into an integrated environment for high-performance computing. This project will support heterogeneous processors/operating systems and provide multiple paradigms for application development. 1 Introduction A modern personal computer equipped with the latest Intel Pentium 4 or AMD Athlon XP microprocessor is definitely a powerful data processing tool. These products are the results of continuing advances in VLSI technology, ushering us further into an exciting future. Running at about 3 GHz and containing more than 40 million transistors, these processors offer its users ever higher levels of productivity never before possible. While a single desktop computing system with such processing capability is certainly impressive, consider a scenario where a networked collection of 16, 32, or 64 such systems is available for processing. As a whole, such systems offer an immense processing capability. For quite a number of computational problems in the sciences and engineering, the use of multiple processors can reduce the execution times and increase productivity. This area of research is the concern of parallel computing. The general idea is to build an application in such a way that permits multiple processors to be employed to execute it. The main goal: high performance, i.e., running a computation as fast as possible. High-performance computing is a valuable tool to solve the most computation intensive problems in the sciences and engineering. 1.1 Technological Requirements Parallel computing requires two primary components: hardware and software. The hardware can range from proprietary commercial parallel computers to networks of commodity-off-theshelf (COTS) components like personal computers/workstations. Parallel computers are usually built from ground up to contain hundreds to thousands of processors. While these offer the best platform for parallel computing, they are also quite expensive. On the other hand, networks

2 of workstations (NOWs) are available in most modern workplaces, e.g., university computing laboratories, medium to large offices, etc. Due to its wide availability, there has been a growing trend to use networks of workstations (NOWs) for parallel computing. As a result, NOWs make parallel computing quite readily available to a wide range of users. The remainder of this paper is concerned primarily with parallel computing on a network of workstations. Since individual systems on a network are likely to be geographically distributed, parallel computing on NOWs take on a further dimension that we may not have full control over all the processing facilities. This aspect is addressed by distributed computing. While parallel computing deals with high performance, distributed computing is concerned with effective and efficient cooperation between systems that are geographically separated. Distributed computing must handle issues like: (a) data accessibility data on one machine is not accessible directly by another; (b) reliability of communication messages sent from one machine may be lost in transit; and (c) dynamically changing configuration machines may shutdown without prior notice and new ones may be attached to the network at any time. While parallel computing hardware are quite readily available, writing applications that can take advantage of the large processing capability offered by these systems is not a trivial task. Various software is needed to provide an environment that seamlessly integrate the processing capability. This software infrastructure needs to address two issues: (a) resource management; and (b) programming paradigms. Resource management has to deal with allocation and control over the processing capabilities of each member of the network. The programming paradigm is the programmer view of the entire system. Applications are written using one or more paradigms. Each paradigm offers a programmer with a set of programming constructs to build parallel programs. These constructs are used to structure the application, making it amenable to parallel execution. In particular, these constructs specifies how the components of an application are to interact with each other. 1.2 Current Parallel Programming Paradigms There are several paradigms for parallel programming: (a) message passing; (b) remote procedure call (RPC); (c) distributed shared memory (DSM); and (d) Netfiles model. In this section, we briefly describe each. On all networks, the underlying mechanism for communication between processes is message passing. Several message passing facilities are available, among them: Parallel Virtual Machine (PVM) [2] and various implementations of Message Passing Interface (MPI) [5] standard. Writing parallel programs using message passing is difficult. It has been likened to writing in assembly language [3]. For many who are used to sequential programming, writing a message passing application requires understanding a new paradigm. For instance, parallel programming using MPI/PVM typically involves embedding functions calls to perform process communication and synchronization. Remote procedure calls (RPCs) uses the familiar notion of local procedure calls and parameter passing as a means for exchanging data between processes. RPC makes it possible for a program to cause a procedure/function residing on a server to be executed. Conventional RPC requires the calling process to wait for the called procedure, therefore, there is a need to change this to asynchronous (non-waiting) remote procedure calls.

3 Distributed shared memory (DSM) is another approach that aims to simplify the parallel programming process. A DSM provides programmers an abstraction of a globally accessible shared memory over a network of computers [4]. The DSM paradigm provides a more familiar programming environment. Processes communicate implicitly by means of accessing shared variables. All message passing activities are handled by the underlying DSM software. More recently, we have proposed is the use of file I/O as abstraction of interprocess communication called Netfiles. The general idea is that a parallel application is written as a collection of independent programs which communicate by reading and writing to conventional files. This simplifies programming since he can test the application on a single computer. When the application is ready, it is re-linked with the Netfiles library and all the file I/O are implemented via messages. Effectively, the programmer has written a parallel application. Our preliminary research [1] on this is very promising. 2 Challenges To achieve a truly usable environment of high-performance parallel/distributed computing, several challenges must be overcome by the software infrastructure. Heterogeneity The system should hide differences among the systems in a NOW. It is to be expected that a NOW will contain computers of various architectures (e.g., PowerPC, Intel, etc.) and operating systems (Linux, Unix, Windows, MacOS X, etc.). Easy to Use This may be addressed by adopting a multi-paradigm approach. This means our system should support all the four programming paradigms, providing the programmer with options to select from. Transparency The fact that the environment is networked and distributed should be hidden from the users. All processing capabilities should be integrated in a seamless way. Ubiquity Users should be able to access their applications and data from any participating system on the network. This encourages mobility of the user. In the future, access should be possible using portable handheld devices. Scalable and Fault Tolerant The system (ideally) must be able to provide reasonable service even on a large collection of machines. On these environments, it is to be expected that hosts may fail and the system must be able to deal with such failures. Good Performance The primary goal is to provide a programming environment for high-performance computing. System overhead should be kept to minimum. This goal, however, is quite difficult to achieve in practice.

4 HEEPP Shell Visual Prog g Environment HEEPP Runtime Monitor NetFile RTL DSM RTL ARPC RTL??? Network Abstraction Layer SEL SEL SEL SEL SEL Linux Unix MacOS Windows Solaris 3 The HEEPP Project Fig. 1: System Architecture of HEEPP We propose a project as an initial step to addressing the above challenges. We have named this the HEEPP project, short for Heterogeneous Environment for Efficient Parallel Programming. The goal of the project is to develop an integrated programming environment that supports multiple paradigms on top of a network of heterogeneous machines. This system should be able to mask the differences in the underlying operating systems and processor configurations from the user. In other words, a user should be able to run his application on top of MacOS, Linux and Windows without much trouble. This may not be achievable for all cases, however. In addition, the system should also have a resource management scheme that seeks out under-utilized computing resources and allocate these as necessary. Fig. 1 shows the primary components of the HEEPP system. We briefly describe each component below. Services Export Layer (SEL) This component hides the differences between operating systems and machine architectures. It also exports a set of services that the underlying system supports. These include: process creation, local resource reservation, etc. Network Abstraction Layer (NAL) This software layer provides network-wide services, including: message passing, identification of systems, and resource management. RunTime Layers (RTLs) Each programming paradigm is supported by a corresponding runtime layer. For instance, the DSM RTL supports distributed shared memory functions. HEEPP Shell This component provides a parallel scripting language to stage and configure execution of HEEPP applications. With this language, it becomes possible to separate computational issues (written using the conventional languages like Java, C and C++) from the parallel execution issues. HEEPP Visual Programming Environment This provides a visual interface for parallel programming in HEEPP supporting all paradigms. It aims to simplify the task of writing and executing parallel applications.

5 HEEPP Runtime Monitor This component acts like a visual management tool to allow the user to see the progress of his application. This makes it easier to determine and diagnose any problems in the execution. 4 Conclusions HEEPP is in the initial stages of development. We hope that this concept paper will serve to generate enough interest in this promising area. Ultimately, we hope to participate in grid computing research which aims to integrate computing resources globally. References [1] Philip Chan and David Abramson. NetFiles: A Novel Approach to Parallel Programming of Master/Worker Applications. In Proc. of the 5th International Conference and Exhibition on High-Performance Computing in the Asia-Pacific Region (HPCAsia 2001), Queensland, Australia, September [2] A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sunderam. PVM Parallel Virtual Machine: A User s Guide and Tutorial for Network Parallel Computing. MIT Press, [3] Claudia Leopold. Parallel and Distributed Computing: A Survey of Models, Paradigms, and Approaches. John Wiley and Sons, [4] Kai Li and Paul Hudak. Memory Coherence in Shared Virtual Memory Systems. ACM Transactions on Computer Systems, 7(4): , November [5] Message Passing Interface Forum. MPI: A Message-Passing Interface Standard. May 5, 1994.