Grid-Enabled Visualization of Large Datasets

Transcription

1 Grid-Enabled Visualization of Large Datasets Damon Shing-Min Liu* Department of Computer Science and Information Engineering National Chung Cheng University, Chiayi, Taiwan Abstract In the past years, scientific visualization has become a rapid, emerging discipline aiming at developing approaches and tools to facilitate the intepretation of and interaction with large amounts of data, possibly with high resolution and long time series. Typical visualizations are compute-intensive, therefore, single-system visualization software cannot scale up to this large data size. One effective approach to solve such problems is to parallelize or distribute the needed computation on different machines in order to reduce the overall computing time. In this paper, we describe a work that integrates the Grid technologies to distributedly process the multiple visualization requests, thereby optimizing the overall system performance. In it, we adopted Grid s functionalities and advantages to develop compute-intensive systems. The work we present may have a significant impact as techniques developed in the system can provide the basic building block for future efficient implementation of large complex data visualization systems. Keyword: Scientific visualization, Grid computing, Resource management, Data partitioning, Load-balancing 1. Introduction Typical scientific visualizations are computeintensive and the size of invoved datasets is usually large. This is mostly due to that the increased capabilities of workstations have been used to dramatically increase the size and complexity of numerical simulations. When the simulation s size increases, the size of solution datasets also increases, resulting in immense datasets representing the physical characteristics of a studied field. Most solutions have been developed to resolve such problems. One effective approach is to parallelize or distribute the needed computing to different machines in order to reduce the overall computing time. Moreover, we can decompose the large data set into many smaller data partitions suitable for parallelism. If we exploit such data partitioning techniques, we also need a robust scheme (for example, a load-balancing algorithm) to coordinate the multiple requests at multiple computing machines. In other words, we must always get access to the static and dynamic system-wide information on the multiple computing machines in order to provide the timely information for task allocation decision involving distributed resources. Grid Computing or Computational Grid is a type of parallel and distributed system that refers to an emerging network-based computing infrastructure providing security, resource access, information, and other services that enable controlled and coordinated sharing of resources by individuals and institutions. It also brings together widely distributed storage systems and computing platforms [1, 2]. It can be used in a wide range of applications such as scientific and teleimmersive environments. We consider that Grid Computing will be advantageous in developing scientific visualization systems because there are important functionalities in Grid such as information service and resource management that can help to construct the parallel or distributed computing environment. Grid information service can provide and discover the available resource on each individual computing machine. Grid resource management service can provide a strategy for enabling remote computations. In this work, we seek to explore the techniques of Grid Computing and construct a Grid-enabled scientific visualization system. 2. Related Work 2.1 Visualization Systems [3] presented an interaction model to enable collaborative scientific visualization. The system is Web based, supporting multi-user in a multidimensional graphics display, and was developed in Java with CORBA support for external server access. [4] discussed how technology for Web programming applications may be employed for providing interactive visualization on the Internet. Their system uses Java Applets that access VTK (Visualization Toolkit) [5]. [6] presented a Web based scientific visualization system that supports effective load-balancing between clients and server. The visualization system can be constructed according to a fat server, a fat client, or a hybrid approach and can even adapt itself to different kinds of client's computing capabilities. Foster et al. [7] presented an * Corresponding author. 117

2 approach where components of the visualization pipeline are spreaded between different systems and architectures. They claimed that the new visualization architecture must incorporate distance issue as a basic concept and called it distance visualization. 2.2 Grid Systems One of the key challenges in Grid is to develop a flexible, scalable, and self-adaptive resource management system. The Grid infrastructure needs to allow a user to submit jobs to a Grid without necessarily knowing where it will be executed. It is the responsibility of the Grid resource management system to distribute such jobs among a heterogeneous pool of servers, trying to optimize the resource usage and provide the best possible quality of service [8]. [9] provided a tool set for performance oriented development of Grid applications. Tools implemented include resource brokerage and monitoring, scalable discovery and organization of computing resources, workflow-based dynamic and fault-tolerant execution of activities, performance prediction for estimating job execution time, and performance monitoring. [10] is a framework designed to solve numerical applications over Grid. An important aspect of this framework is the ability to decide when to migrate jobs based on actual and predicted execution times. If the system detects minimal differences between the actual and predicted performance of the application, it avoids job migration. [11] implemented an adaptive Grid resource broker to facilitate adaptation during run-time to variations in system resources, such as load or CPU utilization of a particular job. They also implemented prediction based modeling that is used as a basis for a decision as to whether job migration is required to satisfy a pre-specified time constraint. A good survey of work towards the development of Grid technologies can be found in [12]. 3. Architectural Design Essentially our approach integrates a prototype visualization system [13] with Globus Toolkit [14] and Java CoG Kit [15] to build a scientific visualization architecture on the Grid. The Globus Toolkit provides us with the necessary resource allocation and data management tools. In Globus, a term called gatekeeper is the one used to process incoming requests and execute them. Here we call the Globus gatekeeper a compute node and the cluster of compute nodes, which constructs our computing environment, the compute resources in the architecture. Besides, in our visualization model, a typical visualization job needs to execute a series of tasks (usually called a visualization pipeline) in order to generate the visualization result. In the system, we decompose, distribute, and execute different stages of the task at various compute nodes in order to Journal of Information Technology and Applications 118 parallelize the visualization pipeline. Therefore, a visualization job may be splitted into many tasks. We specify these remote visualization tasks with embedded VTK executable commands [16], and send these tasks to different compute nodes for execution. Our Grid-enabled visualization system typically consists of three tiers: 1. Backend computing 2. Middle tier 3. Client tier The backend computing is responsible for handling the remote visualization tasks. Besides having Globus installed, compute nodes must have VTK installed in order to successfully perform the visualization tasks. Furthermore, if the current compute node (in backend computing) does not finish all visualization stages, it generates intermediate files and submits another job (which is used to finish the remaining stages) to middle tier. Middle tier is responsible for information gathering, job's brokerage and dispatching. It communicates with both backend computing and client tier. To the backend computing, middle tier submits clients' requests to compute nodes and waits for computing results. To the client tier, middle tier returns the jobs' results to it. The general structure of the architecture is shown in Figure 1. The detailed individual components and the architectural overview are shown in Figure 2. Note that there is a data center that lies in backend computing tier as a sub-component enabling storage of all visualization data. In the following, we describe in detail each tier in this Grid based visualization architecture. 3.1 Backend Computing In the backend computing of our architecture, there are two major components: 1. Compute resources 2. Data center Each individual compute node in compute resources has the following services and components available: Figure 1: The general structure of a visualization system in Grid

3 Figure 2: Actual components of our visualization system in Grid 1. Resource and data management service: In the backend of our architecture, each compute node is responsible for executing jobs remotely to yield the results. The operational scenario is that the resource management daemon acting in compute nodes translates the incoming request, processes the request, and returns its execution results. This component also handles file transmissions on the Grid and is used to retrieve and store the results or intermediate outputs on the data center. 2. Information providing service: In our architecture, we need to collect the information in compute resources, because we want to choose an appropriate compute node from the compute resource pool to execute jobs. Grid actually supports the functionality of providing such information which can give us the static and dynamic system-wide information of each compute node. 3. VTK module: Every compute node must have VTK libraries locally resident in order to be able to execute the visualization tasks. Moreover, the needed services automatically initiate when system cold starts or warm starts. The data center serves as the storage of visualization raw datasets. Final computing results or intermediate outputs are also stored on the data center. 3.2 Middle Tier The middle tier contains the following: 1. Public queue: The public queue is where the requests are placed and sent for further processing. A queue manager is running to support the routines and primitive operations such as push and pop. 2. Information gathering agent: The information gathering agent interfaces with each compute node's MDS (Globus Monitoring and Discovery Service) [17], and provides necessary information for resource broker. 3. Resource broker: The resource broker plays an important role in this architecture. It is responsible for coordinating the resource allocation of multi-requests at multiple compute nodes. In resource broker, the information gathering agent interfaces with the aggregate directory service and queries to supply the necessary information. Besides, a load-balancing algorithm was implemented to take information from the information gathering agent and select the best suitable compute node for processing the visualization tasks. 3.3 Client Tier Each client in client tire contains a graphics user interface as well as other necessary components such as VTK libraries. For example in isosurface construction, each client sends its visualization jobs that contain data source, data format and isovalue or time parameter to middle tier. If compute nodes do not finish all stages in the visualization pipeline, client may be asked to continue the remaining stages. On the contrary, client may only need to get the final results (in image or VRML file format) if compute nodes have finished all stages in the visualization pipeline. In current implementation, client receives the remote location of intermediate files or final results, and writes onto local disk because VTK is strictly limited to access files locally. 4. Implementation Methodology In this work, we aimed to convert an Internet-based visualization system implemented in Java into a Grid-enabled visualization system. In [18], they demonstrated the services of Globus can be accessed using interfaces written in C or Java. Moreover, [19] described the reasons and advantages of using Java in Grid computing. We favored Java for constructing our Grid environment. Therefore, Java CoG Kit, which combines Java technology with Grid computing, became our choice for developing the architecture [20]. We discuss the details of our system's implementation as follows. 4.1 Implementing Backend Computing The operating environment of compute node in backend computing is Linux with Globus Toolkit and Visualization Toolkit installed. The compute node must satisfy some requirements, such as GRAM (Grid Resource Allocation and Management) [21] and MDS, in order to be Grid-enabled. Besides, we developed a Java program that handles visualization 119

4 techniques (including streamline and isosurface) in each compute node [22]. The input datasets (including grid data and solution) of the program are in PLOT3D format. Raw data were previously partitioned into smaller data size to fit in the compute node's main memory. In the following, we describe the requirement and detailed knowledge of compute node's functions regarding how it can run remote jobs and provide the pertinent information. In order to run the remote jobs, compute nodes must have Globus GRAM service installed and configured correctly. Note that the reason we did not use Java CoG Kit to create a gatekeeper to provide the functionality as the traditional Globus gatekeeper is because the Java gatekeeper is simply not the same as the C version of the Globus gatekeeper. Java does not allow remapping of the client to run using its userid. Even worse, the available Java CoG Kit only supports the implementation of globus-personal-gatekeeper in Globus. Here, we truly like the compute node can share its globus-gatekeeper with many others. As soon as a compute node gets Globus RSL (Resource Specification Language) [23] from middle tier, the globus-job-manager is activated and forks a process (aforementioned Java program) to perform the visualization task. This Java program reads the original dataset from data center. After processing, it produces either the intermediate output (in VTK format) or final result (in JPEG or VRML format) for subsequent compute node or client. Besides, it also returns a string indicating the location of output through GASS (Global Access to Secondary Storage) [24] protocol. Subsequent compute node or client must retrieve this output using that string in order to continue executing the remaining stages in the visualization pipeline. Figure 3 shows the use of GRAM to execute remote visualization jobs in each compute node. For providing the current system-wide information, compute nodes must also have the Globus MDS service installed. This information can be categorized into static and dynamic information. The static information includes the computing power expressed in term of MHz in the processor, MB in memory size, and so on. Dynamic information, such as the size of free memory space and CPU current usage, is also evaluated with respect to some pre-measured parameters. Moreover, we completed constructing a hierarchical GIIS (Grid Index Information Service) [25] in the backend computing tier. In order to do that, we configured the parameter files associated with MDS that are located in $GLOBUS LOCATION/. By querying the hierarchical GIIS, we can have the knowledge of the computing environment. Figure 4 depicts the overall picture of our hierarchical GIIS. Figure 4: The overview of our hierarchical GIIS. GIIS root is the registrar and other machines are registrants. The registrants send their information to the registrar Besides, one important MDS attribute we like to obtain is Mds-Cpu-Free-1minX100 which is the average value of the CPU load in previous one minute. However, precision of this data is not high enough. We therefore wrote an embedded C program to calculate the most recent CPU load. The new reporting value is much more accurate than Mds-Cpu-Free-1minX100. Furthermore, the new value can be retrieved using an attribute called Mds-ProcessorLoad-Free. We also publish this information to the MDS. Moreover, the data center was constructed using the Globus GASS. Figure 3: The overview of using GRAM to execute visualization tasks 4.2 Implementing Middle Tier In our architecture, the middle tier is mainly responsible for: 1. Resource brokerage 2. Information gathering 120

5 When middle tier is activated, it starts the information gathering agent and receives visualization jobs from clients. These visualization jobs contain the primary variables such as dataset, isovalue, and so on. Middle tier evaluates these variables and creates a corresponding RSL representing these visualization jobs. The execution command in RSL is java (in our system, it is /usr/bin/java). Location of Java executable is usually specified by setting the executable tag in RSL (e.g., &(executable=/usr/bin/java)). Moreover, in the body of RSL, we have to provide the necessary information for variables such as LD_LIBRARY_PATH, CLASSPATH and DISPLAY (Without the DISPLAY variable information, the main procedure of VTK cannot be executed correctly.). We also assign these variables by setting the environment tag in RSL. An example RSL is as follows: &(executable=/usr/bin/java) (environment=(classpath Classpath-for-Java) (LD_LIBRARY_PATH The-dynamic-load-library-path.) (DISPLAY The-display-host-for-VTK.)) (arguments=name-and-arguments-of-the-program) Besides these RSL tags, other RSL arguments are described in [23]. Moreover, a GASS server must also be running in the middle tier because Globus GRAM uses GASS protocol to listen for GRAM output. Regarding information gathering, Java CoG Kit provides MDS APIs to access the MDS services for collecting different static and dynamic information in compute nodes. But the MDS APIs are depreciated to use. Consequently, we used Java Naming and Directory Interface (JNDI) [26], which is a standard extension to the Java platform, providing Java technology-enabled applications with a unified interface to multiple naming and directory services to access the MDS. Therefore, in middle tier, an information gathering agent was created to obtain information using JNDI. When agent was activated, it queried the GIIS to get the static information of every compute node. However, while querying the dynamic information, it did not query the GIIS directly. Instead, it queried each GRIS in every compute node in order to get the accurate dynamic information. These information can truly represent the loading on every compute node. For comparison purpose, besides applying Mds-ProcessorLoad-Free attribute in MDS, we also used the traditional Mds-Cpu-Free-1minX100 attribute to retrieve the information that represents the actual computing power in each compute node. From our observation, information obtained from applying two attributes often varies. The least loaded computer nodes selected by load-balancing algorithm are therefore different most of the times. The one selected from applying Mds-ProcessorLoad-Free seems more reasonable. Regarding job's brokerage and dispatching, we can obtain static and dynamic information from the information gathering agent. The load-balancing algorithm evaluates the utilization of each compute node using the queried information. It can help to choose the least loaded compute node to execute the visualization tasks. More powerful compute nodes usually handle the early-stage processing, such as filter of the visualization pipeline, because the computation is often more complex in early-stage processing. Since we have identified the least loaded compute node, we may submit to it the current visualization jobs for execution. In order to submit GRAM jobs through the Java framework, we must create an instance of GramJob in Java CoG Kit and give this instance the aforementioned RSL specifying the job we want to execute. Moreover, we must implement a GramJobListener and a JobOutputListener interface to receive job's output and state. In summary, the procedure of submitting a job includes: 1. Creating an instance of GramJob with given RSL and credential setting. 2. Adding the listeners. 3. Sending the GramJob to compute node and waiting for output. 4.3 Implementing Client Tier In client tier, each client has a graphics user interface written in Java Swing, GASS, and other components such as the VTK libraries. In our Grid-enabled visualization system, we need to obtain the VTK or image files via GASS protocol in client tier after getting the results. Therefore, we create a thread which is used to handle the file transmission. This thread uses the UrlCopy in Java CoG Kit to get data from data center. After the VTK or image files are transferred over, the client either uses VTK libraries to process the rear stages in the visualization or displays the resutlant images. 5. Experimental Results In this Grid based visualization system, client's graphics user interface interacts closely with the server, and the visualization results are displayed on it. The GUI and example visualization results are shown in Figure 5. Figure 6 shows three different CPU availability values on the same compute node. They are: (1) value obtained from applying Mds-Cpu-Free-1minX100 attribute; (2) value obtained from our impoved 121

6 Figure 5: Visualizing NASA Delta-Wing dataset (with isolvalue = 0.3) in our Grid based visualization system Figure 6: Comparison of CPU loading values on the same compute node using three different methods. Mds-Cpu-Free-1minX100 computes the average value of the CPU load in previous one minute; Cpu availability from top is obtained from issuing a Unix top command scheme by applying Mds-ProcessorLoad-Free attribute; and (3) value obtained from issuing a Unix top command. We observed that (3), value obtained from issuing a Unix top command, evaluates to either 0% or 100% most of the times, is considered to be very insensitive and less precise. Therefore, it is not useful in determining loading of a CPU. On the other hand, traces of (1) and (2) are almost the same at the beginning, from Time 0 (secs) to Time 40 (secs). At Time 40, we submitted several heavy visualization tasks to this compute node via Globus GRAM. CPU availability thus decreased in both due to the initialization of globus-job-manager and the task executions are in progress. However, we noticed that (2) responded more instantly and thus more sensibly. The main difference was observed at and after Time 75. The compute node completed most of the tasks, CPU availability in (2) recovered immediately, but the value of (1) did not recover until at Time 125. Time elapses before (1) can precisely reveal the 122

7 actual change in loading information of the CPU. In the duration of the simulation, we found that value computed using (2), our new scheme, is significantly useful in improving the load-balancing algorithm in the system. 6. Conclusions By integrating the Grid technologies, we distributedly process the multiple visualization requests, optimizing the overall system computation. We have successfully adopted several Grid s functionalities and advantages to develop such a compute-intensive scientific visualization system. In the future, we plan to continue applying various multidisciplinary applications to our Grid architecture and evolve it into a OGSA-based system using another Grid development the Open Grid Services Architecture (OGSA) [27]. References [1] Foster, I. and Kesselman, C., 2003, The Grid 2: Blueprint for a New Computing Infrastructure, 2nd ed., Morgan Kaufmann. [2] Berman, F., Fox, G.., and Hey, A., 2003, Grid Computing: Making the Global Infrastructure a Reality, John Wiley & Sons. [3] Bajaj, C. and Cutchin, S., 1999, Web-Based Collaborative Visualization of Distributed and Parallel Simulation, IEEE Symposium on Computer Graphics and Image Processing, pp [4] Alves, A., Ferreira de Oliveira, M., Minghim, R., and Nonato, L., 2000, Interactive Visualization over the WWW, XIII Brizilian Symposium on Computer Graphics and Image Processing (SIB-GRAPI 00), pp [5] Schroeder, W., Martin, L., and Lorensen, B., 2006, Visualization Toolkit: An Object-Oriented Approach to 3D Graphics, 4th ed., Kitware. [6] Kuo, C.H. and Liu, D.S.M., 2003, A Dynamic Load-Balancing Approach for Efficient Remote Interactive Visualization, International Conference on Information Technology: Computers and Communications, pp [7] Foster, I., Insley, J., von Laszewski, G.., Kesselman, C., and Thiebaux, M., 1999, Distance Visualization: Data Exploration on the Grid, IEEE Computer, Vol. 14, pp [8] Pathak, J., Treadwell, J., Kumar, R., Vitale, P., and Fraticelli, F., 2005, A Framework for Dynamic Resource Management on the Grid, HP Laboratories Palo Alto. [9] Fahringer, T., Jugravu,A., Pllana, S., Prodan, R., Junior, C.S., and Truong, H.L., 2005, ASTALON: A Tool Set for Cluster and Grid Computing, Concurrency and Computation: Journal of Information Technology and Applications 123 Practice and Experience, Vol. 17. No. 2, pp [10] Vadhiyar, S. and Dongarra, J., 2005, Self Adaptivity in Grid Computing, Concurrency and Computation: Practice and Experience, Vol. 17, No. 2, pp [11] Othman, A., Dew, P., Djemame, K., and Gourlay, I., 2003, Adaptive Grid Resource Brokering, IEEE International Conference on Cluster Computing. [12] Krauter, K., Buyya, R., and Maheswaran, M., 2002, A Taxonomy and Survey of Grid Resource Management Systems for Distributed Computing, Software Practice and Experience, Vol. 32, No. 2, pp [13] Kuo, C.H., Shiu, C.J., and Liu, D.S.M., 2002, A Framework for Web-Based Scientific Visualization, Computer Graphics Workshop. [14] Globus Toolkit, [15] Java CoG Kit, g/java/ [16] Tseng, C.H., Tseng, C.W., and Liu, D.S.M., 2004, A Generic Functional Architecture for Scientific Visualization Using Grid, First Workshop on Grid Technologies and Applications, pp [17] Globus Monitoring and Discovery Service (MDS), [18] von Laszewski, G., Jarek, G., Carlos, J., and Foster, I., 2002, InfoGram: A Grid Service That Supports Both Information Queries and Job Execution, IEEE International Symposium on High Performance Distributed Computing (HPDC-11), pp [19] Getov, V., von Laszewski, G., Philippsen, M., and Foster, I., 2001, Multiparadigm Communications in Java for Grid Computing, Communications of the ACM, Vol. 44. No. 10, pp [20] von Laszewski, G., Foster, I., and Gawor, J., 2000, CoG Kits: A Bridge Between Commodity Distributed Computing and High-Performance Grids, ACM Conference on Java Grande, pp [21] Grid Resource Allocation and Management (GRAM), 2/gram/ws/ [22] Shiu, C.J. and Liu, D.S.M., 2003, A Grid-Enabled Framework for Scientific Visualization, Digital Life and Internet Technologies. [23] Globus Resource Specification Language (RSL), _spec1.html [24] Global Access to Secondary Storage (GASS),

8 [25] Grid Index Information Service (GIIS), [26] Java Naming and Directory Interface (JNDI), [27] Open Grid Services Architecture (OGSA), Journal of Information Technology and Applications Biography Damon Shing-Min Liu is on the faculty of the Department of Computer Science and Information Engineering at National Chung Cheng University. Previously in the San Francisco Bay Area, he worked at Litton Industries, TRW Inc, (subsidiary of Northrop Grumman Corp of Los Angeles) and the prestigious Xerox Palo Alto Research Center (PARC). His research interests include computer graphics and animation, virtual reality and simulation systems, high-performance I/O for data-intensive Grid computing applications, interactive real-time scientific visualization and data analysis. He received his BSc in Electrical Engineering from National Taiwan University, his MSc in Computer Science from Brown University, and his PhD in Computer Science from University of California, Los Angeles (UCLA). He is a member of the ACM and the IEEE Computer Society. 124