A High-Performance Virtual Storage System for Taiwan UniGrid

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "A High-Performance Virtual Storage System for Taiwan UniGrid"

Transcription

1 Journal of Information Technology and Applications Vol. 1 No. 4 March, 2007, pp A High-Performance Virtual Storage System for Taiwan UniGrid Chien-Min Wang; Chun-Chen Hsu and Jan-Jan Wu Institute of Information Science, Academia Sinica, Taipei, Taiwan {cmwang, seeme, Hsi-Min Chen Department of Computer Science and Information Engineering, National Central University, Taoyuan, Taiwan Abstract In Taiwan, a community of educational and research organizations interested in Grid computing technologies founded a Grid computing platform, called Taiwan UniGrid. Taiwan UniGrid consists of three primary portions: Computational Grid, Data Grid, and Web Portal. In this paper, we present the development of a virtual data storage system for Taiwan UniGrid. In addition to developing basic data storage functions, we identify three main requirements of the current development: high-performance data transfer, data sharing and single sing-on. For these requirements, we come up with three corresponding features in our data storage system: Self-Adaptation for high-performance data transfer, forming user groups and specifying admission control for data sharing, and adopting GSI authentication to enable single sing-on. Besides, we also develop a Java-based graphic user interface of the storage system that allows Grid users to manage data transparently as using local file systems. Keyword: Data Grid, data storage system, data transfer, web service, and single sign-on. 1. Introduction With the rapid growth of computing power and storage capacity of computers, many researchers and scientists have been concentrated on the development of various Grid systems to efficiently utilize distributed computing and storage resources in the recent years. In Taiwan, a community of educational and research organizations interested in Grid computing technologies founded a Grid computing platform, called Taiwan UniGrid [1]. These organizations contribute their resources of computer clusters for sharing and collaboration. The objective of Taiwan UniGrid is to provide educational and research organizations with a powerful computing platform where they can study Grid-related issues, practice parallel programming on Grid environments and execute computing/dataintensive applications. As similar to other Grid systems, Taiwan UniGrid consists of three primary portions: Computational Grid, Data Grid and Web Portal. Computational Grid is responsible for managing scattered and heterogeneous computing resources and scheduling the jobs submitted by users. Data Grid is a virtual storage infrastructure that integrates distributed, independently managed data resources and allows users to save and retrieve their data with ease. Web Portal, developed by National Tsing Hua University, is a uniform user interface by which Grid users can design workflow, submit jobs, manage data, monitor job and resource status, etc. In this paper, we will present the development of the data management system for Taiwan UniGrid. As the distribution of storage resources and the growth of data size, the needs for efficient Grid data management are continuously increasing. In these years, many research and scientific organizations have engaged in building data management and storage tools for Grids, such as SDSC SRB (Storage Resource Broker) [2], SciDAC Data Grid Middleware [3], GriPhyN Virtual Data System [4], etc. SRB is a general Data Grid middleware that integrates distributed and heterogeneous storage resources and provides virtualized access interface. It has been a production data management tool and adopted by several Grid projects. Thus, among these tools, we decide to build our virtual storage system for Taiwan UniGrid based on SRB, while developing additional features that are not well supported by SRB. Before implementing the virtual storage system, we elicited requirements from the user and manager needs. Herein, in additional to the basic Data Grid functions provided by SRB, we identify three main requirements of the current development listed as follows. High-performance data transfer: Since the size of data generated by scientific instruments and Grid applications has grown into the range of Terabytes, large data transfer over the Internet usually leads to a long latency and becomes a bottleneck for job executions. Thus, the need for

2 A High-Performance Virtual Storage System for Taiwan UniGrid high-performance data transfer is an important issue in Taiwan UniGrid. Data sharing: Two important concepts of Grids are sharing and collaboration. Grid users, such as scientists and researchers, are accustomed to retrieve data collected by remote scientific instruments, analyze these retrieved data via various analysis tools, and share the analyzed results for further processing. Therefore, how to facilitate Grid users to contribute or get shared data with ease is a crucial requirement in the development of a data management system. Single sign-on: In essence, physical resources within a Grid system are distributed in different organizations and managed independently. Each organization has its own security policy. Without single sign-on mechanisms, Grid users have to keep a list of accounts for each machine by themselves. This becomes an obstacle for users to use Grid systems. Hence, we have to take the problem of single sign-on into account when we integrate our system with Computing Grid and UniGrid Portal. Consequently, in our system, we come up with three features with respect to the corresponding requirements. For high-performance data transfer, we propose a multi-source data transfer algorithm, called Self Adaptation [5], which can speed up the data transfer rate in data replication, downloading, moving, and copying. For data sharing, our system allows Grid users to share their data in a manner of forming user groups and specifying admission control on each data object. For the issue of single sign-on, we choose GSI (Grid Security Infrastructure) [6] as our user certification mechanism by which Grid users only have to login once and utilize Grid resources through certificates, so that they have no need to keep all accounts for each machine. Besides these features, we also develop a Java-based graphic user interface of the storage system that allows Grid users to manipulate data transparently as using local file systems. The remainder of the paper is organized as follows. In Section 2, we explain the system framework and deployment. Section 3 presents main features, including multi-source data transfer, data sharing, single sign-on, and the data management client. An operational scenario of Taiwan UniGrid is demonstrated in Section 4. Finally, we present some concluding remarks in the last section. 2. System Framework and Deployment Figure 1 shows the framework of our virtual storage system. In the server side, the left bottom of the framework is a set of physical storage resources, including hard disks, tapes and databases, contributed by the members of Taiwan UniGrid. We adopt SRB as a data management middleware to integrate these scattered storage resources. It provides a list of data and storage management functions. Although SRB has furnished an efficient data transfer approach by using multiple TCP streaming, we propose an alternative, called Self Adaptation, to get a higher data transfer rate in comparison with the original one. We will explain the detail of Self Adaptation in section 3. Therefore, we add the alternative (Self Adaptation patch) into the original functions of SRB. A set of extended SRB APIs are built on top of SRB and the Self Adaptation patch. The extended SRB APIs consist of primary APIs provided by SRB and the APIs for highperformance data transfer, such as MSDTReplicate() and MSDTCopy(). Figure 1. The framework of the virtual storage system for Taiwan UniGrid. The right of the server side of the framework is a number of Web services used for data management. Web service technologies are playing an increasingly important role in the new generation of Grids. Such technologies encapsulate heterogeneous software components, legacy systems and resources as services and simply describe their interfaces in a standard description language, i.e. WSDL [7]. Service providers can advertise their services in a registry, i.e., the UDDI [8] server for clients to browse. If clients want to use the services advertised in a registry, the SOAP [9] technology helps them access the services through standard transport protocols, such as HTTP and SMTP. Therefore, we adopt Web services technologies in our system to integrate other software developed by third parties. There are two services implemented in the current system: the AutoReplicator service and the Account Management service. The AutoReplicator service is developed by Chung Hua University. Grid users can utilize it to set various replication policies. We develop the Account Management service to wrap up the functions of user authentication in UniGrid Portal for single sign-on. In the client side, the bottom is the data management library for UniGrid which connects to the corresponding server-side extended SRB APIs

3 Journal of Information Technology and Applications Vol. 1 No. 4 March, 2007, pp and data management services. We implemented two versions of the library. One is Java-based and another is C-based. The data management library provides a uniform interface of data and storage management by which programmers can build various Grid applications to access the underling storage resources. the data objects. Then the SRB server will automatically ask the MCAT server, which registers the SRB server, to update the metadata of the operated data and synchronize with other MCAT servers. Thus, a Grid user can login to one of SRB servers that are closest to him/her and utilize all storage resources in any zone of Taiwan UniGrid. 3. Main Features In this section, we present the main features, including multi-source data transfer, data sharing, single sign-on, which implement the requirements listed in Section 1. In addition, we also develop a friendly graphic user interface of the virtual storage system that allows Grid users to manage their data as using local file systems. Figure 2. The deployment of the virtual storage system for Taiwan UniGrid. Figure 2 presents the deployment of our virtual storage system. Since there is a huge amount of storage resources distributed in Taiwan UniGrid, using a single information server to maintain the metadata regarding users, data and storages may cause the problems of server overloading and single point of failure. To avoid these problems, we divided all storage resources in Taiwan UniGrid into five zones, i.e. Taipei_UuiGrid, Hsinchu_UniGrid, Taichung_UniGrid, Tainan_UniGrid and Hualien_UniGrid. Each zone has a MCAT (SRB Metadata Catalog) server installed for maintaining the metadata of the users, data and storage resources. To enable the flexibility of sharing, the administrators of a MCAT server can specify their won sharing policies, for instance, some resources can be shared with users registered in other zones, but some are utilized in private. In addition, each MCAT server periodically synchronizes its metadata with each other to keep the metadata consistency among zones. By synchronization, Grid users registered in one zone can access any storage resources located in other zones and retrieve sharing data timely. The members of Taiwan UniGrid can contribute their storage resources by setting up SRB servers. Each SRB server consists of one or more storage resources and is registered to a MCAT server. Gird users can manipulate data objects in a storage resource of a SRB server, for example uploading data objects, creating replicas and modifying metadata of Figure 3 (a) The replica selection approach. (b) The multi-source data transfer approach Multi-source Data Transfer To achieve high-performance data transfer, data replication has been a widely used technique that facilitates a Grid user to select a best replica site closest to the specific destination and transfer the selected replica to it. Instead of transferring data from the source site, selecting the best replica can reduce the data transfer time on the Internet. A number of approaches have been proposed for selecting the best replica based on various criteria [10][11][12]. However, as shown in Figure 3(a), since such an approach only allows users to specify one replica for transfer in each selection, they have two major shortcomings: When several replicas have almost the same network performance, choosing a slightly better replica and discarding all others does not fully utilize network resources. Selecting only one replica may degrade transfer reliability because, if the connection to the selected replica fails, it has to execute the

4 A High-Performance Virtual Storage System for Taiwan UniGrid selection algorithm again and reconnect to other replicas. Some multi-source data transfer mechanisms have been presented recently to solve the above problems [13][14], whereby a transferred data object can be assembled in parallel from multiple distributed replica sources as shown in Figure 3(b). To improve the data transfer rate, we propose an efficient data transfer algorithm, called Self- Adaptation. It not only enables the data transfer from multiple replica sites as other multi-source data transfer algorithms, but is also more adaptive to the variability of network bandwidths. Self-Adaptation assigns proper segments of transferred data to each replica site based on the overhead and bandwidth measured from the previous data transfer, so that it can achieve higher aggregate bandwidth. More information of Self-Adaptation and performance comparisons with other approaches can be found in [5]. Multi-source data transfer is the major contribution to the development of the data storage system. In the client-side library of the current system, we implement three alternative functions of data transfer based on Self-Adaptation to enable high-performance data transfer. MSDTDownload(): Grid users or programs can download data objects to their local file systems and the downloaded objects are reassembled in parallel from the source and replica sites. MSDTReplicate(): Grid users or programs, for example the AutoReplicator service, can make new data replicas to the specified destination resources and the new replicas are reassembled in parallel from the source and replica sites. MSDTCopy(): Grid users or programs can make copies of data objects to the specified directories of the virtual storage system and the copies are reassembled in parallel from the source and replica sites of the original data objects Data Sharing According to our experience, we found that Grid users usually need a platform where they can work collaboratively. Although most Data Grid middleware provides the sharing of storage resources, data sharing for collaborative work is not well supported. Therefore, in our system, we develop a collaborative platform through the combinations of forming user groups and specifying access permissions on each data object. A group of users who need to work collaboratively can ask the administrators to form a user group. For instance, a user group can be built according to some research topics in which a group of users are interested. Each Grid user can take part in many user groups simultaneously as long as he/she gets the grants from the administrators. Once an administrator creates a user group, the system will create a group workspace, i.e. a group home directory, for sharing and collaboration. Each group workspace can assign one or more owners to manage the admission of the workspace. In general, Grid users have their own personal workspace, i.e. a user home directory, where they can manage their private data objects. Data objects can be files, directories, replicas or links. Grid users can share their private data objects with others via specifying access permissions on data objects. Figure 4 shows a screenshot of admission control for data sharing, by which Grid users can specify read or write permission for each data object to other users or groups. It also supports the owner change of a specific data object. On the other hand, Grid users can share their data by uploading or copying private data objects directly to the group workspaces. Figure 4. Admission control for data sharing Single Sign-on Because software components and resources within a Grid system are distributed in different organizations and managed independently, using one account for a Grid user to utilize all these software components and resources becomes a crucial issue. GSI (Grid Security Infrastructure) [6] is a promising solution to the issue in Grids. GSI uses X.509 certificates to securely authenticate users across the network. Moreover, SRB supports two main methods of authentication, GSI and an SRB secure password system known as Encrypt1. GSI in SRB makes use of the same certificates and Public Key Infrastructure (PKI) [15] as do Globus Toolkit [16] such as GridFTP [17]. Since we adopt Globus Toolkit as the middleware for managing computing resources, in order to enable the single sign-on for utilizing Computing Grid and Data Grid, we choose GSI as the main user authentication mechanism in our system. To use Taiwan UniGrid, Grid users have to register in UniGrid Portal first. The users will receive certificates issued from UniGrid CA after approved by system administrators. Meanwhile, the users profiles are also registered to Computing Grid and Data Grid, i.e. Globus and SRB. Once users want

5 Journal of Information Technology and Applications Vol. 1 No. 4 March, 2007, pp to use Taiwan UniGrid, they can login to UniGrid Portal through their certificates and the system will automatically generate corresponding secure proxy certificates which are good for a few hours to submit jobs and manage data in distributed resources. Figure 5. The cross-zone problem. However, the current implementation of SRB does not well support the resource utilization cross difference zones by GSI authentication. As shown in Figure 5, for example, Grid_User1 and SRB_Server1 are registered in Zone A, as well as SRB_Server2 is registered in Zone B. If we adopt the Java-based client-side APIs, named Jargon, provided by SRB, Grid_User1 connecting to SRB_Server2 by GSI authentication will be failed to access the resources (Resouce3 and Resource4) in Zone B. We call this incident as the cross-zone problem. At present, SRB only supports the access to cross-zone resources through secure password authentication, Encrypt1. Since we deployed our system in five zones and developed Self-Adaptation to reassemble data objects in parallel from multiple replica sources, which may be located in different zones, it causes the cross-zone problem. We will address this problem from two perspectives, users and programs, in the following paragraphs. From the perspective of users, we intent to make Grid users login once by certificates and launch the data management client to manipulate their data without concerning with the cross-zone problem. Thus, we propose an authentication process, as shown in Figure 6, to enable single sing-on for UniGrid Portal and the data management client. Figure 6. The proposed authentication process enabling single sing-on for UniGrid Portal and the data management client. After a Grid user logins to the portal successfully, the portal asks the Account Management service to create a session and returns necessary information, including a generated session key and a profile for SRB to connect. The Grid user can launch the data management client to access data in storage resources after login to the portal. While launching the data management client, the portal passes the session key and SRB-related information to the client and then the client uses the session key to obtain the user s password through SSL from the Account Management service. Finally, the client uses the password and SRB-related information to connect to a SRB server in Encrypt1. Once connecting successfully, the Account Management service removes the session. This prevents malicious users from using the cached session keys to retrieve passwords from the Account Management service. Figure 7. The proposed authentication process enabling single sing-on for computing nodes. From the perspective of programs, Resource Broker delegates submitted jobs to computing nodes with limited proxy certificates, not full proxy certificate, for authentication. However, in the current implementation of SRB, the limited proxy certificates will be failed in accessing storage resources located in different zones. Only full proxy certificates are allowed to access the cross-zone resources in SRB. Hence, we propose an authentication process, as shown in Figure 7, to deal with this problem. After Resource Broker submits jobs to computing nodes with limited proxy certificates, the computing nodes use the limited proxy certificates to get full proxy certificates from the Account Management service. Finally, the nodes can connect to SRB servers located in different zones with full proxy certificates and access programs and data in storage resources. 3.4 The Data Management Client We develop two kinds of clients of the virtual storage system. One is Java-based standalone version not integrated with UniGrid Portal and Computing Gird. It is suitable for users who just want to store

6 A High-Performance Virtual Storage System for Taiwan UniGrid their data without the need of computation support. Another one is Java Web Start version which is embedded in UniGrid Portal. Grid users can launch the client directly from UniGrid Portal after they login. Figure 8. A screenshot of the data management client. Figure 8 shows a screenshot of the data management client. The left of the client is the file and directory list of local storage drives and the right is the file and directory list of SRB storage drives. Once Grid users login to our system, the system directs them to their default home directories automatically, and then they can access data or traverse the whole storage system. As shown in Table 1, for various data objects, we provide difference operations on them in the current implementation. Data object Operations File download, upload, delete, copy, paste, rename Directory download, upload, delete, copy, paste, rename Link create, download, delete, copy, paste, rename Replica create, delete Table 1. The supported operations for data objects in the virtual storage system. Unlike other FTP systems, our system allows users to specify resources, for instance closest resources, to store uploaded data. An uploaded data object can further be made several copies, i.e. replicas, disturbed in different resources for reliability and efficiency of data transfer. In addition to creating replicas by users, we also integrate the AutoReplicator service in the client. Users can set replica policies on data objects via the client. The AutoReplicator service will automatically create replicas according to the specified policies. Furthermore, through the data management client, users can also specify access permissions on data objects, as shown in Figure 4, for sharing. 4. Operation Scenario of Taiwan UniGrid In this section, we will demonstrate an operation scenario of using Taiwan UniGrid. Figure 9 shows the major components of Taiwan UniGrid and their interactions. The high-level operation scenario is explained as follows. A Grid user logins to UniGrid Portal by entering his/her account and password and UniGrid Portal employs Account Management service to verify user s identity. If login successfully, UniGrid Portal directs the user to his/her working web page, as shown in Figure 10. He/she launches the data management client (Figure 8) and uploads programs and data needed for the jobs, which will be submitted later, to the data storage system. The user makes an execution plan for a job or designs a workflow of jobs on the working web page. Once the user has submitted a job, the portal asks Resource Broker to select computing resources based on the requirement of the submitted job. Resource Broker assigns the submitted job to the selected computing nodes. The selected computing nodes then retrieve programs and data from the storage system. The selected computing nodes start computing. Once all computing nodes finish their work, the computed results are merged and stored back to the storage system. For reliability, the newly stored data can be replicated to other storage resources by the user or the AutoReplocator service.

7 Journal of Information Technology and Applications Vol. 1 No. 4 March, 2007, pp Figure 9. The major components of Taiwan UniGrid and their interactions. 5. Concluding Remarks In this paper, we present the development of a high-performance virtual storage system for Taiwan UniGrid. We employ SRB (Storage Resource Broker) as a basis to implement the functions of the storage system. Besides, we identify three main requirements in the current implementation: high-performance data transfer, data sharing, and single sign-on. To meet these requirements, we propose the corresponding features: Self-Adaptation for highperformance data transfer, forming user groups and specifying admission control for data sharing, and adopting GSI authentication to enable single sing-on. We also develop a Java-based user interface of the storage system allowing Grid users to manage their data transparently without concerning the low-level deployment of storage resources. In the future, we will continue improving our system to make it more powerful and useful. Acknowledgement This work was supported in part by the National Center for High-performance Computing under the national project, Taiwan Knowledge Innovation National Grid, and in part by National Science Council under Contract No. NSC E Figure 10. UniGrid Portal. Reference [1] Taiwan UniGrid, [2] Chaitanya Baru, R. Moore, A. Rajasekar and M. Wan, The SDSC storage resource broker, CASCON '98: Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, Canada, 1998, also available at [3] B. Allcock, A. Chervenak, I. Foster, C. Kesselman, and M. Livny, Data Grid tools: enabling science on big distributed data, Journal of Physics: Conference Series 16, 2005, also available at [4] Y. Zhao, M. Wilde, I. Foster, J. Voeckler, J. Dobson, E. Glibert, T. Jordan, and E. Quigg, Virtual Data Grid Middleware Services for Data-Intensive Science, Concurrency and Computation: Practice & Experience, Vol. 18, Issue 6, 2004, also available at view/vdsweb/webmain [5] Chien-Min Wang, C.C. Hsu, H.M. Chen, J.J. Wu, Efficient multi-source data transfer in data grids, 6 th IEEE International Symposium

8 A High-Performance Virtual Storage System for Taiwan UniGrid on Cluster Computing and the Grid, Singapore, May 2006 [6] I. Foster, C. Kesselman, G. Tsudik, and S. Tuecke, A security architecture for computational grids, In ACM Conference on Computers and Security, pages 83 91, ACM Press, [7] WSDL: Web Services Description Language 1.1. Available at [8] UDDI: Universal Description, Discovery and Integration. Available at [9] SOAP: Simple Object Access Protocol 1.1. Global Grid Forum, available at [10] Bill Allcock, J. Bester, J. Bresnahan, A. L. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke, Data management and transfer in high-performance computational grid environment, Parallel Computing, 28(5): , [11] Kavitha Ranganathan and I. Foster, Design and evaluation of dynamic replication strategies for a high performance data grid, In International Conference on Computing in High Energy and Nuclear Physics, 2001 [12] S. Vazhkudai, S. Tuecke, and I. Foster, Replica selection in the globus data grid, In 1 st International Symposium on Cluster Computing and the Grid, pages , [13] Jun Feng and M. Humphrey, Eliminating Replica Selection - Using Multiple Replicas to Accelerate Data Transfer on Grids, In 10 th International Conference on Parallel and Distributed Systems (ICPADS 2004), pages , [14] C.T. Yang, S.Y. Wang, C.H. Lin, M.H. Lee, and T.Y Wu, Cyber-Transformer: A Toolkit for Files Transfer with Replica Management in Data Grid Environments, In the 2 nd Workshop on Grid Technologies and Applications (WoGTA 05), Taiwan, 2005 [15] Carlisle Adams and Steve Lloyd, Understanding Public-Key Infrastructure: Concepts, Standards, and Deployment Considerations, New Riders Publishing, [16] Ian Foster and C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit, The International Journal of Supercomputer Applications and High Performance Computing, vol. 11, No. 2, pp , [17] B. Allcock, J. Bester, J. Bresnahan, A. L. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke, Data Management and Transfer in High-Performance Computational Grid Environments, Parallel Computing

Web Service Based Data Management for Grid Applications

Web Service Based Data Management for Grid Applications Web Service Based Data Management for Grid Applications T. Boehm Zuse-Institute Berlin (ZIB), Berlin, Germany Abstract Web Services play an important role in providing an interface between end user applications

More information

GridFTP: A Data Transfer Protocol for the Grid

GridFTP: A Data Transfer Protocol for the Grid GridFTP: A Data Transfer Protocol for the Grid Grid Forum Data Working Group on GridFTP Bill Allcock, Lee Liming, Steven Tuecke ANL Ann Chervenak USC/ISI Introduction In Grid environments,

More information

THE CCLRC DATA PORTAL

THE CCLRC DATA PORTAL THE CCLRC DATA PORTAL Glen Drinkwater, Shoaib Sufi CCLRC Daresbury Laboratory, Daresbury, Warrington, Cheshire, WA4 4AD, UK. E-mail: g.j.drinkwater@dl.ac.uk, s.a.sufi@dl.ac.uk Abstract: The project aims

More information

Web Service Robust GridFTP

Web Service Robust GridFTP Web Service Robust GridFTP Sang Lim, Geoffrey Fox, Shrideep Pallickara and Marlon Pierce Community Grid Labs, Indiana University 501 N. Morton St. Suite 224 Bloomington, IN 47404 {sblim, gcf, spallick,

More information

Information Sciences Institute University of Southern California Los Angeles, CA 90292 {annc, carl}@isi.edu

Information Sciences Institute University of Southern California Los Angeles, CA 90292 {annc, carl}@isi.edu _ Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing Bill Allcock 1 Joe Bester 1 John Bresnahan 1 Ann L. Chervenak 2 Ian Foster 1,3 Carl Kesselman 2 Sam

More information

A Survey Study on Monitoring Service for Grid

A Survey Study on Monitoring Service for Grid A Survey Study on Monitoring Service for Grid Erkang You erkyou@indiana.edu ABSTRACT Grid is a distributed system that integrates heterogeneous systems into a single transparent computer, aiming to provide

More information

Archiving, Indexing and Accessing Web Materials: Solutions for large amounts of data

Archiving, Indexing and Accessing Web Materials: Solutions for large amounts of data Archiving, Indexing and Accessing Web Materials: Solutions for large amounts of data David Minor 1, Reagan Moore 2, Bing Zhu, Charles Cowart 4 1. (88)4-104 minor@sdsc.edu San Diego Supercomputer Center

More information

Deploying a distributed data storage system on the UK National Grid Service using federated SRB

Deploying a distributed data storage system on the UK National Grid Service using federated SRB Deploying a distributed data storage system on the UK National Grid Service using federated SRB Manandhar A.S., Kleese K., Berrisford P., Brown G.D. CCLRC e-science Center Abstract As Grid enabled applications

More information

MIGRATING DESKTOP AND ROAMING ACCESS. Migrating Desktop and Roaming Access Whitepaper

MIGRATING DESKTOP AND ROAMING ACCESS. Migrating Desktop and Roaming Access Whitepaper Migrating Desktop and Roaming Access Whitepaper Poznan Supercomputing and Networking Center Noskowskiego 12/14 61-704 Poznan, POLAND 2004, April white-paper-md-ras.doc 1/11 1 Product overview In this whitepaper

More information

Data Management System for grid and portal services

Data Management System for grid and portal services Data Management System for grid and portal services Piotr Grzybowski 1, Cezary Mazurek 1, Paweł Spychała 1, Marcin Wolski 1 1 Poznan Supercomputing and Networking Center, ul. Noskowskiego 10, 61-704 Poznan,

More information

Australian Synchrotron, Storage Gateway

Australian Synchrotron, Storage Gateway Australian Synchrotron, Storage Gateway User Help Manual Version 1.2 Storage Gateway User Help Manual 2 REVISION HISTORY Date Version Description Author 2 May 2008 1.0 Document creation Chris Myers 13

More information

Data Grids. Lidan Wang April 5, 2007

Data Grids. Lidan Wang April 5, 2007 Data Grids Lidan Wang April 5, 2007 Outline Data-intensive applications Challenges in data access, integration and management in Grid setting Grid services for these data-intensive application Architectural

More information

Open DMIX - Data Integration and Exploration Services for Data Grids, Data Web and Knowledge Grid Applications

Open DMIX - Data Integration and Exploration Services for Data Grids, Data Web and Knowledge Grid Applications Open DMIX - Data Integration and Exploration Services for Data Grids, Data Web and Knowledge Grid Applications Robert L. Grossman, Yunhong Gu, Dave Hanley, Xinwei Hong and Gokulnath Rao Laboratory for

More information

Grid Technology and Information Management for Command and Control

Grid Technology and Information Management for Command and Control Grid Technology and Information Management for Command and Control Dr. Scott E. Spetka Dr. George O. Ramseyer* Dr. Richard W. Linderman* ITT Industries Advanced Engineering and Sciences SUNY Institute

More information

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4 Concepts and Architecture of the Grid Summary of Grid 2, Chapter 4 Concepts of Grid Mantra: Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations Allows

More information

An approach to grid scheduling by using Condor-G Matchmaking mechanism

An approach to grid scheduling by using Condor-G Matchmaking mechanism An approach to grid scheduling by using Condor-G Matchmaking mechanism E. Imamagic, B. Radic, D. Dobrenic University Computing Centre, University of Zagreb, Croatia {emir.imamagic, branimir.radic, dobrisa.dobrenic}@srce.hr

More information

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets!! Large data collections appear in many scientific domains like climate studies.!! Users and

More information

A Web Services Data Analysis Grid *

A Web Services Data Analysis Grid * A Web Services Data Analysis Grid * William A. Watson III, Ian Bird, Jie Chen, Bryan Hess, Andy Kowalski, Ying Chen Thomas Jefferson National Accelerator Facility 12000 Jefferson Av, Newport News, VA 23606,

More information

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007 Data Management in an International Data Grid Project Timur Chabuk 04/09/2007 Intro LHC opened in 2005 several Petabytes of data per year data created at CERN distributed to Regional Centers all over the

More information

The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets

The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets Ann Chervenak Ian Foster $+ Carl Kesselman Charles Salisbury $ Steven Tuecke $ Information

More information

Abstract. 1. Introduction. Ohio State University Columbus, OH 43210 {langella,oster,hastings,kurc,saltz}@bmi.osu.edu

Abstract. 1. Introduction. Ohio State University Columbus, OH 43210 {langella,oster,hastings,kurc,saltz}@bmi.osu.edu Dorian: Grid Service Infrastructure for Identity Management and Federation Stephen Langella 1, Scott Oster 1, Shannon Hastings 1, Frank Siebenlist 2, Tahsin Kurc 1, Joel Saltz 1 1 Department of Biomedical

More information

A Web Services Data Analysis Grid *

A Web Services Data Analysis Grid * A Web Services Data Analysis Grid * William A. Watson III, Ian Bird, Jie Chen, Bryan Hess, Andy Kowalski, Ying Chen Thomas Jefferson National Accelerator Facility 12000 Jefferson Av, Newport News, VA 23606,

More information

WebGReIC: Towards ubiquitous grid data management services

WebGReIC: Towards ubiquitous grid data management services Rochester Institute of Technology RIT Scholar Works Articles 2006 WebGReIC: Towards ubiquitous grid data management services Giovanni Aloisio Massimo Cafaro Sandro Fiore Follow this and additional works

More information

Introduction. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Introduction. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Introduction 1 Distributed relating to a computer network in which at least some of the processing is done by the individual computers and information is shared by and often stored at the computers Enabling

More information

Early Cloud Experiences with the Kepler Scientific Workflow System

Early Cloud Experiences with the Kepler Scientific Workflow System Available online at www.sciencedirect.com Procedia Computer Science 9 (2012 ) 1630 1634 International Conference on Computational Science, ICCS 2012 Early Cloud Experiences with the Kepler Scientific Workflow

More information

Grid Computing @ Sun Carlo Nardone. Technical Systems Ambassador GSO Client Solutions

Grid Computing @ Sun Carlo Nardone. Technical Systems Ambassador GSO Client Solutions Grid Computing @ Sun Carlo Nardone Technical Systems Ambassador GSO Client Solutions Phases of Grid Computing Cluster Grids Single user community Single organization Campus Grids Multiple user communities

More information

Grid Computing: A Ten Years Look Back. María S. Pérez Facultad de Informática Universidad Politécnica de Madrid mperez@fi.upm.es

Grid Computing: A Ten Years Look Back. María S. Pérez Facultad de Informática Universidad Politécnica de Madrid mperez@fi.upm.es Grid Computing: A Ten Years Look Back María S. Pérez Facultad de Informática Universidad Politécnica de Madrid mperez@fi.upm.es Outline Challenges not yet solved in computing The parents of grid Computing

More information

Secure Federated Light-weight Web Portals for FusionGrid

Secure Federated Light-weight Web Portals for FusionGrid Secure Federated Light-weight Web Portals for FusionGrid By: D. Aswath, M. Thompson, M. Goode, X. Lee, N. Y. Kim Presented by: Dipti Aswath GCE Workshop 2006 Second International Workshop on Grid Computing

More information

Concepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen

Concepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen Concepts and Architecture of Grid Computing Advanced Topics Spring 2008 Prof. Robert van Engelen Overview Grid users: who are they? Concept of the Grid Challenges for the Grid Evolution of Grid systems

More information

000-596. IBM Security Access Manager for Enterprise Single Sign-On V8.2 Implementation Exam. http://www.examskey.com/000-596.html

000-596. IBM Security Access Manager for Enterprise Single Sign-On V8.2 Implementation Exam. http://www.examskey.com/000-596.html IBM 000-596 IBM Security Access Manager for Enterprise Single Sign-On V8.2 Implementation Exam TYPE: DEMO http://www.examskey.com/000-596.html Examskey IBM 000-596 exam demo product is here for you to

More information

A P2P SERVICE DISCOVERY STRATEGY BASED ON CONTENT

A P2P SERVICE DISCOVERY STRATEGY BASED ON CONTENT A P2P SERVICE DISCOVERY STRATEGY BASED ON CONTENT CATALOGUES Lican Huang Institute of Network & Distributed Computing, Zhejiang Sci-Tech University, No.5, St.2, Xiasha Higher Education Zone, Hangzhou,

More information

PROGRESS Portal Access Whitepaper

PROGRESS Portal Access Whitepaper PROGRESS Portal Access Whitepaper Maciej Bogdanski, Michał Kosiedowski, Cezary Mazurek, Marzena Rabiega, Malgorzata Wolniewicz Poznan Supercomputing and Networking Center April 15, 2004 1 Introduction

More information

A Metadata Catalog Service for Data Intensive Applications

A Metadata Catalog Service for Data Intensive Applications A Metadata Catalog Service for Data Intensive Applications Gurmeet Singh, Shishir Bharathi, Ann Chervenak, Ewa Deelman, Carl Kesselman, Mary Manohar, Sonal Patil, Laura Pearlman Information Sciences Institute,

More information

Monitoring Data Archives for Grid Environments

Monitoring Data Archives for Grid Environments Monitoring Data Archives for Grid Environments Jason Lee, Dan Gunter, Martin Stoufer, Brian Tierney Lawrence Berkeley National Laboratory Abstract Developers and users of high-performance distributed systems

More information

G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids

G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids Martin Placek and Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Lab Department of Computer

More information

Information Sciences Institute University of Southern California Los Angeles, CA 90292 {annc, carl}@isi.edu

Information Sciences Institute University of Southern California Los Angeles, CA 90292 {annc, carl}@isi.edu _ Data Management and Transfer in High-Performance Computational Grid Environments Bill Allcock 1 Joe Bester 1 John Bresnahan 1 Ann L. Chervenak 2 Ian Foster 1,3 Carl Kesselman 2 Sam Meder 1 Veronika Nefedova

More information

Collaborative & Integrated Network & Systems Management: Management Using Grid Technologies

Collaborative & Integrated Network & Systems Management: Management Using Grid Technologies 2011 International Conference on Computer Communication and Management Proc.of CSIT vol.5 (2011) (2011) IACSIT Press, Singapore Collaborative & Integrated Network & Systems Management: Management Using

More information

DEPLOYMENT GUIDE Version 1.0. Deploying the BIG-IP Edge Gateway for Layered Security and Acceleration Services

DEPLOYMENT GUIDE Version 1.0. Deploying the BIG-IP Edge Gateway for Layered Security and Acceleration Services DEPLOYMENT GUIDE Version 1.0 Deploying the BIG-IP Edge Gateway for Layered Security and Acceleration Services Table of Contents Table of Contents Using the BIG-IP Edge Gateway for layered security and

More information

It takes know-how to retrieve large files over public networks

It takes know-how to retrieve large files over public networks It takes know-how to retrieve large files over public networks Adam H. Villa and Elizabeth Varki University of New Hampshire Department of Computer Science Durham, NH, 03824 USA Abstract Retrieving large

More information

1 Introduction to Microsoft Enterprise Desktop Virtualization (MED-V)... 3 1.1 Terminology... 4 1.2 Key Capabilities... 4

1 Introduction to Microsoft Enterprise Desktop Virtualization (MED-V)... 3 1.1 Terminology... 4 1.2 Key Capabilities... 4 MED-V v1 Contents 1 Introduction to Microsoft Enterprise Desktop Virtualization (MED-V)... 3 1.1 Terminology... 4 1.2 Key Capabilities... 4 2 High-level Architecture... 6 2.1 System Requirements for MED-V

More information

Diagram 1: Islands of storage across a digital broadcast workflow

Diagram 1: Islands of storage across a digital broadcast workflow XOR MEDIA CLOUD AQUA Big Data and Traditional Storage The era of big data imposes new challenges on the storage technology industry. As companies accumulate massive amounts of data from video, sound, database,

More information

2 Transport-level and Message-level Security

2 Transport-level and Message-level Security Globus Toolkit Version 4 Grid Security Infrastructure: A Standards Perspective The Globus Security Team 1 Version 4 updated September 12, 2005 Abstract This document provides an overview of the Grid Security

More information

Multilingual Interface for Grid Market Directory Services: An Experience with Supporting Tamil

Multilingual Interface for Grid Market Directory Services: An Experience with Supporting Tamil Multilingual Interface for Grid Market Directory Services: An Experience with Supporting Tamil S.Thamarai Selvi *, Rajkumar Buyya **, M.R. Rajagopalan #, K.Vijayakumar *, G.N.Deepak * * Department of Information

More information

Grid Computing With FreeBSD

Grid Computing With FreeBSD Grid Computing With FreeBSD USENIX ATC '04: UseBSD SIG Boston, MA, June 29 th 2004 Brooks Davis, Craig Lee The Aerospace Corporation El Segundo, CA {brooks,lee}aero.org http://people.freebsd.org/~brooks/papers/usebsd2004/

More information

PoS(ISGC 2013)021. SCALA: A Framework for Graphical Operations for irods. Wataru Takase KEK E-mail: wataru.takase@kek.jp

PoS(ISGC 2013)021. SCALA: A Framework for Graphical Operations for irods. Wataru Takase KEK E-mail: wataru.takase@kek.jp SCALA: A Framework for Graphical Operations for irods KEK E-mail: wataru.takase@kek.jp Adil Hasan University of Liverpool E-mail: adilhasan2@gmail.com Yoshimi Iida KEK E-mail: yoshimi.iida@kek.jp Francesca

More information

Modeling for Web-based Image Processing and JImaging System Implemented Using Medium Model

Modeling for Web-based Image Processing and JImaging System Implemented Using Medium Model Send Orders for Reprints to reprints@benthamscience.ae 142 The Open Cybernetics & Systemics Journal, 2015, 9, 142-147 Open Access Modeling for Web-based Image Processing and JImaging System Implemented

More information

Features of AnyShare

Features of AnyShare of AnyShare of AnyShare CONTENT Brief Introduction of AnyShare... 3 Chapter 1 Centralized Management... 5 1.1 Operation Management... 5 1.2 User Management... 5 1.3 User Authentication... 6 1.4 Roles...

More information

SSL VPN Technology White Paper

SSL VPN Technology White Paper SSL VPN Technology White Paper Keywords: SSL VPN, HTTPS, Web access, TCP access, IP access Abstract: SSL VPN is an emerging VPN technology based on HTTPS. This document describes its implementation and

More information

A Federated Grid Environment with Replication Services

A Federated Grid Environment with Replication Services A Federated Grid Environment with Replication Services Vivek Khurana, Max Berger & Michael Sobolewski SORCER Research Group, Texas Tech University Grids can be classified as computational grids, access

More information

Gladinet Cloud Backup V3.0 User Guide

Gladinet Cloud Backup V3.0 User Guide Gladinet Cloud Backup V3.0 User Guide Foreword The Gladinet User Guide gives step-by-step instructions for end users. Revision History Gladinet User Guide Date Description Version 8/20/2010 Draft Gladinet

More information

Grid Computing Vs. Cloud Computing

Grid Computing Vs. Cloud Computing International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 6 (2013), pp. 577-582 International Research Publications House http://www. irphouse.com /ijict.htm Grid

More information

Authentication Mechanism for Private Cloud of Enterprise. Abstract

Authentication Mechanism for Private Cloud of Enterprise. Abstract Authentication Mechanism for Private Cloud of Enterprise Mei-Yu Wu *, and Shih-Pin Lo Department of Information Management, Chung Hua University, Hsinchu, Taiwan {mywu, e10010008}@chu.edu.tw Abstract Enterprises

More information

GridCopy: Moving Data Fast on the Grid

GridCopy: Moving Data Fast on the Grid GridCopy: Moving Data Fast on the Grid Rajkumar Kettimuthu 1,2, William Allcock 1,2, Lee Liming 1,2 John-Paul Navarro 1,2, Ian Foster 1,2,3 1 Mathematics and Computer Science Division Argonne National

More information

TIBCO Spotfire Platform IT Brief

TIBCO Spotfire Platform IT Brief Platform IT Brief This IT brief outlines features of the system: Communication security, load balancing and failover, authentication options, and recommended practices for licenses and access. It primarily

More information

16th International Conference on Control Systems and Computer Science (CSCS16 07)

16th International Conference on Control Systems and Computer Science (CSCS16 07) 16th International Conference on Control Systems and Computer Science (CSCS16 07) TOWARDS AN IO INTENSIVE GRID APPLICATION INSTRUMENTATION IN MEDIOGRID Dacian Tudor 1, Florin Pop 2, Valentin Cristea 2,

More information

Taking Big Data to the Cloud. Enabling cloud computing & storage for big data applications with on-demand, high-speed transport WHITE PAPER

Taking Big Data to the Cloud. Enabling cloud computing & storage for big data applications with on-demand, high-speed transport WHITE PAPER Taking Big Data to the Cloud WHITE PAPER TABLE OF CONTENTS Introduction 2 The Cloud Promise 3 The Big Data Challenge 3 Aspera Solution 4 Delivering on the Promise 4 HIGHLIGHTS Challenges Transporting large

More information

Globus Striped GridFTP Framework and Server. Raj Kettimuthu, ANL and U. Chicago

Globus Striped GridFTP Framework and Server. Raj Kettimuthu, ANL and U. Chicago Globus Striped GridFTP Framework and Server Raj Kettimuthu, ANL and U. Chicago Outline Introduction Features Motivation Architecture Globus XIO Experimental Results 3 August 2005 The Ohio State University

More information

BlackBerry Enterprise Service 10. Version: 10.2. Configuration Guide

BlackBerry Enterprise Service 10. Version: 10.2. Configuration Guide BlackBerry Enterprise Service 10 Version: 10.2 Configuration Guide Published: 2015-02-27 SWD-20150227164548686 Contents 1 Introduction...7 About this guide...8 What is BlackBerry Enterprise Service 10?...9

More information

2. Requirements for Metadata Management on the Grid

2. Requirements for Metadata Management on the Grid Grid-Based Metadata Services Ewa Deelman 1, Gurmeet Singh 1, Malcolm P. Atkinson 2, Ann Chervenak 1, Neil P Chue Hong 3, Carl Kesselman 1, Sonal Patil 1, Laura Pearlman 1, Mei-Hui Su 1 1 Information Sciences

More information

Configuration Guide BES12. Version 12.2

Configuration Guide BES12. Version 12.2 Configuration Guide BES12 Version 12.2 Published: 2015-07-07 SWD-20150630131852557 Contents About this guide... 8 Getting started... 9 Administrator permissions you need to configure BES12... 9 Obtaining

More information

Digital libraries of the future and the role of libraries

Digital libraries of the future and the role of libraries Digital libraries of the future and the role of libraries Donatella Castelli ISTI-CNR, Pisa, Italy Abstract Purpose: To introduce the digital libraries of the future, their enabling technologies and their

More information

Design of Data Archive in Virtual Test Architecture

Design of Data Archive in Virtual Test Architecture Journal of Information Hiding and Multimedia Signal Processing 2014 ISSN 2073-4212 Ubiquitous International Volume 5, Number 1, January 2014 Design of Data Archive in Virtual Test Architecture Lian-Lei

More information

Content Distribution Management

Content Distribution Management Digitizing the Olympics was truly one of the most ambitious media projects in history, and we could not have done it without Signiant. We used Signiant CDM to automate 54 different workflows between 11

More information

AquaLogic Service Bus

AquaLogic Service Bus AquaLogic Bus Wolfgang Weigend Principal Systems Engineer BEA Systems 1 What to consider when looking at ESB? Number of planned business access points Reuse across organization Reduced cost of ownership

More information

CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL

CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL This chapter is to introduce the client-server model and its role in the development of distributed network systems. The chapter

More information

Presentation of Multi Level Data Replication Distributed Decision Making Strategy for High Priority Tasks in Real Time Data Grids

Presentation of Multi Level Data Replication Distributed Decision Making Strategy for High Priority Tasks in Real Time Data Grids Presentation of Multi Level Data Replication Distributed Decision Making Strategy for High Priority Tasks in Real Time Data Grids Naghmeh Esmaieli Esmaily.naghmeh@gmail.com Mahdi Jafari Ser_jafari@yahoo.com

More information

Tenrox. Single Sign-On (SSO) Setup Guide. January, 2012. 2012 Tenrox. All rights reserved.

Tenrox. Single Sign-On (SSO) Setup Guide. January, 2012. 2012 Tenrox. All rights reserved. Tenrox Single Sign-On (SSO) Setup Guide January, 2012 2012 Tenrox. All rights reserved. About this Guide This guide provides a high-level technical overview of the Tenrox Single Sign-On (SSO) architecture,

More information

elearning Content Management Middleware

elearning Content Management Middleware elearning Content Management Middleware Chen Zhao Helsinki 18.2.2004 University of Helsinki Department of Computer Science Authors Chen Zhao Title elearning Content Management Middleware Date 18.2.2004

More information

DFSgc. Distributed File System for Multipurpose Grid Applications and Cloud Computing

DFSgc. Distributed File System for Multipurpose Grid Applications and Cloud Computing DFSgc Distributed File System for Multipurpose Grid Applications and Cloud Computing Introduction to DFSgc. Motivation: Grid Computing currently needs support for managing huge quantities of storage. Lacks

More information

Monitoring Hybrid Cloud Applications in VMware vcloud Air

Monitoring Hybrid Cloud Applications in VMware vcloud Air Monitoring Hybrid Cloud Applications in ware vcloud Air ware vcenter Hyperic and ware vcenter Operations Manager Installation and Administration Guide for Hybrid Cloud Monitoring TECHNICAL WHITE PAPER

More information

GSiB: PSE Infrastructure for Dynamic Service-oriented Grid Applications

GSiB: PSE Infrastructure for Dynamic Service-oriented Grid Applications GSiB: PSE Infrastructure for Dynamic Service-oriented Grid Applications Yan Huang Department of Computer Science Cardiff University PO Box 916 Cardiff CF24 3XF United Kingdom Yan.Huang@cs.cardiff.ac.uk

More information

Classic Grid Architecture

Classic Grid Architecture Peer-to to-peer Grids Classic Grid Architecture Resources Database Database Netsolve Collaboration Composition Content Access Computing Security Middle Tier Brokers Service Providers Middle Tier becomes

More information

Grid Delegation Protocol

Grid Delegation Protocol UK Workshop on Grid Security Experiences, Oxford 8th and 9th July 2004 Grid Delegation Protocol Mehran Ahsant a, Jim Basney b and Olle Mulmo a a Center for Parallel Computers,Royal Institute of Technology,

More information

Development of a file-sharing system for educational collaboration among higher-education institutions

Development of a file-sharing system for educational collaboration among higher-education institutions Development of a file-sharing system for educational collaboration among higher-education institutions Takuya Matsuhira, Yoshiya Kasahara, and Yoshihiro Takata Abstract Opportunities for educational, research-oriented,

More information

A File Transfer Component for Grids

A File Transfer Component for Grids A File Transfer Component for Grids Gregor von Laszewski Mathematics and Computer Science Division, Argonne National Laboratory Argonne, Il 60440, U.S.A. Alunkal Beulah Kurian Mathematics and Computer

More information

Sector vs. Hadoop. A Brief Comparison Between the Two Systems

Sector vs. Hadoop. A Brief Comparison Between the Two Systems Sector vs. Hadoop A Brief Comparison Between the Two Systems Background Sector is a relatively new system that is broadly comparable to Hadoop, and people want to know what are the differences. Is Sector

More information

Writing Grid Service Using GT3 Core. Dec, 2003. Abstract

Writing Grid Service Using GT3 Core. Dec, 2003. Abstract Writing Grid Service Using GT3 Core Dec, 2003 Long Wang wangling@mail.utexas.edu Department of Electrical & Computer Engineering The University of Texas at Austin James C. Browne browne@cs.utexas.edu Department

More information

Cisco Wide Area Application Services Optimizes Application Delivery from the Cloud

Cisco Wide Area Application Services Optimizes Application Delivery from the Cloud Cisco Wide Area Application Services Optimizes Application Delivery from the Cloud What You Will Learn The adoption of cloud-based computing and applications promises to improve the agility, efficiency,

More information

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

Digital Library for Multimedia Content Management

Digital Library for Multimedia Content Management Digital Library for Multimedia Content Management Cezary Mazurek, Maciej Stroinski, Sebastian Szuber Pozna_ Supercomputing and Networking Centre, ul. Noskowskiego 10, 61-704 Pozna_, POLAND tel. +48 61

More information

How to Install Microsoft Mobile Information Server 2002 Server ActiveSync. Joey Masterson

How to Install Microsoft Mobile Information Server 2002 Server ActiveSync. Joey Masterson How to Install Microsoft Mobile Information Server 2002 Server ActiveSync Joey Masterson How to Install Microsoft Mobile Information Server 2002 Server ActiveSync Joey Masterson Copyright Information

More information

Towards Autonomic Grid Data Management with Virtualized Distributed File Systems

Towards Autonomic Grid Data Management with Virtualized Distributed File Systems Towards Autonomic Grid Data Management with Virtualized Distributed File Systems Ming Zhao, Jing Xu, Renato Figueiredo Advanced Computing and Information Systems Electrical and Computer Engineering University

More information

VMware vsphere Data Protection Evaluation Guide REVISED APRIL 2015

VMware vsphere Data Protection Evaluation Guide REVISED APRIL 2015 VMware vsphere Data Protection REVISED APRIL 2015 Table of Contents Introduction.... 3 Features and Benefits of vsphere Data Protection... 3 Requirements.... 4 Evaluation Workflow... 5 Overview.... 5 Evaluation

More information

Setting Up Resources in VMware Identity Manager

Setting Up Resources in VMware Identity Manager Setting Up Resources in VMware Identity Manager VMware Identity Manager 2.4 This document supports the version of each product listed and supports all subsequent versions until the document is replaced

More information

Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources

Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources Asif Saleem Marko Krznarić Jeremy Cohen Steven Newhouse John Darlington London e-science Centre,

More information

The glite File Transfer Service

The glite File Transfer Service The glite File Transfer Service Peter Kunszt Paolo Badino Ricardo Brito da Rocha James Casey Ákos Frohner Gavin McCance CERN, IT Department 1211 Geneva 23, Switzerland Abstract Transferring data reliably

More information

Setup Guide Access Manager Appliance 3.2 SP3

Setup Guide Access Manager Appliance 3.2 SP3 Setup Guide Access Manager Appliance 3.2 SP3 August 2014 www.netiq.com/documentation Legal Notice THIS DOCUMENT AND THE SOFTWARE DESCRIBED IN THIS DOCUMENT ARE FURNISHED UNDER AND ARE SUBJECT TO THE TERMS

More information

Object Request Reduction in Home Nodes and Load Balancing of Object Request in Hybrid Decentralized Web Caching

Object Request Reduction in Home Nodes and Load Balancing of Object Request in Hybrid Decentralized Web Caching 2012 2 nd International Conference on Information Communication and Management (ICICM 2012) IPCSIT vol. 55 (2012) (2012) IACSIT Press, Singapore DOI: 10.7763/IPCSIT.2012.V55.5 Object Request Reduction

More information

GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid

GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid Wantao Liu, 1,2 Rajkumar Kettimuthu, 3,4 Brian Tieman, 5 Ravi Madduri, 3,4 Bo Li, 1 Ian Foster 2,3,4 1 School of Computer Science and Engineering,

More information

XSEDE Service Provider Software and Services Baseline. September 24, 2015 Version 1.2

XSEDE Service Provider Software and Services Baseline. September 24, 2015 Version 1.2 XSEDE Service Provider Software and Services Baseline September 24, 2015 Version 1.2 i TABLE OF CONTENTS XSEDE Production Baseline: Service Provider Software and Services... i A. Document History... A-

More information

DiPerF: automated DIstributed PERformance testing Framework

DiPerF: automated DIstributed PERformance testing Framework DiPerF: automated DIstributed PERformance testing Framework Ioan Raicu, Catalin Dumitrescu, Matei Ripeanu Distributed Systems Laboratory Computer Science Department University of Chicago Ian Foster Mathematics

More information

Oracle Service Bus Examples and Tutorials

Oracle Service Bus Examples and Tutorials March 2011 Contents 1 Oracle Service Bus Examples... 2 2 Introduction to the Oracle Service Bus Tutorials... 5 3 Getting Started with the Oracle Service Bus Tutorials... 12 4 Tutorial 1. Routing a Loan

More information

Globus XIO Pipe Open Driver: Enabling GridFTP to Leverage Standard Unix Tools

Globus XIO Pipe Open Driver: Enabling GridFTP to Leverage Standard Unix Tools Globus XIO Pipe Open Driver: Enabling GridFTP to Leverage Standard Unix Tools Rajkumar Kettimuthu 1. Steven Link 2. John Bresnahan 1. Michael Link 1. Ian Foster 1,3 1 Computation Institute 2 Department

More information

Interwise Connect. Working with Reverse Proxy Version 7.x

Interwise Connect. Working with Reverse Proxy Version 7.x Working with Reverse Proxy Version 7.x Table of Contents BACKGROUND...3 Single Sign On (SSO)... 3 Interwise Connect... 3 INTERWISE CONNECT WORKING WITH REVERSE PROXY...4 Architecture... 4 Interwise Web

More information

WAN Optimization, Web Cache, Explicit Proxy, and WCCP. FortiOS Handbook v3 for FortiOS 4.0 MR3

WAN Optimization, Web Cache, Explicit Proxy, and WCCP. FortiOS Handbook v3 for FortiOS 4.0 MR3 WAN Optimization, Web Cache, Explicit Proxy, and WCCP FortiOS Handbook v3 for FortiOS 4.0 MR3 FortiOS Handbook WAN Optimization, Web Cache, Explicit Proxy, and WCCP v3 13 January 2012 01-433-96996-20120113

More information

KNOWLEDGE GRID An Architecture for Distributed Knowledge Discovery

KNOWLEDGE GRID An Architecture for Distributed Knowledge Discovery KNOWLEDGE GRID An Architecture for Distributed Knowledge Discovery Mario Cannataro 1 and Domenico Talia 2 1 ICAR-CNR 2 DEIS Via P. Bucci, Cubo 41-C University of Calabria 87036 Rende (CS) Via P. Bucci,

More information

File and Object Replication in Data Grids

File and Object Replication in Data Grids File and Object Replication in Data Grids Heinz Stockinger 1,2, Asad Samar 3, Bill Allcock 4, Ian Foster 4,5, Koen Holtman 3, Brian Tierney 1,6 1) CERN, European Organization for Nuclear Research, CH-1211

More information

GENERIC DATA ACCESS AND INTEGRATION SERVICE FOR DISTRIBUTED COMPUTING ENVIRONMENT

GENERIC DATA ACCESS AND INTEGRATION SERVICE FOR DISTRIBUTED COMPUTING ENVIRONMENT GENERIC DATA ACCESS AND INTEGRATION SERVICE FOR DISTRIBUTED COMPUTING ENVIRONMENT Hemant Mehta 1, Priyesh Kanungo 2 and Manohar Chandwani 3 1 School of Computer Science, Devi Ahilya University, Indore,

More information

BlackBerry Enterprise Service 10. Secure Work Space for ios and Android Version: 10.1.1. Security Note

BlackBerry Enterprise Service 10. Secure Work Space for ios and Android Version: 10.1.1. Security Note BlackBerry Enterprise Service 10 Secure Work Space for ios and Android Version: 10.1.1 Security Note Published: 2013-06-21 SWD-20130621110651069 Contents 1 About this guide...4 2 What is BlackBerry Enterprise

More information

SiteCelerate white paper

SiteCelerate white paper SiteCelerate white paper Arahe Solutions SITECELERATE OVERVIEW As enterprises increases their investment in Web applications, Portal and websites and as usage of these applications increase, performance

More information