WP5 Progress Report. John Gordon EDG Conference Barcelona, May escience Centre

Size: px
Start display at page:

Download "WP5 Progress Report. John Gordon EDG Conference Barcelona, May 2003. escience Centre"

Transcription

1 WP5 Progress Report EDG Conference Barcelona, May 2003 escience Centre

2 Outline Status of release 2.0 Testing and support structures Priorities up to TB3 Open issues

3 Status of release 2.0 Core SE Software Interfaces to various MSS CLI Built from C programme creating and transferring XML to core via Apache Web Service Developed using AXIS and Tomcat Now using WP2 build system Java API, CLI, demo Reptor interface getsecosts for Optor Prototype via MDS, now a webservice GLUE Storage Schema information providers

4 Release 2.0 SRM-like API Needs more work before it will interwork with other implementations of SRM Layered beneath Replica Manager API and CLI can be used directly by users Not a simple interface eg create/getturl/copy/commit cache/getturl/copy/abandon Wrapped by rm-commands

5 Release 2.0 MSS supported Castor, RAL DataStore, Disk Coming soon HPSS via RFIO (CC-IN2P3) Started discussions with ESA/ESRIN, SARA, (INSA/WP10) Ready to talk to other Castor sites

6 Testing and support structures Handlers unit tested Test harness for individual functions Uses curl to send sample XML without client Webservice test suite for API Limited performance tests Create=0.3sec using CLI on modest PC Total transaction time dominated by data transfer Need stress tests

7 Testing and support structures Q staff months delivered to WP5 ~ 7FTE of which 2 working on Castor

8 WP5 roles WP Manager PTB, WPM, emergency meetings, crisis meetings, emergency crisis meetings. ATF Iteam - 2 people now Packaging, configuration, CVS, autobuild QAG Internal roles Documentation, maintaining web site Not much left for coding or support from 5 FTE

9 Testing and support structures Q staff months delivered to WP5 ~ 7FTE of which 2 working on Castor Support structure is Owen assigns Bugzilla reports Individuals support an area of SE Security (Jens), Handlers (Regina), Core (Owen) Information providers (Regina), webservice (Glen), API (Jens) No guarantee we have a second person

10 Priorities up to TB3 Address some deficiencies now Would like release 2.1 for LCG1 Rest for TB3.0

11 TB2.1 Improve error handling RFIO TURL Non-blocking calls Generate request_id and GetStatus SRM v1.1 single file use For LCG1 Native data transfer Eg GridFTP support in Castor allows SE to return a TURL that points direct to Castor

12 TB3.0 Security Fuller SRM File instance metadata Required for pinning

13 Security Webservice will use edg-trust-manager (+authorisation manager) Gives access to user DN now and VOMS credentials later Implement file and transaction level ACLs Can this user access this file in this mode? Can this user perform this action? Intend to use GACL

14 GACL From Andrew McNab V3 supports VOMS credentials in ACLs ACL can use DN, groups, roles Already have a hook in file metadata for ACL/file Implemented a prototype handler with GACL v2 library but it didn t make TB2.0 cut Just add check_acl handler to sequence of handlers called for each function Initial step to implement checking only sysadmins can add ACLs to VO tree by hand (or rpm) Later add API to manipulate ACLs

15 Open issues Webservice only? Data transfer? Posix I/O? SRMcopy requires proxy delegation. Can we do this? How do we replicate ACLs with files? Where do we get default ACLs for a VO? What is the trust tree for ACLs? Ie who do we allow to define ACLs for a VO? When does the emergency stop?

Analisi di un servizio SRM: StoRM

Analisi di un servizio SRM: StoRM 27 November 2007 General Parallel File System (GPFS) The StoRM service Deployment configuration Authorization and ACLs Conclusions. Definition of terms Definition of terms 1/2 Distributed File System The

More information

Plateforme de Calcul pour les Sciences du Vivant. SRB & glite. V. Breton. http://clrpcsv.in2p3.fr

Plateforme de Calcul pour les Sciences du Vivant. SRB & glite. V. Breton. http://clrpcsv.in2p3.fr SRB & glite V. Breton http://clrpcsv.in2p3.fr Introduction Goal: evaluation of existing technologies for data and tools integration and deployment Data and tools integration should be addressed using web

More information

The dcache Storage Element

The dcache Storage Element 16. Juni 2008 Hamburg The dcache Storage Element and it's role in the LHC era for the dcache team Topics for today Storage elements (SEs) in the grid Introduction to the dcache SE Usage of dcache in LCG

More information

1967-35. Advanced School in High Performance and GRID Computing. 3-14 November 2008

1967-35. Advanced School in High Performance and GRID Computing. 3-14 November 2008 1967-35 Advanced School in High Performance and GRID Computing 3-14 November 2008 Grid tutorial hands-on session on data management and advanced job submission TAFFONI Giuliano Osservatorio Astronomico

More information

The glite File Transfer Service

The glite File Transfer Service Enabling Grids Enabling for E-sciencE Grids for E-sciencE The glite File Transfer Service Paolo Badino On behalf of the JRA1 Data Management team EGEE User Forum - CERN, 2 Mars 2006 www.eu-egee.org Outline

More information

Managed Storage @ GRID or why NFSv4.1 is not enough. Tigran Mkrtchyan for dcache Team

Managed Storage @ GRID or why NFSv4.1 is not enough. Tigran Mkrtchyan for dcache Team Managed Storage @ GRID or why NFSv4.1 is not enough Tigran Mkrtchyan for dcache Team What the hell do physicists do? Physicist are hackers they just want to know how things works. In moder physics given

More information

MIGRATING DESKTOP AND ROAMING ACCESS. Migrating Desktop and Roaming Access Whitepaper

MIGRATING DESKTOP AND ROAMING ACCESS. Migrating Desktop and Roaming Access Whitepaper Migrating Desktop and Roaming Access Whitepaper Poznan Supercomputing and Networking Center Noskowskiego 12/14 61-704 Poznan, POLAND 2004, April white-paper-md-ras.doc 1/11 1 Product overview In this whitepaper

More information

glibrary: Digital Asset Management System for the Grid

glibrary: Digital Asset Management System for the Grid glibrary: Digital Asset Management System for the Grid Antonio Calanducci INFN Catania EGEE User Forum Manchester, 09 th -11 th May 2007 www.eu-egee.org EGEE and glite are registered trademarks Outline

More information

Data management in EGEE

Data management in EGEE Home Search Collections Journals About Contact us My IOPscience Data management in EGEE This content has been downloaded from IOPscience. Please scroll down to see the full text. 2010 J. Phys.: Conf. Ser.

More information

GridSite security update

GridSite security update GridSite security update Andrew McNab University of Manchester Outline Credential types Attribute URIs New chain checking API clarifications Logging Level of Assurance Shibboleth GridSite GridSite consists

More information

Hadoop@LaTech ATLAS Tier 3

Hadoop@LaTech ATLAS Tier 3 Cerberus Hadoop Hadoop@LaTech ATLAS Tier 3 David Palma DOSAR Louisiana Tech University January 23, 2013 Cerberus Hadoop Outline 1 Introduction Cerberus Hadoop 2 Features Issues Conclusions 3 Cerberus Hadoop

More information

dcache, a managed storage in grid

dcache, a managed storage in grid dcache, a managed storage in grid support and funding by Patrick for the dcache Team Topics Project Topology Why do we need storage elements in the grid world? The idea behind the LCG (glite) storage element.

More information

Experiences with the GLUE information schema in the LCG/EGEE production Grid

Experiences with the GLUE information schema in the LCG/EGEE production Grid Experiences with the GLUE information schema in the LCG/EGEE production Grid Stephen Burke, Sergio Andreozzi and Laurence Field CHEP07, Victoria, Canada www.eu-egee.org EGEE and glite are registered trademarks

More information

GSI with OpenSSL. Vincenzo Ciaschini. Prague, 4-7/11/08. www.eu-egee.org. EGEE and glite are registered trademarks. egee EGEE-II INFSO-RI-031688

GSI with OpenSSL. Vincenzo Ciaschini. Prague, 4-7/11/08. www.eu-egee.org. EGEE and glite are registered trademarks. egee EGEE-II INFSO-RI-031688 GSI with OpenSSL Vincenzo Ciaschini EGEE-3 All-Hands Prague, 4-7/11/08 www.eu-egee.org egee EGEE and glite are registered trademarks Layout GSI/SSL Differences and Issues VOMS without Globus GSI/SSL Differences

More information

Data Management System for grid and portal services

Data Management System for grid and portal services Data Management System for grid and portal services Piotr Grzybowski 1, Cezary Mazurek 1, Paweł Spychała 1, Marcin Wolski 1 1 Poznan Supercomputing and Networking Center, ul. Noskowskiego 10, 61-704 Poznan,

More information

Open Source Backup with Amanda

Open Source Backup with Amanda Open Source Backup with Amanda Peninsula Linux Users Group (Jan 2008) Paddy Sreenivasan paddy@zmanda.com Copyright 2007 Zmanda, Inc. All rights reserved. 1 Amanda network backup and recovery Easy to use

More information

THE CCLRC DATA PORTAL

THE CCLRC DATA PORTAL THE CCLRC DATA PORTAL Glen Drinkwater, Shoaib Sufi CCLRC Daresbury Laboratory, Daresbury, Warrington, Cheshire, WA4 4AD, UK. E-mail: g.j.drinkwater@dl.ac.uk, s.a.sufi@dl.ac.uk Abstract: The project aims

More information

Chapter 12 Distributed Storage

Chapter 12 Distributed Storage Chapter 12 Distributed Storage 1 2 Files File location and addressing What is a file? Normally we collapse. Concepts: name; contents; gui. What about the backup of this file? How do we distinguish? File

More information

Michael Thomas, Dorian Kcira California Institute of Technology. CMS Offline & Computing Week

Michael Thomas, Dorian Kcira California Institute of Technology. CMS Offline & Computing Week Michael Thomas, Dorian Kcira California Institute of Technology CMS Offline & Computing Week San Diego, April 20-24 th 2009 Map-Reduce plus the HDFS filesystem implemented in java Map-Reduce is a highly

More information

DataNet Flexible Metadata Overlay over File Resources

DataNet Flexible Metadata Overlay over File Resources 1 DataNet Flexible Metadata Overlay over File Resources Daniel Harężlak 1, Marek Kasztelnik 1, Maciej Pawlik 1, Bartosz Wilk 1, Marian Bubak 1,2 1 ACC Cyfronet AGH, 2 AGH University of Science and Technology,

More information

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets!! Large data collections appear in many scientific domains like climate studies.!! Users and

More information

Oracle Communications Cartridge Feature Specification for Broadsoft Broadworks Enterprise Services

Oracle Communications Cartridge Feature Specification for Broadsoft Broadworks Enterprise Services Oracle Communications ASAP Cartridge for Broadsoft Broadworks Enterprise s Cartridge Version 1.0 Oracle Communications Cartridge Feature Specification for Broadsoft Broadworks Enterprise s Vendor: BroadSoft(BS)

More information

GlassFish v3. Building an ex tensible modular Java EE application server. Jerome Dochez and Ludovic Champenois Sun Microsystems, Inc.

GlassFish v3. Building an ex tensible modular Java EE application server. Jerome Dochez and Ludovic Champenois Sun Microsystems, Inc. GlassFish v3 Building an ex tensible modular Java EE application server Jerome Dochez and Ludovic Champenois Sun Microsystems, Inc. Agenda Java EE 6 and GlassFish V3 Modularity, Runtime Service Based Architecture

More information

EUROPEAN MIDDLEWARE INITIATIVE

EUROPEAN MIDDLEWARE INITIATIVE EUROPEAN MIDDLEWARE INITIATIVE GRIDSITE ADMINISTRATOR GUIDE Document version: 2.0.0-1 EMI Component Version: 1.x, 2.x 1/9 GridSite Administrator Guide This work is co-funded by the European Commission

More information

dcache - Managed Storage - LCG Storage Element - HSM optimizer Patrick Fuhrmann, DESY for the dcache Team

dcache - Managed Storage - LCG Storage Element - HSM optimizer Patrick Fuhrmann, DESY for the dcache Team dcache - Managed Storage - LCG Storage Element - HSM optimizer, DESY for the dcache Team dcache is a joint effort between the Deutsches Elektronen Synchrotron (DESY) and the Fermi National Laboratory (FNAL)

More information

Data-Intensive Programming. Timo Aaltonen Department of Pervasive Computing

Data-Intensive Programming. Timo Aaltonen Department of Pervasive Computing Data-Intensive Programming Timo Aaltonen Department of Pervasive Computing Data-Intensive Programming Lecturer: Timo Aaltonen University Lecturer timo.aaltonen@tut.fi Assistants: Henri Terho and Antti

More information

GridSite 1.5.x update

GridSite 1.5.x update GridSite 1.5.x update Andrew McNab University of Manchester Outline Credential types Attribute URIs New chain checking API clarifications Logging Level of Assurance Shibboleth SlashGrid Credential types

More information

The glite File Transfer Service

The glite File Transfer Service The glite File Transfer Service Peter Kunszt Paolo Badino Ricardo Brito da Rocha James Casey Ákos Frohner Gavin McCance CERN, IT Department 1211 Geneva 23, Switzerland Abstract Transferring data reliably

More information

EDG Project: Database Management Services

EDG Project: Database Management Services EDG Project: Database Management Services Leanne Guy for the EDG Data Management Work Package EDG::WP2 Leanne.Guy@cern.ch http://cern.ch/leanne 17 April 2002 DAI Workshop Presentation 1 Information in

More information

Chapter 6, The Operating System Machine Level

Chapter 6, The Operating System Machine Level Chapter 6, The Operating System Machine Level 6.1 Virtual Memory 6.2 Virtual I/O Instructions 6.3 Virtual Instructions For Parallel Processing 6.4 Example Operating Systems 6.5 Summary Virtual Memory General

More information

Software Architecture Document

Software Architecture Document Software Architecture Document Natural Language Processing Cell Version 1.0 Natural Language Processing Cell Software Architecture Document Version 1.0 1 1. Table of Contents 1. Table of Contents... 2

More information

Introduction to HDFS. Prasanth Kothuri, CERN

Introduction to HDFS. Prasanth Kothuri, CERN Prasanth Kothuri, CERN 2 What s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand. HDFS is the primary distributed storage for Hadoop applications. HDFS

More information

Software Architecture Document

Software Architecture Document Software Architecture Document Project Management Cell 1.0 1 of 16 Abstract: This is a software architecture document for Project Management(PM ) cell. It identifies and explains important architectural

More information

Qt and Cloud Services. Sami Makkonen Qt R&D Digia

Qt and Cloud Services. Sami Makkonen Qt R&D Digia Qt and Cloud Services Sami Makkonen Qt R&D Digia Content Different types of Cloud services Qt and Cloud Services Cloud API for Qt Using PaaS Services from Qt application engin.io Using BaaS from Qt application

More information

Direct Issuance of Proxy Certificate on P-GRADE Grid Portal Without Using MyProxy

Direct Issuance of Proxy Certificate on P-GRADE Grid Portal Without Using MyProxy Direct Issuance of Proxy Certificate on P-GRADE Grid Portal Without Using MyProxy by Ng Kang Siong (ksng@mimos.my) Galoh Rashidah Haron (rashidah@mimos.my) MIMOS Berhad, Malaysia www.eu-egee.org EGEE and

More information

File Transfer Software and Service SC3

File Transfer Software and Service SC3 File Transfer Software and Service SC3 Gavin McCance JRA1 Data Management Cluster Service Challenge Meeting April 26 2005, Taipei www.eu-egee.org Outline Overview of Components Tier-0 / Tier-1 / Tier-2

More information

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Global Grid User Support - GGUS - within the LCG & EGEE environment

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Global Grid User Support - GGUS - within the LCG & EGEE environment Global Grid User Support - GGUS - within the LCG & EGEE environment Abstract: For very large projects like the LHC Computing Grid Project (LCG) involving some 8,000 scientists from universities and laboratories

More information

Customer Bank Account Management System Technical Specification Document

Customer Bank Account Management System Technical Specification Document Customer Bank Account Management System Technical Specification Document Technical Specification Document Page 1 of 15 Table of Contents Contents 1 Introduction 3 2 Design Overview 4 3 Topology Diagram.6

More information

(RH 7.3, gcc 2.95.2,VDT 1.1.6, EDG 1.4.3, GLUE, RLS) Tokyo BNL TAIWAN RAL 20/03/2003 20/03/2003 CERN 15/03/2003 15/03/2003 FNAL 10/04/2003 CNAF

(RH 7.3, gcc 2.95.2,VDT 1.1.6, EDG 1.4.3, GLUE, RLS) Tokyo BNL TAIWAN RAL 20/03/2003 20/03/2003 CERN 15/03/2003 15/03/2003 FNAL 10/04/2003 CNAF Our a c t i v i t i e s & c o n c e rn s E D G - L C G t r a n s i t i o n / c o n v e r g e n c e p l a n EDG s i d e : i n t e g r a t i o n o f n e w m i d d l e w a r e, t e s t b e d e v a l u a t

More information

HDFS: Hadoop Distributed File System

HDFS: Hadoop Distributed File System Istanbul Şehir University Big Data Camp 14 HDFS: Hadoop Distributed File System Aslan Bakirov Kevser Nur Çoğalmış Agenda Distributed File System HDFS Concepts HDFS Interfaces HDFS Full Picture Read Operation

More information

Customize Mobile Apps with MicroStrategy SDK: Custom Security, Plugins, and Extensions

Customize Mobile Apps with MicroStrategy SDK: Custom Security, Plugins, and Extensions Customize Mobile Apps with MicroStrategy SDK: Custom Security, Plugins, and Extensions MicroStrategy Mobile SDK 1 Agenda MicroStrategy Mobile SDK Overview Requirements & Setup Custom App Delegate Custom

More information

FreeIPA Cross Forest Trusts

FreeIPA Cross Forest Trusts Alexander Bokovoy Andreas Schneider May 10th, 2012 1 FreeIPA What is FreeIPA? Cross Forest Trusts 2 Samba 3 Demo Talloc Tutorial Pavel Brezina wrote Talloc tutorial! http://talloc.samba.org/

More information

The EU DataGrid Data Management

The EU DataGrid Data Management The EU DataGrid Data Management The European DataGrid Project Team http://www.eu-datagrid.org DataGrid is a project funded by the European Union Grid Tutorial 4/3/2004 n 1 EDG Tutorial Overview Workload

More information

D5.4.4 Integrated SemaGrow Stack API components

D5.4.4 Integrated SemaGrow Stack API components ICT Seventh Framework Programme (ICT FP7) Grant Agreement No: 318497 Data Intensive Techniques to Boost the Real Time Performance of Global Agricultural Data Infrastructures Deliverable Form Project Reference

More information

Using Databases to Manage State Information for. Globally Distributed Data

Using Databases to Manage State Information for. Globally Distributed Data Storage Resource Broker Using Databases to Manage State Information for Globally Distributed Data Reagan W. Moore San Diego Supercomputer Center moore@sdsc.edu http://www.sdsc sdsc.edu/srb Abstract The

More information

Cloud Computing. Lecture 5 Grid Case Studies 2014-2015

Cloud Computing. Lecture 5 Grid Case Studies 2014-2015 Cloud Computing Lecture 5 Grid Case Studies 2014-2015 Up until now Introduction. Definition of Cloud Computing. Grid Computing: Schedulers Globus Toolkit Summary Grid Case Studies: Monitoring: TeraGRID

More information

www.basho.com Technical Overview Simple, Scalable, Object Storage Software

www.basho.com Technical Overview Simple, Scalable, Object Storage Software www.basho.com Technical Overview Simple, Scalable, Object Storage Software Table of Contents Table of Contents... 1 Introduction & Overview... 1 Architecture... 2 How it Works... 2 APIs and Interfaces...

More information

CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT

CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline

More information

Objectives. Chapter 2: Operating-System Structures. Operating System Services (Cont.) Operating System Services. Operating System Services (Cont.

Objectives. Chapter 2: Operating-System Structures. Operating System Services (Cont.) Operating System Services. Operating System Services (Cont. Objectives To describe the services an operating system provides to users, processes, and other systems To discuss the various ways of structuring an operating system Chapter 2: Operating-System Structures

More information

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. dcache Introduction

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. dcache Introduction dcache Introduction Forschungszentrum Karlsruhe GmbH Institute for Scientific Computing P.O. Box 3640 D-76021 Karlsruhe, Germany Dr. http://www.gridka.de What is dcache? Developed at DESY and FNAL Disk

More information

Apigee Gateway Specifications

Apigee Gateway Specifications Apigee Gateway Specifications Logging and Auditing Data Selection Request/response messages HTTP headers Simple Object Access Protocol (SOAP) headers Custom fragment selection via XPath Data Handling Encryption

More information

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

More information

Sector vs. Hadoop. A Brief Comparison Between the Two Systems

Sector vs. Hadoop. A Brief Comparison Between the Two Systems Sector vs. Hadoop A Brief Comparison Between the Two Systems Background Sector is a relatively new system that is broadly comparable to Hadoop, and people want to know what are the differences. Is Sector

More information

/ Preparing to Manage a VMware Environment Page 1

/ Preparing to Manage a VMware Environment Page 1 Configuring Security for a Managed VMWare Enviroment in VMM Preparing to Manage a VMware Environment... 2 Decide Whether to Manage Your VMware Environment in Secure Mode... 2 Create a Dedicated Account

More information

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

More information

Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources

Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources Asif Saleem Marko Krznarić Jeremy Cohen Steven Newhouse John Darlington London e-science Centre,

More information

Distributed Database Access in the LHC Computing Grid with CORAL

Distributed Database Access in the LHC Computing Grid with CORAL Distributed Database Access in the LHC Computing Grid with CORAL Dirk Duellmann, CERN IT on behalf of the CORAL team (R. Chytracek, D. Duellmann, G. Govi, I. Papadopoulos, Z. Xie) http://pool.cern.ch &

More information

Client-aware Cloud Storage

Client-aware Cloud Storage Client-aware Cloud Storage Feng Chen Computer Science & Engineering Louisiana State University Michael Mesnier Circuits & Systems Research Intel Labs Scott Hahn Circuits & Systems Research Intel Labs Cloud

More information

About This Document 3. Integration and Automation Capabilities 4. Command-Line Interface (CLI) 8. API RPC Protocol 9.

About This Document 3. Integration and Automation Capabilities 4. Command-Line Interface (CLI) 8. API RPC Protocol 9. Parallels Panel Contents About This Document 3 Integration and Automation Capabilities 4 Command-Line Interface (CLI) 8 API RPC Protocol 9 Event Handlers 11 Panel Notifications 13 APS Packages 14 C H A

More information

Hadoop IST 734 SS CHUNG

Hadoop IST 734 SS CHUNG Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to

More information

Apache Sentry. Prasad Mujumdar prasadm@apache.org prasadm@cloudera.com

Apache Sentry. Prasad Mujumdar prasadm@apache.org prasadm@cloudera.com Apache Sentry Prasad Mujumdar prasadm@apache.org prasadm@cloudera.com Agenda Various aspects of data security Apache Sentry for authorization Key concepts of Apache Sentry Sentry features Sentry architecture

More information

tibbr Now, the Information Finds You.

tibbr Now, the Information Finds You. tibbr Now, the Information Finds You. - tibbr Integration 1 tibbr Integration: Get More from Your Existing Enterprise Systems and Improve Business Process tibbr empowers IT to integrate the enterprise

More information

Improvement Options for LHC Mass Storage and Data Management

Improvement Options for LHC Mass Storage and Data Management Improvement Options for LHC Mass Storage and Data Management Dirk Düllmann HEPIX spring meeting @ CERN, 7 May 2008 Outline DM architecture discussions in IT Data Management group Medium to long term data

More information

GetLibraryUserOrderList

GetLibraryUserOrderList GetLibraryUserOrderList Webservice name: GetLibraryUserOrderList Adress: https://www.elib.se/webservices/getlibraryuserorderlist.asmx WSDL: https://www.elib.se/webservices/getlibraryuserorderlist.asmx?wsdl

More information

The saga of WebFTS and Federated Identity

The saga of WebFTS and Federated Identity The saga of WebFTS and Federated Identity Andrey Kiryanov IT/SDC 15/12/2014 The Reason: 2 What is a Federated Identity? It is the means of linking a person's electronic identity and attributes, stored

More information

File S1: Supplementary Information of CloudDOE

File S1: Supplementary Information of CloudDOE File S1: Supplementary Information of CloudDOE Table of Contents 1. Prerequisites of CloudDOE... 2 2. An In-depth Discussion of Deploying a Hadoop Cloud... 2 Prerequisites of deployment... 2 Table S1.

More information

IT/CZ cluster meeting, Milano, September 23 24, 2004. CESNET Report. Aleš Křenek et al.

IT/CZ cluster meeting, Milano, September 23 24, 2004. CESNET Report. Aleš Křenek et al. IT/CZ cluster meeting, Milano, September 23 24, 2004 CESNET Report Aleš Křenek et al. Activities overview vacation security: transition to GSS, framework for webservices new development: L&B webservice

More information

Distributed File Systems

Distributed File Systems Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.

More information

Use of Hadoop File System for Nuclear Physics Analyses in STAR

Use of Hadoop File System for Nuclear Physics Analyses in STAR 1 Use of Hadoop File System for Nuclear Physics Analyses in STAR EVAN SANGALINE UC DAVIS Motivations 2 Data storage a key component of analysis requirements Transmission and storage across diverse resources

More information

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14 Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases Lecture 14 Big Data Management IV: Big-data Infrastructures (Background, IO, From NFS to HFDS) Chapter 14-15: Abideboul

More information

Report from SARA/NIKHEF T1 and associated T2s

Report from SARA/NIKHEF T1 and associated T2s Report from SARA/NIKHEF T1 and associated T2s Ron Trompert SARA About SARA and NIKHEF NIKHEF SARA High Energy Physics Institute High performance computing centre Manages the Surfnet 6 network for the Dutch

More information

OSG PUBLIC STORAGE. Tanya Levshina

OSG PUBLIC STORAGE. Tanya Levshina PUBLIC STORAGE Tanya Levshina Motivations for Public Storage 2 data to use sites more easily LHC VOs have solved this problem (FTS, Phedex, LFC) Smaller VOs are still struggling with large data in a distributed

More information

SDK Code Examples Version 2.4.2

SDK Code Examples Version 2.4.2 Version 2.4.2 This edition of SDK Code Examples refers to version 2.4.2 of. This document created or updated on February 27, 2014. Please send your comments and suggestions to: Black Duck Software, Incorporated

More information

Adding Panoramas to Google Maps Using Ajax

Adding Panoramas to Google Maps Using Ajax Adding Panoramas to Google Maps Using Ajax Derek Bradley Department of Computer Science University of British Columbia Abstract This project is an implementation of an Ajax web application. AJAX is a new

More information

rpafi/jl open source Apache Axis2 Web Services 2nd Edition using Apache Axis2 Deepal Jayasinghe Create secure, reliable, and easy-to-use web services

rpafi/jl open source Apache Axis2 Web Services 2nd Edition using Apache Axis2 Deepal Jayasinghe Create secure, reliable, and easy-to-use web services Apache Axis2 Web Services 2nd Edition Create secure, reliable, and easy-to-use web services using Apache Axis2 Deepal Jayasinghe Afkham Azeez v.? w rpafi/jl open source I I I I community experience distilled

More information

SWIFT. Page:1. Openstack Swift. Object Store Cloud built from the grounds up. David Hadas Swift ATC. HRL davidh@il.ibm.com 2012 IBM Corporation

SWIFT. Page:1. Openstack Swift. Object Store Cloud built from the grounds up. David Hadas Swift ATC. HRL davidh@il.ibm.com 2012 IBM Corporation Page:1 Openstack Swift Object Store Cloud built from the grounds up David Hadas Swift ATC HRL davidh@il.ibm.com Page:2 Object Store Cloud Services Expectations: PUT/GET/DELETE Huge Capacity (Scale) Always

More information

WP36: Composed Service Accounting Architecture Definition

WP36: Composed Service Accounting Architecture Definition WP36: Composed Service Accounting Architecture Definition D36.4: A set of Accounting Building Blocks for Automatically Composed Services Project funded by the European Community under the Information Society

More information

CA Identity Manager. Glossary. r12.5 SP8

CA Identity Manager. Glossary. r12.5 SP8 CA Identity Manager Glossary r12.5 SP8 This documentation, which includes embedded help systems and electronically distributed materials, (hereinafter referred to as the Documentation ) is for your informational

More information

Onset Computer Corporation

Onset Computer Corporation Onset, HOBO, and HOBOlink are trademarks or registered trademarks of Onset Computer Corporation for its data logger products and configuration/interface software. All other trademarks are the property

More information

Architecture and Mode of Operation

Architecture and Mode of Operation Open Source Scheduler Architecture and Mode of Operation http://jobscheduler.sourceforge.net Contents Components Platforms & Databases Architecture Configuration Deployment Distributed Processing Security

More information

Client Overview. Engagement Situation

Client Overview. Engagement Situation Client Overview Our client is a provider of Operational Analytics and Visualization solutions for cloud/datacenters that enables IT function of an organization to monitor, and plan complex cloud and data

More information

000-420. IBM InfoSphere MDM Server v9.0. Version: Demo. Page <<1/11>>

000-420. IBM InfoSphere MDM Server v9.0. Version: Demo. Page <<1/11>> 000-420 IBM InfoSphere MDM Server v9.0 Version: Demo Page 1. As part of a maintenance team for an InfoSphere MDM Server implementation, you are investigating the "EndDate must be after StartDate"

More information

The full setup includes the server itself, the server control panel, Firebird Database Server, and three sample applications with source code.

The full setup includes the server itself, the server control panel, Firebird Database Server, and three sample applications with source code. Content Introduction... 2 Data Access Server Control Panel... 2 Running the Sample Client Applications... 4 Sample Applications Code... 7 Server Side Objects... 8 Sample Usage of Server Side Objects...

More information

Gluster Filesystem 3.3 Beta 2 Hadoop Compatible Storage

Gluster Filesystem 3.3 Beta 2 Hadoop Compatible Storage Gluster Filesystem 3.3 Beta 2 Hadoop Compatible Storage Release: August 2011 Copyright Copyright 2011 Gluster, Inc. This is a preliminary document and may be changed substantially prior to final commercial

More information

Data Grids. Lidan Wang April 5, 2007

Data Grids. Lidan Wang April 5, 2007 Data Grids Lidan Wang April 5, 2007 Outline Data-intensive applications Challenges in data access, integration and management in Grid setting Grid services for these data-intensive application Architectural

More information

1. INTERFACE ENHANCEMENTS 2. REPORTING ENHANCEMENTS

1. INTERFACE ENHANCEMENTS 2. REPORTING ENHANCEMENTS W E L C O M E T O M O N I T O R I N G H E A V E N NEW THINGS ABOUT PANDORA FMS 5.0 A new version of Pandora FMS full of enhancements is about to hit the market. Pandora FMS 5.0 will be released by the

More information

Chillout - the Open Source DRM Software. Filippo Chiariglione CEDEO.net Yuqiang Liao Joint Digital Laboratory China

Chillout - the Open Source DRM Software. Filippo Chiariglione CEDEO.net Yuqiang Liao Joint Digital Laboratory China Chillout - the Open Source DRM Software Filippo Chiariglione CEDEO.net Yuqiang Liao Joint Digital Laboratory China 1 Abstract This paper presents Chillout [1], the Open Source DRM Software implementing

More information

Detailed Outline of Hadoop. Brian Bockelman

Detailed Outline of Hadoop. Brian Bockelman Detailed Outline of Hadoop Brian Bockelman Outline of Hadoop Before we dive in to an installation, I wanted to survey the landscape. HDFS Core Services Grid services HDFS Aux Services Putting it all together

More information

JVA-561. Developing SOAP Web Services in Java

JVA-561. Developing SOAP Web Services in Java JVA-561. Developing SOAP Web Services in Java Version 2.2 A comprehensive look at the state of the art in developing interoperable web services on the Java EE 6 platform. Students learn the key standards

More information

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4 Concepts and Architecture of the Grid Summary of Grid 2, Chapter 4 Concepts of Grid Mantra: Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations Allows

More information

Analysis and Design of Software Systems Practical Session 01. System Layering

Analysis and Design of Software Systems Practical Session 01. System Layering Analysis and Design of Software Systems Practical Session 01 System Layering Outline Course Overview Course Objectives Computer Science vs. Software Engineering Layered Architectures Selected topics in

More information

Exam Name: IBM InfoSphere MDM Server v9.0

Exam Name: IBM InfoSphere MDM Server v9.0 Vendor: IBM Exam Code: 000-420 Exam Name: IBM InfoSphere MDM Server v9.0 Version: DEMO 1. As part of a maintenance team for an InfoSphere MDM Server implementation, you are investigating the "EndDate must

More information

MicroStrategy Intelligence Server Configuration Table of contents

MicroStrategy Intelligence Server Configuration Table of contents MicroStrategy Intelligence Server Configuration Table of contents 1 Introduction... 2 1.1 Checklist...2 2 Connection to Metadata... 2 3 Connection to Knoa Databases... 10 1 Introduction This section configures

More information

How to Make the Client IP Address Available to the Back-end Server

How to Make the Client IP Address Available to the Back-end Server How to Make the Client IP Address Available to the Back-end Server For Layer 4 - UDP and Layer 4 - TCP services, the actual client IP address is passed to the server in the TCP header. No further configuration

More information

Software Development Kit (SDK) Technical Overview and Specifications

Software Development Kit (SDK) Technical Overview and Specifications Image Access Standard Software Development Kit (SDK) Technical Overview and Specifications Introduction On October 31, 2001, Kodak announced the KODAK Image Access Standard for Retail Digital Systems.

More information

Perceptive Experience Single Sign-On Solutions

Perceptive Experience Single Sign-On Solutions Perceptive Experience Single Sign-On Solutions Technical Guide Version: 2.x Written by: Product Knowledge, R&D Date: January 2016 2016 Lexmark International Technology, S.A. All rights reserved. Lexmark

More information

An Introduction to Globus Toolkit 3

An Introduction to Globus Toolkit 3 An Introduction to Globus Toolkit 3 -Developing Interoperable Grid services 1 Outline Cornerstones New Concepts Software Stack Core Higher Level Services Developing and Using Grid Services Development

More information

VMware vsphere Data Protection Advanced 5.5

VMware vsphere Data Protection Advanced 5.5 Data Protection Advanced 5.5 EMC Data Domain Integration TECHNICAL WHITE PAPER Table of Contents Data Domain Configuration... 4 Architecture Overview.... 4 vsphere Data Protection Advanced Client Support....

More information

Authorize.net modules for oscommerce Online Merchant.

Authorize.net modules for oscommerce Online Merchant. Authorize.net Authorize.net modules for oscommerce Online Merchant. Chapters oscommerce Online Merchant v2.3 Copyright Copyright (c) 2014 oscommerce. All rights reserved. Content may be reproduced for

More information