LCG POOL, Distributed Database Deployment and Oracle

Transcription

1 LCG POOL, Distributed Database Deployment and Oracle Dirk Düllmann, D CERN HEPiX Fall 04, BNL Outline: POOL Persistency Framework and its use in LHC Data Challenges LCG 3D Project scope and first ideas for a LCG Database service CERN Databases Services for Physics plans

2 POOL Objectives To allow the multi-pb of experiment data and associated meta data to be stored in a distributed and Grid enabled fashion various types of data of different volume and access pattern event data, physics and detector simulation, detector data and bookkeeping data Hybrid technology approach,, combining C++ object streaming technology (Root I/O), for the bulk data transactionally safe Relational Database (RDBMS) services, (MySQL, Oracle) for catalogs, collections and other meta data In particular, POOL provides In particular, POOL provides Persistency for C++ transient objects Transparent navigation from one object across file and technology boundaries Integrated with a external File Catalog to keep track of the file physical location, allowing files to be moved or replicated D.Duellmann, CERN 2

3 POOL Storage Hierarchy A simple and generic model is exposed to the application A application may access databases (eg ROOT files) from a set of catalogs Each database/file has containers of one specific technology (eg ROOT trees) Smart Pointers are used to transparently load objects into a client side cache define object associations across file or technology boundaries POOL Context FileCatalog Database Container Object D.Duellmann, CERN 3

4 Mapping to Technologies Identify commonalties and differences between technologies Generic ROOT(Tree) ROOT(D ir) My SQ L Oracle Dat abase File File Dat abase User Space Cont ainer Tree/Branch Directory Table Table Object ID Event Number Uniqu e key Prim.Key Prim.Key Model adapts to (almost) any storage technology with direct access Record identifier needs to be known before flushing to disk Use of RDBMS rather conventional Use of RDBMS rather conventional No special object support in SQL required D.Duellmann, CERN 4

5 POOL Component Breakdown POOL is (mainly) a client-side package Coupling to standard file, database and grid services No specialized POOL servers! Storage Manager Streams transient C++ objects to/from disk Resolves a logical object reference to a physical object I/O via ROOT (rfio/dcache) or Database(Oracle/MySQL/SQLite) File Catalog Maintains consistent lists of accessible files together with their unique identifiers (FileID), Used to resolves the logical file reference (from a POOL Object ID) to a physical file Collections Defines (large) containers for objects (eg event collections) stored via POOL ROOT I/O Storage Svc Relational Storage Svc Storage Service XML Catalog MySQL Catalog Grid Replica Catalog POOL API FileCatalog Explicit Collection Implicit Collection Collections D.Duellmann, CERN 5

6 POOL Grid Connected User Application Experiment Framework Collections File Catalog RootI/O Meta Data Grid Dataset Registry Replica Location Replica Manager Service Meta Data Catalog Grid Resources LCG POOL Grid Middleware D.Duellmann, CERN 6

7 POOL Standalone User Application Experiment Framework Collections File Catalog RootI/O Meta Data MySQL or RootIO Collections XML or MySQL Catalogs MySQL or SQLite LCG POOL Disconnected Laptop D.Duellmann, CERN 7

8 Why a Relational Abstraction Layer (RAL)? Goal: Vendor independence for the relational components of POOL, ConditionsDB and user code Continuation of the component architecture as defined in the LCG Blueprint File catalog, collections and object storage run against all available RDBMS plug-ins To reduced code maintenance effort All RDBMS client components can use all supported back-ends Bug fixes can be applied once centrally To minimise risk of vendor binding Allows to add new RDBMS flavours later or use them in parallel and are picked up by all RDBMS clients RDBMS market is still in flux.. To address the problem of distributing data in RDBMS of different flavours Common mapping of application code to tables simplifies distribution of RDBMS data in a generic application independent way D.Duellmann, CERN 8

9 Software Interfaces and Plugins Experiment framework FileCatalog Collection StorageSvc RelationalStorageSvc Relational Catalog Relational Collection ObjectRelationalAccess RelationalAccess Seal reflection MySQL Oracle SQLite uses implements Implementation Abstract interface Technology dependent plugin D.Duellmann, CERN 9

10 POOL in 2004 Data Challenges Experience with POOL framework gained in Data Challenges is positive! No major POOL-related problems Close collaboration between POOL developers and experiments invaluable! EDG-RLS as POOL back-end catalog Deployment based on Oracle services provided by CERN Database group Stable service throughout the 2004 Data Challenges! Input concerning performance and required functionality for future Grid File Catalogs Successful integration and use in LHC Data Challenges! Data volume stored in POOL: ~400TB! Data volume stored in POOL: ~400TB! Similar to that stored in / migrated from Objectivity/DB! D.Duellmann, CERN 10

11 Why a LCG Database Deployment Project? LCG today provides an infrastructure for distributed access to file f based data and file replication Physics applications (and grid services) require a similar services for data stored in relational databases Several applications and services already use RDBMS Several sites have already experience in providing RDBMS services Goals for common project (LCG 3D) increase the availability and scalability of LCG and experiment components allow applications to access data in a consistent, location independent way allow to connect existing db services via data replication mechanisms simplify a shared deployment and administration of this infrastructure during 24 x 7 operation Need to bring service providers (site technology experts) closer to database users/developers to define a LCG database service Time frame: First deployment in 2005 data challenges (autumn 05) D.Duellmann, CERN 11

12 Project Non-Goals Store all database data Experiments are free to deploy databases and distribute data under their responsibility Setup a single monolithic distributed database system Given constraints like WAN connections one can not assume that a single synchronously updated database work and provide sufficient availability. Setup a single vendor system Technology independence and a multi-vendor implementation will be required to minimize the long term risks and to adapt to the different requirements/constraints on different tiers. Impose a CERN centric infrastructure to participating sites CERN is one equal partner of other LCG sites on each tier Decide on an architecture, implementation, new services, policies Produce a technical proposal for all of those to LCG PEB/GDB D.Duellmann, CERN 12

13 Database Services at LCG Sites Today In contact with database teams at Tier1 : ASCC, BNL, CNAF, GridKa, FNAL, IN2P3 and RAL Potential Tier2: ANL and U Chicago Several sites provide Oracle production services for HEP and non-hep applications Significant deployment experience and procedures exist but can not be changed easily without affecting other site activities MySQL is very popular in the developer community Initial choice often made by s/w developers Not always with full deployment picture in mind MySQL used for production purposes in LHC Not at very large scale, though Expected to deployable with limited db administration resources Several applications are bound to MySQL Expect a significant role for both database flavors Expect a significant role for both database flavors To implement different parts of the LCG infrastructure D.Duellmann, CERN 13

14 Situation on the Application Side Databases are used by many applications in the LHC physics production chains Project members from ATLAS, CMS and LHCb Alice interested in replication for online / T0 Currently many of these applications are run centralized Several of these applications expect to move to a distributed model for scalability and availability reasons Move to distributed mode can be simplified by a generic LCG database distribution infrastructure Still, this will require some development work Need to make key applications vendor neutral DB abstraction layers become available in many foundation libraries Applications which are only available for one DB vendor limit deployment Distribution would profit from policy in the area D.Duellmann, CERN 14

15 Distributed Databases vs. Distributed Caches FNAL experiments deploy a combination of http based database access with web proxy caches close to the client Performance gains reduced real database access for largely read-only data reduced transfer overhead compared to low level SOAP RPC based approaches Deployment gains Web caches (eg squid) are much simpler to deploy than databases and could remove the need for a local database deployment on some tiers No vendor specific database libraries on the client side Firewall friendly tunneling of requests through a single port Expect cache technology to play a significant role towards the higher tiers which may not have the resources to run a reliable database service D.Duellmann, CERN 15

16 Application s/w stack and Distribution Options APP client s/w web cache RAL RAL = relational abstraction layer SQLite SQLite file file network web cache Oracle MySQL db db & cache servers db db file storage D.Duellmann, CERN 16

17 Tiers, Resources and Level of Service Different requirements and service capabilities for different tiers Tier1 Database Backbone High volume, often complete replication of RDBMS data Can expect good network connection to other T1 sites Asynchronous, possibly multi-master replication Large scale central database service, local dba team Tier2 Medium volume, often only sliced extraction of data Asymmetric, possibly only uni-directional replication Part time administration (shared with fabric administration) Higher Tiers and Laptop extraction Support fully disconnected operation Low volume, sliced extraction from T1/T2 Need to deploy more than one replication/distribution technology Need to deploy more than one replication/distribution technology Each addressing specific parts of the distribution problem But all together forming a consistent distribution model D.Duellmann, CERN 17

18 Starting Point for a Service Architecture? T0 - autonomous reliable service O M M O T1- db back bone - all data replicated - reliable service T2 - local db cache -subset data -only local service T3/4 M O M Oracle Streams Cross vendor extract MySQL Files Proxy Cache D.Duellmann, CERN 18

19 3D Data Inventory WG Collect and maintain a catalog of main RDBMS data types Select from catalog of well defined replication options which can be supported as part of the service Conditions and Collection/Bookkeeping data are likely candidates Experiments and grid s/w providers fill a table for each data type which is candidate for storage and replication via the 3D service Basic storage properties Data description, expected volume on T0/1/2 in 2005 (and evolution) Ownership model: read-only, single user update, single site update, concurrent update Replication/Caching properties Replication model: site local, all t1, sliced t1, all t2, sliced t2 Consistency/Latency: how quickly do changes need to reach other sites/tiers Application constraints: DB vendor and DB version constraints Reliability and Availability requirements Essential for whole grid operation, for site operation, for experiment production, Backup and Recovery policy acceptable time to recover, location of backup(s) D.Duellmann, CERN 19

20 3D Service Definition WG DB Service Discovery How does a job find a close by replica of the database it needs? Need transparent (re)location of services - eg via a database replica catalog Connectivity, firewalls and connection constraints Access Control - authentication and authorization Integration between DB vendor and LCG security models Installation and Configuration Database server and client installation kits Which database client bindings are required (C, C++, Java(JDBC), Perl,..)? Server and client version upgrades (eg security patches) Are transparent upgrades required for critical services? Server administration procedures and tools Need basic agreements to simplify shared administration Monitoring and statistics gathering Backup and Recovery Backup policy templates, responsible site(s) for a particular data type Acceptable latency for recovery Bottom line: service effort should not be underestimated! Bottom line: service effort should not be underestimated! We are rather close to LHC startup and can only afford to propose models that have a good chance of working! Do not just hope for good luck; These services will be a critical part of the experiments infrastructure and should be handled accordingly! D.Duellmann, CERN 20

21 Replacement CERN Physics Services Current systems not scalable to initial exploitation phase of LHC Disk server poor match for DB needs; Sun Cluster under-configured Tests of Oracle 10g RAC on Linux, as proposed to PEB, promising Main goals of replacement service: Isolation 10g services and / or physical separation Scalability - in both database processing power and storage Reliability automatic failover in case of problems Manageability significantly easier to administer than now Timeline: Price Enquiries for front-end PCs, SAN infrastructure and SAN storage completed Orders made, delivery November, pre-production early 2005? D.Duellmann, CERN 21

22 SAN-based DB infrastructure ATLAS CMS LHCb ALICE COMPASS HARP spare spare Mid-range Linux PCs: dual power supply, mirrored system disk, with dual HBAs multiple GbitE (3) F/C switches: 2 x 64 ports Storage: 16 x 400GB disks D.Duellmann, CERN 22

23 Oracle Contract / Distribution Client run-time Oracle 10g Instant Client currently Linux & Windows; Mac still pre-production; needs PEB decision re any support (incl. local copy ) 10g Early Adopters Release 2 ( ) Client developer (Software Developers Kit SDK) Interim solution based on CERN re-packaging Official support expected with Expected around December (OracleWorld) (when 10g R2 will be announced) Server distributions Support issues should not be underestimated See Oracle Security Alert + Data Volume, 3D scenarios, etc. Oracle Standard Edition One will be evaluated Client bundling with ATLAS s/w Oracle have agreed to treat Tier0+Tiern sites as equal parts of LCG No issues regarding use of Oracle client for LCG at sites used for LCG Access to distributions managed by IT-DB still needs to be controlled Oracle Partner status would not provide any added value D.Duellmann, CERN 23

24 Registration for Access to Kits You must be registered in the CERN HR database If you have a CERN computer account you already are In all cases, must fill in Computer Account form Give preferred existing account, valid address and sign or If no account, must also complete other fields, i.e. date of birth etc. and Contract Addendum Name, Institute, Signature Much easier than booking a flight / hotel on the Web! Incomplete / illegible forms cannot be processed D.Duellmann, CERN 24

25 Account Registration Form English version is at div.web.cern.ch/it- div/documents/computerusage/compaccountregistrationform-english.pdf Contains 3 sections: 1. To be completed by the User 2. To be completed by the Group Administrator 3. To be Read and Signed by the User Complete all MANDATORY fields in 1 & 3 If you have an existing computer account, please indicate it under section 2. Please give also a valid e address in section 2. We do not want to generate and maintain a separate form just for these last two points! D.Duellmann, CERN 25

26 Collaborator Agreement Please provide all of the following information: Your name, signature and date The institute that you work for The CERN experiment on which you collaborate D.Duellmann, CERN 26

27 Oracle Security Alert Severity 1 Security Alert issued August 31 This is the highest security level The vulnerability is high risk and requires little specialized knowledge to exploit. Apply the patch and/or workaround to the affected products with the highest priority. IT-DB began immediately to prepare plan for patching servers Major effort, spanning ~4 weeks Lessons: need to limit versions of Oracle s/w in use Several versions of (8), 8i, 9i, 10g in use Often dictated by constraints e.g. 3 rd party s/w (non-physics) Need plan in place so can react quickly to future alerts Propose: a) time slot when routine interventions can be performed b) procedure for performing critical interventions when needed D.Duellmann, CERN 27

28 Summary The POOL persistency framework ( ) been integrated in three experiment s/w frameworks and has been successfully used in the 2004 Data Challlenges Some ~400 TB of data store in POOL POOL has been extended to support RDBMS data in multiple relational databases The LCG 3D project ( ) has together with the experiments and sites started to define and implement a distributed database service fortiers 0-2 Several potential experiment applications and grid services exist but need to be coupled to the upcoming services Difference in available T0/1/2 manpower resources will result in different level of service Currently defining the key requirements and verify/adapt a proposed service architecture Need to start pragmatic and simple to allow for first deployment in 2005 A 2005 service infrastructure can only draw from already existing resources Requirements in some areas will only become clear during first deployment when the computing models in this area firm up Consolidation activities for Physics DB services at CERN Oracle RAC on Linux investigated as main building block for LCG services Pre-production service planned for mid-2005 D.Duellmann, CERN 28