U.S. CMS Software and Computing Project

Transcription

1 U.S. CMS Software and Computing Project Lothar A T Bauerdick, Fermilab U.S CMS Collaboration Meeting at UC Riverside May 2001

2 CMS Software and Computing Scale of LHC computing is greater than anything we have tried in HEP LHC Collaboration Number of collaborating physicists Number of collaborating institutions Number of collaborating nations Geographic distribution Projected length of the experiment Size and complexity of dataset Distributed object data base CMS WorldWide 1800 Physicists 150 Institutes 30 Countries This demands a new model of computing U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 2

3 U.S. Part of World-Wide Computing for CMS LHC Computing Model: distributed multi-tier architecture Regional Computing centers outside CERN CMS Tier1 centers: U.S. Tier1 at Fermilab also Lyon, RAL, INFN, Moscow CMS Tier2 centers: 5 U.S. Tier2 centers CMS in total ~ 25 Coordinated use of distributed computing resources Communication and collaboration at a distance Data Grids as a new form of distributed systems Software Engineering: distributed software development for physics analysis software and computing R&D Research e.g. in Data Grids, distributed object dbase, data challenges as check-points prototype software and prototype computing installations funding for prototypes during 2001 to 2004 CERN, Fermilab, INFN, RAL, Lyon, several Tier2s U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 3

4 (U.S.) CMS Distributed Computing Model Tiered Computing model puts U.S. CMS Computing at Labs and Universities Draws strength and exploits the synergy between U.S. Universities and Fermilab Software Professionals and Physicists Takes advantage of and contributes to key developments in the US in information technology Drive towards a high speed network infrastructure Development of ever better network software and applications such as Grid computing concepts Significant funding, manpower and technologies from Grid projects It takes advantage through the Tier2 centers of the significant strengths of US universities in the area of computer science and information technology These are key elements to its eventual success U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 4

5 Software and Computing Deliverables Deliverables of the S&C Project Will be covered by S&C MOU CORE SOFTWARE: Engineered Infrastructure such as distributed object dbase, software development, analysis tools and processes MAJOR USER FACILITIES and Services: Tier1 and Tier2 Regional Centers, production management systems for distributed analysis, interface to wide area networks HEP base program Non-Deliverables of the S&C Project Sub-detector and Physics Software Local Computing (Workgroup servers, desktops) at home institutions NB: Physicists working on computing R&D, management are off-project U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 5

6 Computing R&D + User Facility Develop software / systems architectures and implementations to deal with a new level of complexity: Complexity in data: rate, #channels, pile up Complexity in data structures: persistent objects, data base computing, Grid Support for an active and growing user community > User Facility Detector simulation and reconstruction, Higher Level Trigger, Physics TDR Production in 2000/1: ~5M events simulated and reconstructed worldwide (1.5M at Fermilab) ~10 Terabyte data stored in an Object Database Distributed computing strategies have to be verified by applying them to real world computing problems! Tier1 and Tier2 prototypes start in 2000/1, increase complexity techniques and strategies for distributed computing operation and management of complex systems 50% complexity in 2004 U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 6

7 Resources are in 3 Categories User Facilities Equipment Prototypes Tier1 at Fermilab Tier2s at 5 U.S. Institutions User Facilities Staff Computing Professionals for Computing-related R&D Tier1 at Fermilab Tier2 Maintenance Core Application Engineers Computing Professionals/Software Engineers for CMS Core Software Support for specific U.S. activities (+ Project Office support, management reserve) U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 7

8 + User Facilities: 3-Phases Tier1 and Tier2 regional centers: R&D, Equipment, Staff Prototyping: has started in 2000 Computing R&D Computing hardware prototyping and test-beds Computing for Physic Reconstruction and Selection Prototype 50% complexity in 2004 Deployment: Assumes LHC startup in 2006 and design Lumi in 2007 Procurement Model: Start deployment in 2005, 30%, 30%, 40% costs Ramp-up of User Facility Staff Maintenance and Operations: 2007 on Constant staff level Rolling Replacement of hardware components, yearly budget 1/3 of initial investment Moore s law takes care of Evolution and Upgrades U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 8

9 Hardware Costs and Procurements Moore s Law Buy late but not too late Requirement based on Computing Model allow to estimate Computing Cost Large extrapolation factors O(20) Large uncertainties > build to cost! Tier1 Computing Resources needed in 2007: CPU 200k SI95 (~5000 1GHz Pentium ///) Disk ~ 1 Petabyte Tapes 1.5 Petabyte Initial Computing Hardware Costs: $10M ~ O (1 RunII exp) = $9.1M U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 9

10 User Facilities Staff Detailed Bottom-up analysis of needs for Tier1 and Tier2 facilities Full time staff working on User Facilities tasks: technicians, support staff, computing professionals Tier 1 Regional Center System and User Support Operations and Infrastructure Tier 2 RCs Networking Computing and Software R&D Construction Phase Computing Support FNAL based Computing User Facilities (total) (Without T2 Personnel) Comparison to RunII: DØ / CDF have 36/33.5 FTE! U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 10

11 CMS Software Cycles and Milestones Software Engineering The OO Functional Prototype phase completed in 2000 Entering new development iteration: Fully Functional pre-production Software in Distributed Object Data Base System Production System Software to be ready for first run in 2006 May 2001 CMS MILESTONES CORE SOFTWARE End of Fortran dev 1998 Jun CMS GEANT4 simulation 1 Jun-98 2 Dec-99 3 Jun-00 4 Dec-03 Framework 1 Jun-98 2 Dec-99 3 Dec-01 4 Dec-03 Det Reconstruction 1 Dec-98 2 Dec-99 3 Jun-02 4 Jun-04 Physics Object Reconstruction 1 Mar-99 2 Jun-00 3 Dec-02 4 Dec-04 User Analysis Environment Dec-98 Jun-00 Dec-02 Dec-04 1 Proof of Concept 3 Fully functional 2 Funct. Prototype 4 Production system These Milestones are linked with the major CMS Milestones: Technical Design Reports on DAQ, Software, Physics U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 11

12 Software Engineering U.S. contributes its share of ~25% to total Core Software Engineering effort Main U.S. contributions in Software Architecture Interactive Analysis Distributed Computing 8 Engineers now, ramping to 14 This will be subject of an imou on Software and computing (End 2001) U.S. Share of CMS Software FTEs Hungary Belgium India China Spain Switzerland (ETHZ) UK Germany Other France- IN2P3 RDMS- Russia RDMS-DMS USA (DOE and NSF) Italy CERN (CMS groups) FTE Profile CMS Core Software and Computing U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab Total CMS Offline + Online S/W FTE U.S. CMS Core-SW contribution

13 CMS and U.S. CMS S&C US plans and progress are important driving forces for discussions of LHC Computing at CERN and elsewhere U.S. Model and Costing sanctioned by Hoffmann Review LHC Computing Review Panels Software Project Worldwide Analysis/Computing Model Management and Resources Report of the Steering Group published end of Feb 2001 Findings total LHC computing hardware initial costs estimated 240 MCHF CERN Tier0 and Tier1 represents ~ 1/3 of overall capacity Recognize need for Prototyping and Software Engineering Deployment , then ~constant budget for rolling replacements LHC Total Initial Computing Cost: ~$150M U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 13

14 US CMS S&C Project Costs (prelim.) Total Costs FY is 54.7 M$ (escalated) in FY07 Integral Funding Profile is 52.5 FY01 to FY06, assume 17.5 in out-years Total Project Costs and Funding Profile Tier1/2 deployment Million AY$ Spread out R&D! Changes since Nov2000: shift in LHC schedule hardware procurement and UF Staff ramp-up shifted by one year (FY2005),some R&D staged U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab CAS labor UF Tier1 labor UF Tier2 labor UF Tier1 h/w UF Tier2 h/w Project Office Mgmt Reserve DOE profile DOE+NSF(assumed) FY Delayed deployment at T1 (starts 2005, in time ) will need some thinking 3.5 FTE from UF til FTE from CAS til FTE in PO Put in reality for 2001 Need to work on FY2002 Staging of R&D in UF

15 Management Team, Committees Project Management team in place since November 2000 Level 1 Project Manager Lothar Bauerdick/Fermilab, Lucas Taylor/North Eastern U deputy Level 2 Project Manager User Facilities Vivian O Dell/Fermilab Level 2 Project Manager Core Application Software Ian Fisk/UC San Diego U.S. Advisory Software and Computing Board Elected: Irwin Gaines/Fermilab chair Paul Avery/Florida, Bob Clare/UC Riverside, Sridhara Dasu/Wisconsin, David Stickland/Princeton, Shuichi Kunori/Maryland Ex-officio: L1PM, L2PMs, chair U.S. CB, PM CCS, L1PM construction, U.S. physics coordinator, head Fermilab CD Fermilab Oversight: PMG U.S. CMS PMG sub-group for Software and Computing Chair: Mike Shaevitz, Fermilab Associate Director of Research Added to PMG: L1/2PMs, U.S.ASCB chair, U.S.CB chair, Fermilab CD head 3 experts: Ruth Pordes, Erik Gottschalk, Mike Diesburg This is our Interface to U.S. CMS! U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 15

16 Reports, reviews etc 2nd Quarterly Progress Report released last Friday: FY2001Q2 SCOP review,, reporting to Fermilab Directors for Project Oversight Friday May 25, 9: :00 at Fermilab Status report to DOE/NSF at Washington/NSF May 30 Proposed date of DOE/NSF review New date November at Fermilab (was: October 9-12) plan: baseline the project Common meetings of (U.S.) LHC S&C community Networking meeting at Indiana U. June 1-2 Database meeting in July, U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 16

17 FY2001 Plans and Funding Make the most of Functional Prototype Software use to simulate and reconstruct events for Higher Level Trigger studies develop tools and systems needed to deliver large and high quality data samples Proceed towards Fully Functional Software milestone Iterate and extend on Software Architecture Develop Distributed Production and Analysis Environment Deploy User Facilities and R&D Systems at Tier1 and Tier2 centres Project Funding in FY2001 from DOE and NSF to support that program: (but problems with NSF part of the funds!) U.S. CMS Software and Computing Caltech UCSD FNAL NEU Princeton UC Davis pt2'2 TOTAL Core Applications Software (CAS) FTE User Facilities (UF) FTE TOTAL FTE CAS Personnel (salary, PC, travel,...) ,234.2 UF Personnel (salary, PC, travel,...) UF Tier 1 Equipment UF Tier 2 Equipment Project Office, Management Reserve TOTAL COST [AY$ x 1000] ,500.0 U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 17

18 Funding Status Received $1500k from DOE in February and April + $500k loan from Construction Project Support CAS and UF personnel, start Project Office Go-ahead for Tier-1 upgrade > Viv s talk yesterday Requested $1500k from NSF NSF Advice: $1000 planned for FY $500 for prototype Tier 2 end FY2000 Received only $320 for CAS engineers (NEU) (+ $80 expected) No concrete news from NSF! Status report meeting at NSF on May 30 All funds in AY$ x 1000 FY2000 FY2001 Total Requested Received DOE NSF Loan from U.S. CMS Detetector Project Total U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 18

19 Grid Projects Two Grid R&D proposals submitted with substantial U.S. CMS contributions PPDG (DOE SciDAC,, CMS PI: Harvey Newman): approved! deployment of Grid technologies in vertically integrated systems for CMS CMS deliverables worked on w/ PPDG provided FTE + CS groups Funding for personnel at Caltech, UCSD, Fermilab ivdgl (NSF ITR, PI: Paul Avery) Funding for CMS Tier-2 prototype efforts: (+ATLAS, Ligo, SDSS) mostly facilities + maintenance Some funds for Grid integration into CMS s/w See Paul s talk IMHO The production Tier-2 funds should be handled in a different way: The Tier-2 funds should be controlled by the CMS project This will require innovative approaches on the side of NSF U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 19

20 Prototype Tier-2 centers Successful deployment and commissioning of (1st stage of) Cal pt2! University of Florida was selected as 2nd prototype Tier-2 site Procedure as described in my letter to collaboration Solicitation for proposals (deadline April 15) Report of UF Level-2 PM to Level-1 PM Discussion to get input from U.S. ASCB (meeting April 24) Selection of Florida unanimous Funding of the prototypes through ivdgl will start FY2002 WBS, MOU and SOW in preparation (also w/ Caltech/UCSD) selection procedure for final Tier2 sites will need discussion in collaboration Should be proposal driven; criteria: U.S. ASCB, but much discussion required Review board to evaluate proposals, membership TBD Recommends sites/proposals with greatest benefits to U.S. CMS and CMS Final decision by L1PM Time scale: end of 2002 U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 20

21 Core Applications Software (CAS) 4 areas of software development WBS 2.1 CMS Software Architecture U.S.: e.g. get CARF, Object Dbase, C++ know-how WBS 2.2 Interactive User Analysis U.S.: e.g. schema for AOD,, prep for Physics TDR tools WBS 2.3 Distributed Data Mgmt and Processing U.S.: e.g. production tools at e.g. Fermilab,Caltech,UCSD,Wisconsin, concepts and tools for distributed analysis WBS 2.4 User Support: 25% of each engineers time U.S.: e.g. software engineering support; physicist formulates a project for CP, engineer can help designing / implementing; need to estimate time & involvement 8 CAS software engineers on project at 5 institutions Vladimir Litvin Caltech DDMP/ Prod. / Arch. Iosef Legrand Caltech (CERN) DDMP Tony Wildish Princeton (CERN) Prod. / DDMP/ Arch. Michael Case UC Davis Arch. Lassi Tuura Northeastern (CERN) Arch./ IGUANA Ianna Gaponenko Northeastern (CERN) IGUANA Hans Wenzel Fermilab Prod./ Arch. Greg Graham Fermilab DDMP U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 21

22 Software req.. from PRS (Paris, CPT week) Analysis tools: : Needed soon. The sooner the better; Physics TDR(s): Clearly, aim is not to determine/influence detector design Probably biggest side-effect/purpose will be the training of people We ll use near-final software and near-final tools Therefore: analysis tools must be ready and people trained on them at least a year before the Physics TDR. That s End Therefore, year 2003 is the year of deployment/training of people (tight, would be better to extend into 2002 ) Will need first full deployment end With the DAQ TDR. Timing would be good; finish TDR, presumably more time(?) Estimated size of event samples needed for Physics TDR studies Expect that we can continue at ~ the same rate as up to now Above statement accurate to a factor 2 (but: CPU for tracking?) More would be welcome (and even helpful) Data access issue(s) could be looked at Organization could be strengthened PRS Groups Getting More Involvded Into Production Side remark: must ensure good access to data for the physics TDRs U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 22

23 Summary of Engineering Efforts Spent effort is assessed on the basis of regular effort reports from engineers and managers, using a bottomup approach, where for each WBS item expenditure of labor is accumulated. For the User Facilities subproject this is reconciled with the monthly effort reports from the departments of Fermilab Computing Division. As part of the reporting for the Core Application Software subproject the engineers submit monthly reports for effort tracking. This system is successful! WBS item FY2001Q1 FY2001Q2 Total effort projected User Facilities FY2001 WBS 1.1 Tier 1 Regional Center inactive WBS 1.2 User Support WBS 1.3 Maintenance and Operations WBS 1.4 Tier 2 Regional Centers WBS 1.5 Tier 1 Network WBS 1.6 Software and Computing R&D WBS 1.7 Detector Construction Phase Computing WBS 1.8 Support for Fermilab Based Computing Total projectfunded effort FY2001 Total FTEs at Fermilab Regional Center Total FTEs WBS item FY2001Q2 Total effort projected Core Application Software FY2001 WBS 2.1 Software Architecture 0.44 WBS 2.2 Interactive Graphics and User Analysis 0.06 WBS 2.3 Distributed Data Management and Proc 0.75 WBS 2.4 User Support 0.75 Total projectfunded effort FY2001 Total FTEs U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 23

24 U.S. CMS S&C Summary A large part of the computing and software engineering for CMS will need to be provided from outside CERN to fully yield the compelling physics potential of CMS The U.S. CMS S&C Project is going to deliver the necessary efforts and computing resources to enable U.S. physicists making a major impact on physics analysis within CMS U.S. CMS Collaboration Meeting UCRiverside May 19, 2001 Lothar A T Bauerdick/Fermilab 24