How To Make A Successful Project

Transcription

1 Building an e-science infrastructure fr Astrnmy L. da Csta LIneA & ON LSST Meeting Camps de Jrdã, April 1 1

2 The Dark Energy Survey DES-Brazil Jined 2007 Blanc 4-meter at CTIO Stage III DE prject using 4 cmplementary* techniques: I. Cluster Cunts II. Weak Lensing III. Large-scale Structure IV. Supernvae Tw multiband surveys: 5000 deg2 grizy t 24th mag 30 deg2 repeat (SNe) Build new 3 deg2 FOV camera and Data management system Survey (525 nights) *in systematics & in csmlgical parameter degeneracies 2 *gemetric+structure grwth: test Dark Energy 2 vs. Gravity LSST Meeting Camps de Jrdã, April

3 BPG Jined 2008 APOGEE BOSS MarvelsMeeting LSST Camps de SEGUE-II Jrdã, April 3 3

4 In-kind cntributin DES QA Science Prtal Tertiary site QR SDSS Mirrr site Interface t SAS LSST Meeting Camps de Jrdã, April 4 4

5 LIneA s creatin Signed Octber 29, 2010 LSST Meeting Camps de Jrdã, April 5

6 12 Technical Staff 30 scientists: 1. AIP 2. OAC 3. Padva 4. UFRJ 5. UFRGS 6. UNESP 7. USP 8. UFF & LSST Meeting Camps de Jrdã, April 6 6

7 Main gals Lgistical supprt t large survey wrk (management, cllabratin tls, finances) Develp e-science infrastructure fr: Prcessing Archiving Analyzing Data mining Data distributin Supprt f ther activities Helpdesk Cde develpment envirnment (science libraries, licenses, cde versining) Cde inheritance R&D (wrkflws, database) Data Curatin Archival f raw data handled by LNA Archive f advanced data prducts DES, SDSS-III, etc LSST Meeting Camps de Jrdã, April 7 7

8 LSST Meeting Camps de Jrdã, April 8 8

9 Wrking Mdel Users prjects utsurce LIneA Cde Heritage Pipelines Prtal Data server Real cases RNP Data transfer Data Strage VO tls LNCC HPC Wrkflws Database LSST Meeting Camps de Jrdã, April 9 9

10 Current Resurces Internal technical team +/- 12 peple Outsurce SLACAM installatin, prcurement, mnitring and maintenance Web designer DBA DR9 installatin T&T sftware develpment Cnsultants LNCC (W. Le, F. Prt) RNP (M. Stantn team) COPPE CEFET LSST Meeting Camps de Jrdã, April10 10

11 Sftware Develpment LSST Meeting Camps de Jrdã, April11 11

12 Examples Science cdes Galaxy mdel Chemical evlutin mdel Galaxy sub-structure detectin QSO mcks Clustering Cluster finder Simulatin and detectin f arcs Data reductin (QR, Precam, DECam) Prtal infrastructure* Science Applicatins Image prcessing* (Precam, QR) QA* Science Pipelines Cluster* Pht-z MW QSO (pster) LSS GE Wrkflws LSST Meeting Camps de Jrdã, April12 12

13 LSST Meeting Camps de Jrdã, April13

14 LSST Meeting Camps de Jrdã, April14

15 LSST Meeting Camps de Jrdã, April15 LSST Dem Ri March

16 CTIO LSST Meeting Camps de Jrdã, April16 16

17 Observing rm 4-m CTIO LSST Meeting Camps de Jrdã, April17 17

18 LSST Meeting Camps de Jrdã, April18 LSST Dem Ri March

19 Presentatins & Dems 1. DES Cllabratin Cllabratin meetings Review Fermilab Oct 2010 Review University Pennsylvania Oct 2011 Review MPE May LSST SLAC (2009) University f Washingtn (2009) PPARC/Caltech (2010) ON (2012) 3. Others SAB(2011) LSST Meeting Camps de Jrdã, April19 19

20 Prtal Reviews The system is actually much mre than a Science Prtal, it is a cmprehensive, web-based escience Analysis Center fr the Dark Energy Survey Cllabratin. It is designed t be very general, and allw the sharing f cde, data, and results f analysis wrld-wide, in a virtual envirnment. We dn't knw that anything quite like it exists yet - it is frward-lking and ambitius The Brazil Prtal cncept includes a place where largecllabratin astrphysics analysis can be directed: a central place fr checking in cdes; cmpiling, linking, and running thse cdes; making plts and sharing them, all in a repeatable, archived fashin; and the ability t run things in parallel in larger vlume. The visin f the Brazil prtal is cmmendable and shuld g frward. The prtfli f tasks is large, yet the resurces t undertake the wrk are limited. LSST Meeting Camps de Jrdã, April20 20

21 Management LSST Meeting Camps de Jrdã, April21 21

22 Prject Schedule Basic pipelines Phase 1 92 scpe dcuments & interviews LSST Meeting Camps de Jrdã, April22 22

23 LSST Meeting Camps de Jrdã, April23 LSST Dem Ri March

24 Over 800 tasks f apprved prjects LSST Meeting Camps de Jrdã, April24 24

25 Internal wiki LSST Meeting Camps de Jrdã, April25 25

26 Helpdesk LSST Demsince Ri March LSST tickets June 2009 Meeting Camps de Jrdã, April26 26

27 Data Center LSST Meeting Camps de Jrdã, April27 27

28 Activities Prcess Lcal develpment Prductin (fr lcal and internatinal members) Remte (SINAPAD, UNESP, Fermilab, NCSA) Archive Backups Catalgs & images Databases Service Cde repsitry Cde develpment and testing Prtal data distributin data curatin (DES, SDSS, SOAR, Gemini, CFHT) access t ther data sets VO tls helpdesk LSST Meeting Camps de Jrdã, April28 28

29 Hardware Infrastucture Number Crunchers Database Current One site 200 cres 800 TB Services End sites 800 cres 800 TB Strage LSST Meeting Camps de Jrdã, April29 LSST Dem Ri March

30 Services Sftware System APACHE Rcks Cndr Glbus Qmail WEB Applicatins ClckingIT Dc-db twiki gitweb trac Database SDSS-II MS SQL DES PstgreSQL git tawala Science libraries Operatins garbage cllectr purge save cutut service view target data server (casjbs) Prtal instances user testing hmlg prductin LSST Meeting Camps de Jrdã, April30 30

31 New Acquisitins 0.54 PB + Altix XE 1300 CLUSTER (600 Cres, 4800GB Mem) LSST Meeting Camps de Jrdã, April31 31

32 Virtual circuits LSST Meeting Camps de Jrdã, April32 32

33 Data Distributin DR8 skyserver nly DR9 skyserver, casjbs LSST Meeting Camps de Jrdã, April33 33

34 DR9 Implementatin LSST Meeting Camps de Jrdã, April34

35 Future LSST Meeting Camps de Jrdã, April35

36 Lng-term view we are here LSST Meeting Camps de Jrdã, April36 36

37 Bth scientific & technical 10/29/09 LSST Meeting Camps de Jrdã, April37 37

38 Summary Remarkable prgress in a few years Wrk dne in crdinatin with ther MCT institutes Mving frm R&D t peratin envirnment LINeA cmmited t: invest in e-science crdinate invlvement f Brazilian teams in large internatinal surveys (DES, SDSS-III, AS3?, LSST?) cllabrate with LSST DM with n strings attached data curatin (surveys, LNA) LSST Meeting Camps de Jrdã, April38 38

39 extra slides LSST Meeting Camps de Jrdã, April39 39

40 Track time + prjects => update MS Prject LSST Meeting Camps de Jrdã, April40 40

41 Tertiary Site Activities Cntribute t data challenges: mck QSO catalg, arcs Generate Milky Way mdel (trilegal+). Generate randm catalgs fr LSS Prepare science-driven VAC (LSS, Cluster, GE, thers) Generate randm catalgs fr LSS Quality Assessment (truth table, ther authrs) Serve as cde repsitry fr the cllabratin Benchmark cmpeting algrithms (cluster finders, pht-z) Carry ut analysis fr cllabratin wide papers LSST Meeting Camps de Jrdã, April