The new feature of the data server for MICS Asia project

Transcription

1 Joint Mics-Asia/TF HTAP Workshop, Beijing, China The new feature of the data server for MICS Asia project Qizhong Wu, Zifa Wang, Zhe Wang, Baozhu Ge, et al

2 Outline Data server: naqpms.iap.ac.cn IP: FTP service: the followed user is just for FTP access. User name: guestuser Password: The FTP account can apply from the Data server website. The data stored in the server. ESGF service HTTP: support chrome, firefox, safari, but not IE Windows, Linux, MacOS But, download file one by one. wget script Linux, MacOS, (may windows+cgywin, but not sure) The required files, support to continue downloading The Mailing list for Mics Asia III

3 Data Management Flow Only for the upload, can t download or delete the dataset Data providers Post messages when upload Mailing list Services Also include leader and admin, who will push provider. Reply the question from mailing list Question on dataset Upload pool /upload FTP Service Download pool /micsasiaiii FTP access Data users check approval Administrators Meta data HTTP access Create account for FTP/ESGF Wget scripts micsasia pool published ESGF Services

4 FTP access Data tree Some Emission and Meteorology have been published in Mics Asia data server. All published misc asia data have been re-mount into FTP server. AIR_SHIP, Anthropogenic, Biogenic, BiomassBurning, Volcano_SO 2 emissions have been updated Boundary Condition data has been added in the past month

5 FTP access Data tree Some Emission and Meteorology have been published in Mics Asia data server. All published misc asia data have been re-mount into FTP server. Any FTP tools Right fig: FTP access in Mac OS. The WRF results in 2010

6 Demo I: download a small file Demo II: download a bigger file Demo III: dowload one WRF result FTP download demo Port and Passive Mode are both active now, and will fix this issue.

7 The directory/data tree IAP or Emis, wrf, cmaq One specie, one file? Create fold Put the dataset More, you can upload the datasets to this mics asia data server though FTP. Upload your dataset into upload/

8 Present progress for Mics Asia III Emissions Emission Size Services Biogenic emissions 33G FTP & ESGF Biomass burning emissions 125M only FTP Anthropogenic emissions 1.3G FTP & ESGF CB05 929M SPRC99 349M Volcano emissions 33M only FTP AIR SHIP emissions 78M FTP & ESGF AIR 56M SHIP 19M Conclusion: Emission is ready for modeling.

9 Present progress for Mics Asia III Meteorological field, from IAP BC/OC conditions, from ftp://presto.snu.ac.kr:/pub/cutoff/base Need more details about this data Type Size Services Meteorology WRF for G FTP & ESGF BC Condition GeoChem for G only FTP

10 FTP User/passwd Allow anonymous FTP? No The guestuser is just public user for beta, anyone need the account need to apply. Max clients=10, max_per_ip=5 Allow ssh or other access into mics asia data server? No, in order to system secuity, this user is a ftpuser. One person, one account The guestuser will be closed. The users are encouraged to apply his own user.

11 FTP/ESGF User/passwd

12 FTP/ESGF User/passwd

13 ESGF services download demo Need openid(username) and password Password: At present, only netcdf can be shared though ESGF services. HTTP: the followed two links point to the same dataset wget script How to generate? How to run?

14 ESGF services download demo I --HTTP website Visit and it will point to

15 ESGF services download demo I --HTTP website Visit and it will point to Choose any dataset you want to download

16 ESGF services download demo I --HTTP website Visit and it will point to Choose any dataset you want to download

17 ESGF services download demo I --HTTP website Visit and it will point to Choose any dataset you want to download

18 ESGF services download demo I --HTTP website Visit and it will point to Choose any dataset you want to download The first time you call for the data will need openid and passwd, it will not need again before the browser has been closed.

19 ESGF services download demo I --HTTP website Visit and it will point to Choose any dataset you want to download The passwd of guestuser is ^_^

20 ESGF services download demo I --HTTP website Visit and it will point to Date: Choose any dataset Download you the dataset at this meeting want to download room, the speed is 2.4M/s The passwd of guestuser is ^_^

21 ESGF services download demo II --HTTP website Visit esg.bnu.edu.cn, and it will point to

22 ESGF services download demo II --HTTP website Visit esg.bnu.edu.cn, and it will point to The published model will list here.

23 ESGF services download demo II --HTTP website Visit esg.bnu.edu.cn, and it will point to The published model will list here.

24 ESGF services download demo II --HTTP website Visit esg.bnu.edu.cn, and it will point to The published model will list here.

25 ESGF services download demo II --HTTP website Visit esg.bnu.edu.cn, and it will point to The published model will list here.

26 ESGF services download demo II --HTTP website

27 ESGF services download demo III --wget scripts Besides, you can use wget scripts to chose dataset and download dataset. How to generate? Put this url in your browser address bar, and enter xperiment=wrf And it will download one wget scripts file named wget sh

28 ESGF services download demo III --wget scripts Besides, you can use wget scripts to chose dataset and download dataset. How to generate? Put this url in your browser address bar, and enter xperiment=wrf And it will download one wget scripts file named wget sh It include the all wrf files for downloading.

29 ESGF services download demo III Besides, you can use wget scripts to chose dataset and download dataset. How to generate? How to run? Run the generated wget scripts file: wget sh Demo run it in mac os(with wget and openssl-0.9.8y), also can be run Linux(need wget and openssl, version is better). --wget scripts

30 ESGF services download demo III Besides, you can use wget scripts to chose dataset and download dataset. How to generate? How to run? Run the generated wget scripts file: wget sh Demo run it in mac os(with wget and openssl-0.9.8y), also can be run Linux(need wget and openssl, version is better). Begin downloading --wget scripts

31 ESGF services download demo III Besides, you can use wget scripts to chose dataset and download dataset. How to generate? How to run? Run the generated wget scripts file: wget sh Demo run it in mac os(with wget and openssl-0.9.8y), also can be run Linux(need wget and openssl, version is better). Download one by one --wget scripts

32 ESGF services download demo III Besides, you can use wget scripts to chose dataset and download dataset. How to generate? How to run? Run the generated wget scripts file: wget sh --wget scripts Demo run it in mac os(with wget and openssl-0.9.8y), also can be run Linux(need wget and openssl, version is better). Interrupt and continue downloading Ctrl+C to interrupt it here Re-run and begin downloading here

33 ESGF services download demo III --wget scripts Besides, you can use wget scripts to chose dataset and download dataset. How to generate? Put this url in your browser address bar xperiment=wrf In this url, limit=2000 means the max number for downloading file is 2000, the defaults is model=wrf, which model dataset you want, the defaults is all, and we need the model list in MICS Asia project experiment=wrf201001, which experiment dataset you want More, you can add institute=iap and so on in this search url, the keyword can be found in the website.

34 Besides, you can use wget scripts to chose dataset and download dataset. How to generate? Put this url in your browser address bar The keyword you can find here. ESGF services download demo III xperiment=wrf In this url, --wget scripts limit=2000 means the max number for downloading file is 2000, the defaults is model=wrf, which model dataset you want, the defaults is all, and we need the model list in MICS Asia project experiment=wrf201001, which experiment dataset you want More, you can add institute=iap and so on in this search url, the keyword can be found in the website.

35 ESGF services download demo IV --wget scripts More, you can generate the wget script from website, but the defaults number of downloading file is Both can generate the wget scripts

36 Mailing list for Mics Asia III Mailing list box: The mail list was established by mailman At present, it is a membership mail list, that only its members will be posted The member list from Jun-ichi Kurokawa More users in the Mics Asia Project will be added in the near future. To post a message to all the list members, send to micsasia@naqpms.iap.ac.cn.

37 Mailing list for Mics Asia III Mailing list box: The mail list was established by mailman At present, it is a membership mail list, that only its members will be posted The member list from Jun-ichi Kurokawa More users in the Mics Asia Project will be added in the near future. To post a message to all the list members, send to micsasia@naqpms.iap.ac.cn.

38 Mailing list for Mics Asia III Mailing list box: The announcement, discussion for the Mics Asia III project are welcomed. If you have any question on the downloading the dataset, you can sent an to this list. In my option, the data provider should been included in this list. And if any have the question on the description of the dataset, the provider may can give some comments.

39