How to deploy BioCloud on own HPC cluster?

Transcription

1 How to deploy BioCloud on own HPC cluster? 1. Introduction Installation Hardware requirement Software requirement Installation Configuration Web Portal Configuration Cluster Service Configuration Usage Usage of System Manager Module User Manager Reset Password Role Manager Authority Manager Machine Manager Machine Account Manager Manachine Job Queue Manager Machine Job Schedule Manager Disk Manager Private Dataspace Manager Public Dataspace Manager Share Dataspace Manager Application Classification Dictionary Application Script Manager Usages of Other Modules Contact and Support... 6

2 1. Introduction BioCloud is a private cloud application platform which dedicated for High Performance Computing (HPC) cluster using PBS to schedule job. BioCloud consists of two parts that is Web Portal and Cluster Service. The web portal can be deployed on either the header node of HPC or a single server while the cluster service must be deployed on the header node of HPC. The download package provides two folders named Web Portal and Cluster Service and the contents are described as following table. The contents of Web Portal package File biocloud.sql biocloud.war The contents of Cluster Service package File biocluster.sql taskservice.war The MySQL database schema for BioCloud as we use MySQL to store information from web portal The whole web project for BioCloud which must be placed at Tomcat The MySQL database schema for Cluster Service as we use MySQL to store information from cluster The web service for task operation on HPC. 2. Installation 2.1 Hardware requirement 1) Web Portal CPU RAM 64 bit Intel or AMD CPU 2 GB to 4 GB per CPU 2) Cluster Service For header node: CPU 64 bit Intel or AMD CPU RAM 2 GB to 4 GB per CPU 2.2 Software requirement 1)Web Portal Operation system: Linux 64-bit JDK 1.6 MySQL Tomcat )Cluster Service For header node Operation system: Linux 64-bit JDK 1.6 MySQL Tomcat

3 PBS 2.3 Installation The following gives an installation description for Web Portal and Cluster Service which will be deployed on different server, which is one single server for Web Portal and cluster header node for Cluster Service. 1) The basic installation of Web Portal Please download the web portal package biocloud.tar.gz and place it on directory such as /home/test/. After that, uncompress it and do the operations as followings. Step one: create one database called biocloud using MySQL root account create database biocloud DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; grant all on biocloud.* to localhost identified by biocloud ; Step two: load BioCloud database schema, the command line like the following: mysql u biocloud biocloud < /home/tangbx/biocloud.sql -pbiocloud Step three: copy biocloud.war to {Tomcat installs directory}/webapps directory and then start tomcat by {Tomcat install directory}/bin/startup.sh. Note: You need to modify the {Tomcat install directory}/conf/server.xml according to your requirement. Add the following configuration inside Host <Context path="" docbase="biocloud" reloadable="true"/> Type address:port/welcome.jsp. If everything is right then it will work. Go to part 4 to find how to use BioCloud. Now, you have done the basic installation of Web Portal. Please move to 3.1 to see the configuration. 2) The basic installation of Cluster Service Please download the web portal package biocluster.tar.gz and place it on directory such as /home/biocloud /. After that, uncompress it and do the operations as followings. Step one: create one database called biocluster using MySQL root account create database biocluster DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; grant all on biocluster.* to localhost identified by biocluster ; Step two: load database schema, the command line like the following: mysql u biocluster biocluster < /home/biocloud/biocluster.sql -pbiocluster

4 Step three: copy taskservice.war to {Tomcat install directory}/webapps directory and then start tomcat by {Tomcat installs directory}/bin/startup.sh Type address:port/taskservice. If everything is right then it will work Now, you have done the basic installation of Cluster Service. Please move to 3.2 to see the configuration. 3. Configuration 3.1 Web Portal Configuration 1) Configure to connect MySQL Database (required) File location: {Tomcat installs directory}/webapps/biocloud/web-inf/ classes/applicationcontext-common.xml Find the datasource node. Name url The MySQL connection information. Change the database name accordingly. username password The database access username, default value is biocloud The database access password, default value is biocloud 3.2 Cluster Service Configuration 1) Configure MySQL database information (required) File location: {Tomcat installs directory}/webapps/taskservice/web-inf/ classes/mysql.properties Name jdbc.url The MySQL connection information. Change configuration accordingly. Such as jdbc\:mysql\://localhost/biocluster?user\=biocluster&password\=biocluster&use Unicode\=true&characterEncoding\=UTF-8 4. Usage 4.1 Usage of System Manager Module The System Manager used to configure some initialize information such machine,jobqueue which will be used by BioCloud. Only Admin user can operation this module. The default admin username is and the password is User Manager This module used to manager all registered users in BioCloud. Admin need to assign Role and

5 Dataspace for each user as well as do Active operation Reset Password This module used to reset user s password Role Manager This module used to manager roles. Each role can be assigned different operation authority which defined in the Authority Manager Authority Manager This module used to manager user s operation authority. It often associates with Role Machine Manager This module used to configure the cluster head node information. The web service URL refers to the URL of cluster service. Remember to do ActiveMonitorWS operation once the cluster service restart Machine Account Manager This module used to manage user s account of HPC. Each register user in BioCloud should associate with one validate HPC account otherwise the submit task will not work right Machine Job Queue Manager This module used to configure the submit job queue of Cluster Machine Job Schedule Manager This module used to configure the job schedule information of HPC. The default job schedule is pbsqsub. If you want to add more job schedule you need to send us an Disk Manager This module used to manager storage information of HPC Private Dataspace Manager This module used to assign private storage space for users in given disk. Make sure the data space directory exists and the user has authority to operate Public Dataspace Manager This module used to configure the public storage space for BioCloud Share Dataspace Manager This module used to configure the share storage space for users. Make sure the data space directory exists and the user has authority to operate Application Classification Dictionary This module used to configure the application type which will be used for publishing the

6 application or pipeline Application Script Manager This module used to validate the user s uploaded script when users create an application or a pipeline. Make sure the Main Script Directory is the existed absolute path for the deployment of script. 4.2 Usages of Other Modules Please go to find the usage information about BioCloud 5. Contact and Support For any question you have encountered when deployed BioCloud, you can to biocloud@big.ac.cn