CLOSHA MANUAL ver1.1. KOBIC (Korean Bioinformation Center) Bioinformatics Workflow management System in Bio-Express

Transcription

1 CLOSHA MANUAL ver1.1 Bioinformatics Workflow management System in Bio-Express Cloud Services for Massive Sequencing Data Analysis KOBIC (Korean Bioinformation Center)

2 1. Introduction CLOSHA is a an open source software developed by the Korean Bioinformation Center (KOBIC) that provides an easy-to-use workflow for next-generation sequence (NGS) data, such as exome or RNA-seq. And, CLOSHA service is a cloud service which enables to analyze massive genome data using high speed data transmission system and analysis pipeline management system even without need to install and manage those software tools. CLOSHA cloud mainly consists of infra system, platform for providing high speed analysis service, and web service. Infra system is a hybrid cluster infra system which can support both of hadoopbased big data platform and Unix/Linux platform in combined manner. Based on this infra system, we implemented high-speed analysis system using an existing big data platform incorporated with Hadoop Distributed File System(HDFS) as a primary storage device. We also developed turbo cache platform so as to support high speed analysis service based on web application. CLOSHA is a system which is designed to efficiently build a model for pipeline-typed research processes on bioinformatics and automate the model. Without the system, user has to conduct a series of complicated analysis processes in several steps separately. The proposed system allows user to conduct this analysis process via single interface in an integrated and easy manner. It also allows user to download intermediate output for each analysis step so that user could verify the research direction in the middle of analysis process. In order to make the analysis environment on top of web-platform based CLOSHA practical use in the field, we aim to support the analysis service on web platform environment using the KoDS solution that can achieve high speed data transmission. 2

3 A. The list of projects B. Explorer file manager C. The schema design of the workflow and a tool box D. Detailed information table of analysis module E. Parameter setting information F. Integrated analysis workflow list registered system G. Module list available unit analysis H. Multi execution status and job histories 2. How to use A. KOBIC integrate user management system join and login 1 Access the closha web-site and click the REGISTER button. 3

4 2 Enter the simple user information for join, and click register button 3 Enter the user id and password for login, then click the login button 4

5 B. Register analysis program 1 Click the Analysis Program Register button on the top menu. 2 Enter the information for the analysis program, then click the Register button 5

6 You can add the input or output parameter 3 You can check the analysis program that you registered on the right side. Registered analysis program 6

7 C. Register project 1 Click the New Project button on the top menu 2 Enter the project name, project description, select the project type and click the Create button. (If you don t want make the shared project or there s no shared project, you just select the new analysis pipeline.) 3 You can check the pipeline project that you registered on the left side. 7

8 D. Upload your NGS data 1 Access the CLOSHA web-site and click the icon for your OS version and running KoDS. (Verion 1.7 based on this document) 2 Select CLOSHA on the list box, then enter the user id and password. And, click 8

9 the login button. 3 After login is completed, you can see the user local file list on the left side. On the left side you can see the CLOSHA s file list. You can upload your local files to storage through the drag and drop. You can upload your local files to storage through the drag and drop 9

10 4 You can check the file transfer completely. Completed the file transfer E. Run the pipeline 1 The project window appears when you click the list of project that you registered step C 10

11 2 You can put the analysis program on the canvas through the drag and drop from analysis program list on right side. And you can connect analysis program to program do drag each analysis program. You can design the pipeline through the drag and drop from this window. 11

12 Connect each programs. 3 When you click the Setting button under the canvas, you can set the parameter that you need. 4 Move the Parameter tab on the Setting window, you can setting the parameters like input and output. After finish setting, click the Register button. As you can see, you can receive each program information through the URL window under the setting parameters. 12

13 5 You can set the parameter through file browser icon ( ), you can choose the files which public data or personal data on the Select input file window. 6 These parameters can be set via the button on the toolbox. After the parameter 13

14 setting, you must save the project through the Save button before running project. And through the 'Running' button, you can run the project. 7 Users can see the process of the pipeline workflow. The process in the pipeline workflow window can be shown three stages: Waiting, Running, and Complete. F. Result file download 1 Click the File browser button on the top menu. When the file browser window show up, you can move the path where to files exist you want. After then check the files checkboxes you want to download. 14

15 2 You can download that you checked files to click the Download button on the bottom menu. (Before you download files, you should change your browser popup setting to always allow) G. Share the pipeline project 1 Click the Share the pipeline button on the top menu. 2 You can multiple choose the projects on the list using Ctrl key. And Click the share project button on the bottom side. 15

16 Project name, Admin, , Created date, etc About each project s information 3 You can choose the users multiple on the integrated login user list. And click the Share button, share is completed. Shared project list H. Description of other menu 1 Project delete. 16

17 Choose the project what you want to delete on pipeline project list on the left side. And click the Project delete button on the top menu. Then, click the ok button on the pop-up window with the message. 2 Close If you click the close button on the top menu, you can close the active tab. I. Description of Tool box 1 Running 17

18 If you finish design pipeline and setting parameter, you can run the pipeline through this Run button on the tool box. Before you run, you must save the project. 2 Status You can update the status of the pipeline to manually through this Status 3 Save 18

19 Save the pipeline project. Before you run the project or after design the pipeline, setting parameter you must save the project. 4 Reset You can reset the pipeline setting to your last saved pipeline setting. 5 Job History 19

20 You can check the pipeline s job history with start time, end time. 6 Connection (Connection informations) You can modify or delete the connection information between analysis programs in the pipeline project. In the analysis program list tab, you can delete what you want, in the connection information tab you can reset or 20

21 modify the connection information between analysis programs. Hadoop / Linux File type Reset connection Connection status Disconnect 7 Parameter setting 21

22 You can set your analysis pipeline parameter where. Parameter type should be set when you register analysis program. Click the file browser icon( ) for choose the input data and set the output path. After all setting the parameter, click the submit button for finish setting. Choose the input data through the File browser Through this pipeline shape icon( ), you can easily set the input or output path 22

23 that was used before another programs. 8 Register Pipeline The same function with 3-C). Open the project window that you want share. Then Click the Register button on the Tool box. You can see the message about 'Do you want to register this pipeline?' If you click yes you should setting some information about sharing pipeline project. After then, you can check your shared project on the right side analysis pipeline list. 23

24 9 Log You can watch the log message during the pipeline running in real time. You can watch the pipeline log message in real time 24