IBIS BIOINFORMATICS COURSES / COURS DE BIOINFORMATIQUE DE L'IBIS

Transcription

1 IBIS BIOINFORMATICS COURSES / COURS DE BIOINFORMATIQUE DE L'IBIS Course 1 - Introduction to Linux and bioinformatics UNIX, GNU and Linux The first computers were both very big and excessively expensive. These characteristics meant that they were mainly used by universities for research purposes. In this era, each new computer came with a different operating system. This created problems for the users since they had to learn a new system for each computer. A team of developers in the Bell Labs laboratories created a new operating system that could fit all machines. That project was named 'UNIX'. UNIX was continually kept compatible with all computer hardware that was being produced, turning it into the operating system of choice. However, one disadvantage of UNIX was that it was not 'free', in the sense that the source code was kept secret. This limited its spread and created a strong potential for people to produce an alternative free system with similar features but that could also be shared and modified at will. As a result, Richard Stallman started the GNU project in 1984 ( in order to provide people with a free alternative to UNIX. Thus, the GNU software was built from the start to be fully UNIX compatible. The GNU project members started to code equivalent tools to those found in UNIX. However, the biggest part of the job was still ahead. They had the system tools, but not the operating system itself, responsible for running the computer. In 1991, Linus Torvalds, a Finnish software engineer, started working on such an operating system, called the 'kernel', that would be completely compatible with the original UNIX, and thus with the GNU tools. From the start, he made his code freely available and others soon joined him in developing his kernel. It was the missing piece to a completely free operating system as envisioned by the GNU team. Thus, the GNU/Linux system, commonly called simply Linux, was born. Linux is thus a full UNIX clone that nowadays runs on many devices, from mobile phones, laptops, desktops, game consoles to servers and super computers. It is in fact the leading operating system run on servers, including servers meant to meet research computing needs. Since the early development, many flavours of Linux, also known as 'distributions', have appeared. These distributions target different audiences or needs (eg: servers, intensive computation, personal computers). Characteristics of Linux Linux has a few pros and cons. Here is a short list of things to consider. First, Linux is free. This has two meanings. The first is that there is likely always going to be a popular version of Linux available for download without any charge. For example, you can download the latest version of Ubuntu from: at no charge. Free also has the meaning expected from 'free software'. In most distribution, this means that the system is composed only of software that can be freely copied, modified and redistributed. You can then choose to add proprietary programs or drivers as needed. Linux is also portable. This means that it can be installed on any hardware configuration. This does not mean, however, that the installation is going to be easy in all cases. Linux

2 is secure and versatile. The security model is borrowed directly from the UNIX systems, which is known for its robustness and security. Critical updates are done frequently. Very few of those, however, require the user to reboot the system. Linux users' biggest security threat lies in phishing scams and thus in using their own judgement about who to trust when browsing the internet or using their . Linux is scalable. From phones to super computers, for internet browsing or intensive calculation, Linux fits the bill. There are thousands of programs available for just about any use you could think of. Even though it will work far better if installed on your system, you can try many Linux distributions without installing them at all on your computer. For this, you need to create and boot from either a Live CD (slow) or a Live USB stick (faster). And now for some of the cons. There are many distributions of Linux available. A nice site to help in choosing one is: This website surveys all the major as well as a lot of minor Linux distributions. It also sorts them according to popularity. We suggest you make your first steps with Ubuntu, an easy-on-newcomers Linux distribution. Ubuntu is the most widespread distribution. This ensures that it is more thoroughly tested and that both help and tutorials are easier to find. Linux is not very user friendly, especially for new users. In order to become a solid Linux user, you do have to learn a bit about computers and the esoteric 'command line'. Luckily, the situation has improved dramatically in the last five to ten years and some of the recent Linux distributions are much more user friendly, with good graphical interfaces. Hopefully, by choosing Ubuntu and following this introductory course to Linux and bioinformatics, you should be able to coast the learning curve and get a solid grasp at the basics of the Linux system. Remember, Linux, from its very origins, is all about community. There is documentation, tutorials and help to be found online. If you have any problem, first try a Google search with appropriate keywords. Someone may have had the exact same problem and you could find your solution in the answers to his question on a forum. If you still have no solution, contact your local geek. He'll try to help you solve your problem and also show you how to find more help by yourself. This is another important value at the core of the Linux community: freedom also means becoming more independent. More independent in learning to use the system, from the need to purchase proprietary software, and in how you choose to use your computer. Bioinformatics In the era of genomics, where gigabytes of information are flowing out of next generation sequencers, where genetic models are becoming increasingly complex, where building new genomes is getting more and more common, computers accordingly play a more important role. Bioinformatics is all about getting things done. Computers are used to generate, gather, store, analyze biological and genetic information with the goal of answering specific biology-related questions. No computers simply means no science. As the diversity of data types, methodological approaches and biological questions increases, one needs to be able to accomplish new types of analyses in order to stay on the edge. This is why bioinformatics capabilities, or simply computer proficiency, is key to accomplishing a research project in genetics or even building an interesting research career. The main goal of these courses is to provide tools that will give individuals more freedom, flexibility, and power when dealing with their data and analyses. Technologies and approaches change fast, but some things are so basic that they remain at the core of

3 any analysis. If you have a genomics project in your hands this year, which program you use is likely to be different than the one you will be using in two to five years. However, some things are going to remain very similar. At some point in the project, you will be handed out one or many data files, sometimes as large as a few (or many) gigabytes. Being able to visualize, query, and analyze these files is, of course, going to be very important. However, it is definitely not practical to manipulate such files using common text editors. How are you going to visualize a 4Gb file, let alone query it? During these courses, we will thus focus on some of the basics so that you become increasingly able to manipulate your data, in any way that you find suitable, in order to answer your questions. In this manner, you will also get to know your data better. Then, whether you end up using an already existing program or writing your own analysis tools, you will have a better control and understanding of your data and analyses. Also, when time will come to write your manuscript, you will have a detailed understanding of the analysis pipeline. Why do bioinformatics with Linux? Since we are going to spend a good amount of time getting accustomed with Linux during these courses, I feel compelled to justify why I believe Linux is a better and more natural operating system for bioinformatics work. Most of the reasons I describe below apply to the UNIX-like family of operating systems. However, for many reasons, Linux is the best choice to get introduced to this family. I got a lot of inspiration from the following forum thread for developing the following discussion about Linux: Technical reasons Using Linux for research purposes in genetics and genomics is a logical choice. The system features a number of tools that are directly aimed at treating text files of unlimited size, the most common way to store data. Moreover, these tools are built with the very aim of using them in an integrative manner, thus making the creation of pipelines very natural. Programming languages are either already installed or prepackaged for installation. A lot of good scientific software is written to work only under Linux or UNIX. This is even more true in the field of genomics. Linux gives the user complete control over their system, and it is thus easier to extend already existing software for new uses. Another benefit of using Linux is access to Bash, a very powerful command-line language that can be used to create pipelines. This makes it easier to increase the reproducibility of analyses by using automated pipelines with user defined parameters. Linux-based systems are also operating systems of choice when it comes to remote computing. Most supercomputers run some flavor of Linux or UNIX. When I asked the following question on the bioinformatics forum (see link above), I got a reply that I think summarizes the position I am trying to defend: Q: Why should someone doing a genomics project ever want to learn Linux? A: Put simply, using anything else hinders your research and provides competitors using UNIX a distinct advantage. Without question, the best tools available in this field are open source tools that are largely written for UNIX compatible systems. You can adapt these tools to Windows environments with Cygwin/VMWare, but part of being a scientist is knowing what the best equipment is for the experiment at hand. UNIX is the best equipment.

4 Career reasons Learning Linux helps to develop a transferable skills that sets you apart from a wet-only biologist; having *NIX skills on your CV is an asset in the post-genomic research world. It will help you better understand how computers and operating systems actually work. It gives you the ability to perform tasks faster and with more freedom, to run bioinformatics resources on your own machine (BLAST, GALAXY, etc), to increase the reproducibility of your research, to test more analysis avenues, and finally to perform analyses on computer clusters, which is important for big/long computational jobs. In other words, it turns you into a more independent and capable scientist. Even if your goal is not to become a pure bioinformatician or Linux power user, learning about Linux will help you work more efficiently and effectively in a team where you have to interact with these people. Here is another part of an answer I got from an evolutionary biologist on the forum: 'I will end by saying that taking the time to learn Linux has been by far one of my best "career" choices.' Philosophical reasons Free software and open source share a lot with the scientific research model. In both approaches, highly dedicated people work to produce results that are made publicly available. They do so by using the previous input of other people and turning it into better software or better understanding of our world. The mere fact of getting in closer contact with the open source movement can bring powerful insights into the nature of science as an activity. Even economy is being influenced by the open source movement. Indeed, more and more businesses are shifting their business model to incorporate some of its philosophy. Your (fictitious) genomics project To exemplify the points above, here is a down-to-earth example that is very likely to happen to you if you get involved in a genomics project these days. Situation: Friday morning, you have just received your new sequencing dataset. It consists in one fastq file that is 4 Go in size. By the way, this was more likely to have happened to you last year; Next year, the file could be much much bigger. The file contains tagged sequences from 20 individuals. Before the weekend, you would like to: Visualize the file (to know what a fastq file looks like) Know how many sequences are present Know the distribution of the sequences' length Know the number of sequences that come from the 20 individuals How would you proceed? What software would you use? What now if you had a few hundred sequence files, each many megabytes in size, and you wanted to combine them into one? Tools developed with normal word processing and spreadsheet office work in mind are very poorly design for these applications. We will need tools that are both more flexible and powerful. During this semester, we will progressively learn tools to help us to these kinds of operations easy and much more complicated ones feasible.

5 Presenting Ubuntu During the course, we will use Ubuntu, a Linux distribution, in order to show all the examples. Ubuntu is currently the most widespread distribution of Linux and among the easiest for new users. Choosing Ubuntu thus means that a lot of information is available online and that solutions are easier to find whenever problems arise. In the first course, I will show a brief overview of the system. In order to follow the courses, and mainly in order to do the exercises, you will need to have access to a machine with Linux installed on it, preferably Ubuntu. During the second course, we will be installing Ubuntu on the computer of those who are interested. A few options will be available: installation (single or dual-boot), virtualization or booting either from a CD or a USB stick. We will discuss these options during the first course Eric Normandeau Professionel de recherche, Laboratoire de Louis Bernatchez Luca Freschi Étudiant au doctorat, Laboratoire de Christian Landry Scott Pavey Candidat post-doctoral Laboratoire de Louis Bernatchez