LOCKSS on LINUX. Installation Manual and the OpenBSD Transition 02/17/2011

Transcription

1 LOCKSS on LINUX Installation Manual and the OpenBSD Transition 02/17/2011 1

2 Table of Contents Overview... 3 LOCKSS Hardware... 5 Installation Checklist... 7 BIOS Settings Installation Firewall Configuration LOCKSS Daemon Configuration Accessing LOCKSS Maintenance Please read these instructions carefully if you have any questions, contact us by at [email protected] we are here to support you through this process. 2

3 Overview This document explains how to install and maintain the LOCKSS software on a dedicated Linux system. This is the recommended way to run LOCKSS and is replacing the LOCKSS OpenBSD CD Platform, which is being phased out. The LOCKSS team has created a downloadable Linux netinstall CD based on the CentOS Linux distribution (see for more information about CentOS). The LOCKSS Net Install CD uses a Kickstart file that automates several parts of the Linux installation to produce a configuration appropriate for LOCKSS. It creates a disk partitioning layout utilizing software RAID when possible, selects a recommended set of Linux packages, and configures the Linux environment for the LOCKSS daemon. Read all the information in the blue boxes if you are transitioning from OpenBSD to Linux. OpenBSD Transition This document contains additional information about transitioning data from OpenBSD to Linux after the Linux installation. Special notes that pertain to the OpenBSD Transition will appear in a blue colored box like this one throughout the document. These Transition instructions should be ignored if you are building a new Linux LOCKSS box. Read all the information in the yellow boxes if you are part of a PLN. Private LOCKSS Network (PLN) This document describes the configuration for a Global LOCKSS Network (GLN). Private LOCKSS Network (PLN) installations have a few minor configuration differences which will be indicated in a yellow colored box like this one. 3

4 This document has a preparation phase and an installation phase. You should complete the entire preparation phase before attempting the installation. Preparation Phase 1. Hardware Choosing a real or virtual hardware configuration to run the LOCKSS software on Linux. 2. Installation Checklist Gathering information about your site and downloading the installation software. Installation Phase 3. BIOS Settings Checking your SATA compatibility and system power settings in your LOCKSS box BIOS. 4. Installation - Installing the Linux operating system and the LOCKSS software using the LOCKSS Net Install CD. 5. Firewall Configuration Setting up the Linux firewall to limit SSH access to your LOCKSS box. 6. LOCKSS Daemon Configuration Configuring LOCKSS for your site. 7. Accessing LOCKSS Connecting to the Web Administrative UI. 8. Maintenance Setting up automatic updates for Linux and LOCKSS software. 4

5 LOCKSS Hardware We recommend installing LOCKSS on a dedicated workstation or server with the following characteristics: An x86 Intel compatible CPU running at 1GHz or better A minimum of 1GB of memory (2 GB of memory is recommended) A bootable CD or DVD drive A SATA disk controller (ideally with support for 4 SATA devices) It s not necessary to buy a new computer for your LOCKSS box many computers built within the past few years meet these requirements. You can also build a LOCKSS box using virtual machine technology, such as a VMWare product, as long as you can allocate the resources to match the minimum requirements stated above. Hard disk capacity is perhaps the most important part of a LOCKSS box. The LOCKSS Linux installation will attempt to partition your disks using software RAID, which improves the availability of your LOCKSS box by storing your data redundantly across multiple hard disks. If a single hard disk in the software RAID fails, the LOCKSS box will continue to provide content and the possibility of data loss is minimal. Without RAID, a hard disk failure will mean data loss and an extended outage period for your LOCKSS box until it can be rebuilt from scratch. For proper RAID operation, it s important that all your drives are of identical make and model number and are all connected directly to your SATA controller. Here is a list of possible hard disk configurations along with minimum disk size in each configuration. Try to exceed the minimum size if possible. Number of Identical Hard Disks Minimum Size of each Hard Disk Notes 4 1 TB 2TB disks are best choice for this option 3 1.5TB 2TB disks are best choice for this option 2 2 TB 1 2 TB RAID not possible, not recommended If you can t meet or exceed one of the hard disk configurations listed above for whatever reason, or if you have drives you want to use that aren t identical to each other, please the LOCKSS team at [email protected] and describe the hard disk configuration you have. 5

6 OpenBSD Transition If you are building your Linux LOCKSS box on new hardware, you can skip this note. If you are planning to reuse the same hardware, please read the additional hardware discussion below. 1) Assessing your LOCKSS box Compare your LOCKSS box to the listed hardware requirements and determine if your OpenBSD LOCKSS box has any deficiencies that need to be addressed before converting it to Linux. Most importantly, if your current LOCKSS box is has an older PATA disk controller on the mother board and contains PATA hard disks, you should replace the entire system with a newer one that matches the hardware requirements. PATA is obsolete technology the largest available hard disk capacity is 500GB and finding replacement PATA hard drives is becoming increasingly difficult and expensive. Re-using your current LOCKSS box is not a requirement you can build your Linux LOCKSS box on a different computer. 2) Additional Hard Disks are needed You need to preserve your OpenBSD disks for the Data Transition process, so you need to obtain additional hard disks that match the requirements listed earlier in this section on which to build your Linux system on. 3) Additional Hard Disk connecting hardware may be needed (Disk to Disk Data Transition) The Disk to Disk Transition method requires both the new Linux hard disks AND the OpenBSD disks be connected to the LOCKSS box at the same time. You may not have enough disk connections and power connections on the motherboard of your LOCKSS box to support this requirement, in which case you ll need to obtain (or borrow) additional USB external enclosures or USB to disk cable adapters to temporarily attach the OpenBSD disks to your LOCKSS box after you have installed Linux. 6

7 Installation Checklist You need to enter several pieces of information during the installation and configuration of LOCKSS. Please collect all this information before you start the installation process so it s available when you need it. OpenBSD Transition In most situations, you should re-use the same configuration parameters for your Linux LOCKSS box that you used for your existing OpenBSD LOCKSS box. LOCKSS box basic network information Value Fully Qualified Hostname IP address Netmask (CIDR format) Gateway IP address Primary DNS IP address Secondary DNS IP address LOCKSS box advanced network information External NAT IP Address (if applicable) Proxy Server IP Address (if applicable) Proxy Port for Configuration URL (if applicable) Value 7

8 LOCKSS box information Mail relay hostname Mail relay username (if applicable) Mail relay password (if applicable) LOCKSS Administrator address Value networks accessing LOCKSS network 1 (CIDR format) Value network 2 (CIDR format) network 3 (CIDR format) passwords root password (Linux 6 character minimum) Web Administrative UI username Web Administrative UI password Value lockss Private LOCKSS Network (PLN) PLN users must enter these values during LOCKSS daemon configuration. These values should be provided by your PLN administrator. LOCKSS Daemon Item Configuration URL Preservation group(s) Value 8

9 Software Download LOCKSS Net Install CD Download and burn the LOCKSS Net Install CD image to a blank CD. The link to the CD image is published on the LOCKSS installation page located here: OpenBSD Transition Software Download Data Transition CD Download and burn the Data Transition CD image to a blank CD. Choose your data transition method There are two supported methods for transitioning data from your OpenBSD LOCKSS box to your Linux LOCKSS box discussed on the LOCKSS installation page ( Review both methods and mark below which method you choose: Method 1: Disk-to-Disk Method 2: Network Contact the LOCKSS support team before you start Please contact the LOCKSS support team by at [email protected] indicating you are preparing to start the installation. Please include the following information in your - the IP address of your LOCKSS box (your IP address must be registered with the central LOCKSS props server) - indicate if there is an external firewall between your LOCKSS box and the Internet - we ll need to give you some firewall rules to add to your external firewall if there is 9

10 BIOS Settings Reboot your LOCKSS box and enter your system BIOS. If you are unsure how to enter your system BIOS, consult your system manual or manufacturer. You are also welcome to us at and we will try to help. In the BIOS, check the following settings and change them if necessary: If you have SATA disks, check that SATA compatibility is set to AHCI (or "RAID") and not IDE-compatible or PATA emulation. Some BIOS systems may instead indicate a native or normal or compatibility mode for the SATA controller always choose the SATA mode that is native or normal. (The compatibility mode on a SATA controller is provided in BIOS for older operating systems that do not support SATA directly) Check that the power setting is set to 'Restore to previous state' -- this allows the LOCKSS box to automatically resume normal operation when power is restored after a power failure. 10

11 Installation OpenBSD Transition If you chose the Disk-to-Disk Transition method back on page 9, follow the disk swapping directions in the section titled Before Installing Linux in the Disk-to-Disk Transition Guide found at Put the LOCKSS Net Install CD in the LOCKSS box CD drive, then power on the LOCKSS box. If the LOCKSS box doesn t boot the CD, you may need to hit a boot order key (such as F12) during power-up to change the boot order. This is the first screen you ll see when booting: Hit the Enter key on your keyboard to boot. 11

12 Many lines of text will scroll down the screen it will be 10 to 40 seconds before the next screen comes up: Choose your language. The mouse does not work you must use the up arrow and down arrow keys on your keyboard to select your language. Then hit the Enter key on your keyboard to accept your choice. 12

13 Choose your keyboard type. You can use the up arrow and down arrow keys on your keyboard to select your language. Then hit the Enter key on your keyboard to accept your choice. 13

14 Configure TCP/IP. You will make several changes on this screen. To begin, hit the down arrow key on your keyboard twice to move the red cursor to Manual configuration under Enable IPv4 support, and then hit the space bar on your keyboard to select it. Then hit the down arrow key on the keyboard one more time to move the red cursor to Enable IPv6 Support and hit the space bar on the keyboard to turn off all IPv6 support. Then hit the down arrow key on the keyboard once so the OK button is highlighted. 14

15 Compare your screen to the picture above they should be identical. Press the Enter key on the keyboard to proceed. 15

16 Manual TCP/IP Configuration. Use up/down arrows to move between fields, use left/right arrows to move within each field. Referring back to the Installation Checklist you filled out earlier: put the IP Address from the Installation Checklist in the first field of the IPv4 address put the Netmask (CIDR format) from the Installation Checklist in the second field of the IPv4 address put the Gateway IP address from the Installation Checklist in the Gateway field put the Primary DNS IP address from the Installation Checklist in the Name Server field Finally, hit the down arrow key to highlight the OK button. 16

17 Compare your screen to the picture above. The values in the fields will be different, but the pattern should look the same. Press the Enter key on the keyboard to proceed. 17

18 This screen should be briefly visible. If it stays up for a 10 to 20 seconds and is replaced by a blank screen for 3-5 minutes and then shows an error message stating it was Unable to retrieve (see screen below) then there was a problem configuring the network. If you see this screen, then power off your LOCKSS box by pressing and holding in the power button until it goes off, then check that the network wiring is good, and then start again from the beginning, taking extra care when you are filling in the network information. 18

19 Install information is now downloading. You might see this screen flash by or it may stay up for up for as long as 10 minutes the length of time this screen is visible depends on how fast the Internet is working at your LOCKSS box location. 19

20 Now your mouse should work, and you should start using it. Click on Next in the lower right corner with your mouse. 20

21 Click on the Next button. 21

22 The Gateway and Primary DNS settings will match the entries you added earlier during the install process. Enter the Fully Qualified Hostname from your Installation Checklist in the hostname field on the screen if it wasn t automatically filled in. Put the Secondary DNS IP address from the Installation Checklist in the Secondary DNS field on the screen. Then click the Next button. 22

23 1) Click on the white box that says System clock uses UTC. A check mark should appear inside the box. 2) Set your time zone by choosing the location closest to you from the list of locations (it s a very long list). OR you may click with your mouse on the map in the part of world where your LOCKSS box is located, and choose a city (one of the dots) that is closest to you and in your time zone. 23

24 Ensure that the checkbox next to System clock uses UTC is checked and that the correct time zone is set for your LOCKSS box location. Press the Next button 24

25 This is the root password screen. Enter the root password you indicated in the Installation Checklist. The password should not be easy to guess. You ll need to enter it twice the characters will not appear as you type them. After entering your password twice, click on the Next button. Please note the root password in a safe place; only you have this information! 25

26 Click on the Next button. 26

27 The install will check package dependencies, this can take about a minute to complete. Then this screen indicates the CentOS installation is about to start. Click on the Next button. 27

28 A Centos Installation screen will appear. First it will indicate that it is formatting your filesystems, and then it will tell you what package it is installing. A progress bar near the bottom tracks the installation. After the last package is installed and the progress bar is full, you ll see a message that says Running post-install scripts. The entire process can take anywhere from 10 minutes to 60 minutes, depending on the speed of your LOCKSS box and its internet connection. 28

29 Remove the LOCKSS CentOS Installation CD from your CD drive, and then click on the Reboot button to complete the CentOS installation and reboot your LOCKSS box. 29

30 Firewall Configuration The LOCKSS daemon requires that port 9729 is accessible to the internet. Ports 22, 8080, and 8081 are used to administrate and monitor the LOCKSS box. The following steps will limit who can access Port 22 (SSH access). Step 1: Log in to the LOCKSS box as root. You will have to give the password you set during installation. Step 2: type /etc/lockss/lockss-config-iptables and hit the enter key. This is what you ll see: Step 3: Enter the networks accessing LOCKSS you listed in the Installation Checklist section and hit enter. If you have more than one network to add, put a space between each network, and don t worry when the line wraps on your screen. Hit the enter key when you are done. 30

31 Step 4: The next question asks you if the LOCKSS support team should be allowed access into your LOCKSS box. The recommended answer is Y (for Yes). Step 5: type service iptables restart and hit the enter key. This loads the firewall configuration you just created into your LOCKSS box. Step 6: Test an SSH connection from a remote PC to the LOCKSS box that is on one of your administrative networks to make sure you can still get in. If possible, test a SSH connection from somewhere else that is not on one of the networks you listed you should not be able to connect. 31

32 LOCKSS Daemon Configuration The LOCKSS daemon requires several configuration values from the Installation Checklist you prepared earlier. Example input is shown in the examples below your input values will be different. Step 1: Log in to the LOCKSS box as root. Step 2: We need to know how many data storage areas the Linux Installation allocated for LOCKSS. Type ls d /cache* and hit the enter key. Here s example output: In this example we see two data storage areas - /cache0 and /cache1. Record your result somewhere you ll need this information soon. Step 3: type /etc/lockss/hostconfig and hit the enter key. This is what you ll see: The Fully Qualified Hostname from your Installation Checklist should appear in the brackets. Press the enter key to accept the value. Step 4: The IP address from your Installation Checklist should appear in the brackets. Press the enter key to accept the value. 32

33 Step 5: If you listed an External NAT IP address for your LOCKSS box in the Installation Checklist, you should enter Y to this question, and then enter the External NAT IP address when prompted. If your LOCKSS box network does not use NAT, then just press the enter key to accept the default value of N. Step 6: Indicate which network you will use to initially access the Web Administrative UI by entering LOCKSS access network 1 from the Installation Checklist. The network listed in brackets is the network that your LOCKSS box is on this network automatically has access to the Web Administrative UI. You will be able to add all the additional networks you have listed in the Installation Checklist inside the Web Administrative UI. Initial Prompt: After entering LOCKSS access network 1: Step 7: LCAP port, accept default by hitting the enter key. Step 8: Proxy port, accept default by hitting the enter key. Step 9: Web Administrative UI port, accept default by hitting the enter key. 33

34 Step 10: Enter Mail Relay hostname from the Installation Checklist. After entering a mail relay hostname: Step 11: If you listed a Mail relay username and password for your LOCKSS box in the Installation Checklist, you should enter Y to this question, and then enter the username and password when prompted. If your mail relay host does not need a username and password, then just press the enter key to accept the default value of N. Step 12: Enter the LOCKSS Administrator address from the Installation Checklist and press the enter key. After entering an example LOCKSS Administrator address: Step 13: Path to java, accept default by pressing the enter key. Step 14: Java switches, accept default by pressing the enter key. 34

35 Step 15: Configuration URL, accept default by pressing the enter key. Private LOCKSS Network (PLN) Do not accept the default value enter the Configuration URL you listed in the Installation Checklist and then press the enter key. Step 16: If you listed a Proxy Server IP Address and a Proxy Port for Configuration URL for your LOCKSS box in the Installation Checklist, you should enter them here (putting a colon between the IP address and the port number) Otherwise you should just press the enter key. Step 17: Preservation group(s), accept the default by pressing the enter key. Private LOCKSS Network (PLN) Do not accept the default value enter the Preservation group(s) you listed in the Installation Checklist and then press the enter key. 35

36 Step 18: At the beginning of this section (Step 2) you recorded the names of the data storage areas we need that information now. In our example earlier, we recorded two data storage areas, /cache0 and /cache1. To derive our Content storage directories list, we append /gamma to each storage area, then use a semicolon to separate the first storage area from the second. After input: NOTE: If you had only one storage area (/cache0) listed from Step 2, then you would enter /cache0/gamma for this step. Step 19: Temporary Storage Directory, accept default by pressing the enter key. Step 20: Web Administrative UI username input lockss and then press the enter key. After input: Step 21: Web Administrative UI password - you are asked to enter it twice. The password will not be visible as you type. Make sure to remember this username and password you ll use the Web Administrative UI to interact with the LOCKSS system. 36

37 Step 22: Compare your settings with your Installation Checklist. If it s OK, press Y to save the LOCKSS configuration you have just created. Otherwise, press N and you will be taken back to the beginning. Step 23: After confirming the configuration, you ll be asked to confirm the creation of your content storage directory(s), /var/log/lockss, and /cache0/gamma/tmp answer Y to all questions. 37

38 OpenBSD Transition STOP HERE DO NOT DO THE NEXT STEP! DO NOT REBOOT! Now you should transition data from your OpenBSD disks to your Linux disks using the instructions below for the method you chose back on page 9. Method 1: Disk-to-Disk Transition Method Follow the instructions in the section titled After Linux is installed in the Disk-to-Disk Transition Guide found at Method 2: Network Transition Method 1. Log out of your newly-installed Linux LOCKSS box. 2. Follow the instructions in the Network Transition Guide found at When you have completed the OpenBSD Transition; return to this document and complete the steps in the last section of this document titled Maintenance on page 41. Step 24: (ONLY if you are not doing an OpenBSD Data Transition) Start the LOCKSS daemon by typing /etc/init.d/lockss start and hit the enter key. 38

39 Accessing LOCKSS The LOCKSS daemon starts automatically when the LOCKSS box is booted. Go to the computer you want to use the Web Administrative UI from. Open an Internet Browser, enter the Fully Qualified Hostname from your Installation Checklist into the address bar, and then add on :8081 to the end. When you connect, you should be challenged for a username and password: Enter the username as lockss and enter the Web Administrative UI password you listed in the Installation Checklist. Click on the OK button, and you should see something the Web Administrative UI: 39

40 40

41 Maintenance You should ensure that Linux is currently with updates and configured correctly to minimize vulnerabilities. We recommend you: 1. Configure the system to automatically maintain itself current with the Linux distribution and LOCKSS software level. 2. Avoid enabling any inessential services on your LOCKSS box. Setting up automatic Linux upgrades is very simple and only needs to be done once. Upgrades to the LOCKSS software will also occur automatically after you complete the following steps. Step 1: Log in to the LOCKSS box as root. You will have to give the password you set during installation. Step 2: type chkconfig yum-cron on and hit the enter key. There is no output from this command Step 3: type service yum-cron start and hit the enter key. Congratulations! You have finished the LOCKSS on Linux installation! We would be delighted to get your feedback and any improvements you might suggest to us please your comments to 41