The Linux Boot Sequence

Transcription

1 The A good understanding of the Linux boot sequence will be useful as the course progresses. First, the actions which start network communication occur during the boot sequence. You ll want to know how this happens so that you can control it. The same shell scripts which start networking during the system boot sequence can be used manually to stop and start network communication. This is useful when you re experimenting with the network configuration. Second, many of the activities that you ll be doing over the course of the semester require you to modify system files. It s entirely likely that at some point you ll find yourself sitting in front of a workstation that no longer works properly but will still respond to commands from the terminal. A good understanding of the Linux boot sequence may allow you to avoid rebooting the workstation. Instead, you can invoke the necessary startup actions manually. This may not seem important in the Network Lab, but it can be critically important when rebooting a production system is just not an option. The concepts used in the Linux boot sequence are common to other operating systems, but the details will certainly differ. The material in this lecture is drawn from the Linux System Administrator s Guide and various other pieces of documentation, from the boot scripts themselves, and from the source code of some of the programs involved. A lot of useful documentation for Linux is available online from the Linux Documentation Project ( Regrettably, some of it is getting to be a bit old and out-of-date. The System Administrator s Guide contains useful introductory material on Linux and pointers to other introductory material. If you re new to Linux, this is mandatory reading. The locate and find commands are helpful when you re looking for a particular file but don t remember the exact name or location. The man and info commands will give you access to much of the online documentation. Try man man and info info. Many Linux software packages include documentation as part of their distribution package. Most often this will end up in /usr/share/program or /usr/share/doc. 1 January, 2014

2 Back to the boot process. Let s begin at the beginning. When a PC (or clone) powers up, the hardware transfers control to a program called the BIOS, which resides in ROM. The BIOS is not too bright, but it knows enough to do two important tasks. The first is what s called power on self test (POST), a combined sanity check and initialisation of the computer s hardware. The BIOS scans the components in the system, executing any initialisation actions that are required. In today s systems, complex components will provide their own initialisation routines, and the BIOS will arrange to execute them. The second is to scan the set of potential boot devices looking for one that contains a program called a boot loader. The boot loader is a slightly more intelligent program whose job is to load the operating system into memory and start it running. Typically the set of potential boot devices will include a selection of removable media devices (e.g., a CD and/or DVD drive, or a device connected to a USB port) and one or more internal disk drives. The device must look like a disk drive with a bootable file system i.e., it must contain a Master Boot Record (MBR) in the first sector of the storage media. The first valid device is used. This description captures the spirit of system startup, but falls a fair bit short of the reality of what a modern BIOS can do. It completely ignores UEFI (Unified Extensible Firmware Interface), the up-and-coming replacement for the BIOS. For Linux, the most common bootloader is GRUB2 (Grand Unified Bootloader). Your best source of information about GRUB2 is the documentation that accompanies the software distribution. GRUB2 is too big to fit in a single sector of a disk. A small piece is placed in the boot sector. Its purpose is to find and load the remaining, larger pieces, which complete the boot by finding and loading the operating system. If your computer is set up to boot multiple operating systems (Linux and Windows, for example), GRUB2 will prompt you to choose the operating system you want to start. Once loaded into memory and started by the bootloader, the Linux kernel begins its startup sequence: It scans the hardware configuration, initialises device drivers, and takes the CPU into protected mode with virtual memory. 2 January, 2014

3 It mounts the root file system (read only). Once these preparations are completed, the kernel starts a user process which is responsible for starting everything else. For most of the history of unix systems, this user process was called init. The init program has process id 1, the root of the process tree. All processes running on the computer are descended from init. init remains in existence until the computer is halted and has overall responsibility for the state of the system. For many years, there was one framework for startup, now referred to as SysV (Unix System V), developed in the early 1980 s. A set of run levels was used to organise shell scripts that were executed to control system services. Recently, alternative frameworks have appeared that address perceived limitations of the SysV framework. Ubuntu distributions use an event-based framework called Upstart. Fedora distributions use a dependency-based framework called Systemd. Both provide legacy support for the SysV framework. This is a good point to explain the concept of a run level. As originally used in the SysV framework, a run level is an integer between 0 and 6 which specifies the overall system state (usually called a mode). Here s the conventional set of run levels for a Linux system: 0: Halt the system; setting the run level to 0 will cause Linux to do an orderly shutdown and halt the computer. 1: Single-user mode; in this state only the kernel and a bare minimum of essential system services are running. 2 5: Multi-user mode; in this state the system is capable of supporting multiple users. 6: Reboot; the system is halted and then automatically starts to boot. Commonly, run levels 2 4 provide a command line interface and run level 5 provides a GUI for login and a window-oriented work environment. Single-user mode (also called rescue mode) is important for system administrators. This state is used for system maintenance, upgrade, and repair which cannot be performed in multi-user mode. Often this is because the system state would not be consistent, and thus would be unsafe, while the work is in progress. Another advantage is that excluding other users avoids unnecessary system activity. 3 January, 2014

4 Run levels are just conventions; while all unix systems follow the same general startup sequence, the specific mapping of run levels to system modes can vary from one to the next. SysV Framework It s useful to know about the SysV framework for two reasons: first, because you may encounter older systems that use it; and second, to understand the compatibility features of the Upstart and Systemd frameworks. In the SysV framework, init looks for the file /etc/inittab for instructions on how to proceed. Lines in inittab have the form id : runlevel(s) : action : command The id is just a convenient identifier, with no particular meaning to init. The command is a command to be executed, either a program or a shell script. runlevel and action control when and how command is executed. The first thing that init looks for in inittab is a line with the keyword sysinit in the action field. This is the command that should be executed to continue system startup. An incomplete list of functions performed at this point includes: set system configuration variables for use as the boot sequence progresses; continue device initialisations: usb, clock, keyboard, network, and disk handling optimisations; enable paging to disk; check the root file system and remount it read/write; check and mount other local file systems; enable logical volume management; clean up system status files; Limited network connectivity may be enabled at this stage to allow some system initialisation over a network. When the command specified for sysinit finishes, control returns to init, which then looks for another line in inittab with the keyword initdefault in 4 January, 2014

5 the action field. This entry has no associated command; its purpose is to specify the default run level which should be entered to complete the boot process. This run level is used to construct the name of a directory, typically of the form rcn.d, where N is replaced by the run level. In the directories rcn.d are symbolic links with names of the form [S K]ddcommand. If you use ls -l to list one of the rcn.d directories, you ll see that the links point to scripts in another directory, typically init.d. Each script controls some system service or activity. The initial S or K specifies whether the script is used to start (S) or kill (K) the service or activity. When entering a run level, the kill scripts are executed first, followed by the start scripts. The two digits dd specify a number between 00 and 99. This number determines the order of execution within the start and kill groups. The command portion of the file name is most often simply the name of the script. Typically, each script can be called with one of four parameters: start, stop, status, and restart; some have additional parameters. For the purposes of Cmpt 471, notice that bringing up network interfaces is one of the key actions in the boot sequence. Typically there is a script in the init.d directory, often called network, which can be invoked from the command line to stop and start network activity. Once init knows the default run level, it looks for all lines in inittab which specify that run level in the runlevel field and executes the command specified in the action field for each line. What other functions are controlled by the scripts in /etc/init.d? An incomplete list includes Internet services (DNS, SNMP, SSH, plus the meta-dæmon xinetd), and a firewall. Remote file sharing using Samba or NFS. Display management (responsible for putting up the GUI login screen). The cron dæmon, which can run a job at a specified time (handy for scheduling backups, accounting, etc.). Printing and logging services. 5 January, 2014

6 The Network Information Service (NIS), formerly known as the yellow pages (YP). This is a set of databases maintained by a central server which can be queried over the network. Remote procedure call and remote execution dæmons. Upstart Framework The SysV framework is giving way to newer boot frameworks that address some of the perceived deficiencies in the SysV framework: lack of parallelism; duplication of effort when creating the shell scripts that control startup and shutdown of services; and an inability to react to dynamic changes in the system configuration (e.g., failures, plug-and-play devices). The older of the new frameworks is Upstart 1. Upstart is used in the Ubuntu systems which make up the Virtual Network Lab (VNL), and in the physical workstations in the CSIL. The operation of upstart is based on the concept of events. Actions are triggered by the occurrence of events. Any program can communicate an event to the upstart dæmon, which will then relay the event to interested jobs. At startup, upstart processes a set of jobs defined by configuration files held in /etc/init (previously, /etc/event.d). A job can request that the upstart dæmon notify it when a particular event or combination of events occurs. To get things rolling, when the upstart dæmon is started as as process 1, it produces a startup event and sends it to all interested jobs. Each configuration file specifies a single executable file or a sequence of shell commands to be executed when triggered by an event. The content of the configuration files is organised in units called stanzas. Commands can be either services or tasks. Services are expected to execute indefinitely and are respawned (restarted) if they exit. Tasks are expected to execute once and exit. 1 For compatibility (and confusion) with the SysV framework, the upstart dæmon binary is also named init. 6 January, 2014

7 The start on stanza defines the events that will cause the associated command to be executed. The stop on stanza defines the events that will cause the associated service to be stopped. Application programs can communicate events to the upstart dæmon using the D-Bus message bus. The initctl application can be used to generate events from the command line and shell scripts. For a complete description of the Upstart framework, see the Upstart Intro, Cookbook, and Best Practices at upstart.ubuntu.com/cookbook. Systemd Framework The Systemd framework is a dependency-based framework. It is based on the philosophy that the vast majority of dependencies between the programs invoked during system startup can be captured in data dependency relationships: communication via well-known sockets and D-Bus names; file system mounts; and other device dependencies (e.g., availability of a network interface or presence of a USB device). Superficially, this sounds a lot like the Upstart framework, but by exploiting buffering capabilities built into the Linux kernel, much greater parallelism can be acheived. To give a concrete example, suppose that service A sends information to service B via a well-known socket. In the SysV or Upstart frameworks, this would require that service B finish its initialisation and be ready to accept data via the socket before service A could begin to start. The Systemd framework provides the ability to create the well-known socket for service B separate from starting the dæmon for service B. Creating a socket typically takes much less time than actually starting a service. Systemd 2 will create the well-known socket for service B, then start service A and service B in parallel. If service A tries to send information to service B before service B is ready to accept it, the data is buffered using the normal data buffers supplied by the kernel for any socket and service A can continue with its own startup actions. When systemd starts service B, it passes the file descriptor for the well-known socket to service B. Once service B is ready to accept data, it begins to read from the socket and the buffered data is processed. No information is lost, and service A was not delayed unnecessarily. Where the Upstart framework uses jobs, the Systemd framework uses units. Unit configuration files are organised in sections using a syntax which has its 2 The developers of the Systemd framework were more sensible than the developers of the Upstart framework and named the dæmon systemd. On a system using the Systemd framework, process 1 will be systemd. 7 January, 2014

8 roots in Microsoft.ini files. As with the Upstart framework, a unit can specify a service or a task. In addition, the Systemd framework defines units that represent sockets, devices, filesystem mount points, and individual files. Another type of unit, a target, is used to represent a named synchronisation point in the startup sequence. To take maximum advantage of parallelism and implicit data dependencies, a service will have a unit configuration file that specifies how to start the service, and a companion unit configuration file specifying the well-known socket or D-Bus name that will be used to communicate with the service. When systemd starts, it looks in /usr/lib/systemd and /etc/systemd for unit configuration files. A unit configuration file can specify explicit dependencies and precedences. Note that dependency and precedence are distinct notions in the Systemd framework. Using the example above, Service A has a dependency ( requires ) service B, but (as outlined above) systemd may still start both services in parallel. If it s necessary that service B be completely functional before any attempt is made to start service A, this must also be specified explicitly (a before precedence). To get things rolling, systemd will attempt to start the unit specified as the default target (either /usr/lib/systemd/system/default.target or a unit specified on the command line when systemd is started). The default target will have dependencies, and those dependencies will themselves have dependencies. Systemd will recursively analyse the dependency tree and construct a transaction that will start the specified services and tasks with as much parallelism as is permitted by the explicit precedence Notifications of state changes for units come from the Linux kernel and from systemd s own monitoring of processes that it starts. The systemctl program is used to control systemd from the command line. For an excellent description of the underlying principles of the Systemd framework, see 0pointer.de/blog/projects/systemd.html. The online documentation is the right place to look for details about Systemd unit configuration files and capabilities. As of 2014, it appears that the Upstart framework is giving way to the Systemd framework. There are other variations on the system startup process. Oracle Solaris, for example, uses a framework called the Service Management Facility (SMF) which 8 January, 2014

9 offers similar capabilities to the Systemd framework (albeit with completely different terminology). Beyond Init Modern Linux systems typically provide a graphical login screen that s displayed on the physical console and a number of virtual consoles that provide independent command-line logins. The exact method of creating the graphical login and virtual consoles differs in each of the startup frameworks described above; here, the notes will concentrate on function. Why is it useful to have virtual consoles that provide a command-line login, distinct from the graphical login displayed on the physical console? Most often, these consoles are used to gain access to the system when the graphical login misbehaves, and to modify configuration files that are written at the end of a login session that uses the graphical login (and therefore cannot be modified successfully from a graphical login session). Some distributions are configured to echo the details of system startup to one of the virtual consoles in addition to recording the information in the system log /var/log/messages. Each console provides an entirely independent login session, just as if the computer provided seven separate displays. In particular, logging off on one console does not automatically log you off from other consoles. To change from one console to another, use the key combination alt-fn, replacing n with 1 7. It may be a little harder to escape from the graphical login console; try ctl-alt-fn if alt-fn doesn t work. A command-line login provides you with a single command shell. Once you ve authenticated yourself to the system, a command shell is started for you. There are many different command shells available; the one that is started for you is specified in your entry in the /etc/passwd file. This first shell is called the login shell. Table 1 lists some of the most common unix command shells. Sh and its descendants ksh and bash share a common syntactic structure. The primitive operations provided by these shells to break up text strings tend to be character oriented. They offer excellent control of i/o redirection. Bash is the most common command shell in use today and is the default shell in the VNL and CSIL. Sh is the lowest common denominator. When writing shell scripts where portability to many varieties of unix is an important goal, restricting yourself to sh capabilities is an effective strategy. 9 January, 2014

10 Shell Login File Init File Comments sh.profile n/a The original shell, and the lowest common denominator. Still important because many system-level shell scripts are written using sh capabilities. ksh.profile.kshrc (Korn shell) Enhanced version of sh. bash.profile.bashrc Bourne again shell. Greatly enhanced version of sh. csh.login.cshrc C shell. Comparable in power to sh, but with a more friendly syntax. tcsh.login.tcshrc Tenex C shell. Greatly enhanced version of csh. Table 1: Common Unix Command Shells Csh and its descendant tcsh also share a common syntactic structure, different from sh and descendants. The name of the C shell comes from a faint resemblance to the syntax of the C programming language. The primitive operations provided by these shells to break up text strings tend to be word oriented. Control of i/o redirection is somewhat clumsy. Once a shell is started, it will search for an initialisation file ( Init File in the table). The name 3 varies from shell to shell. Typically, a system-wide initialisation file, found in /etc or a subdirectory, will be executed first. The shell will then look for an initialisation file in your home directory. The shell initialisation file is executed every time you start a new command shell (which you ll do often in unix). The login shell executes another initialisation file, listed in the Login File column of the table. Again, it will look for a file in /etc and in your home directory. The exact execution order for the Login File and Init File differs from one shell to the next; consult the online documentation. When the login shell finishes executing its initialisation files, it will present you with a prompt and you can get down to work. The console that provides the graphical login screen runs a different greeter program than that run by the command-line virtual consoles. The graphical login is presented courtesy of an X Window System display manager program. As you might suspect, the unix world provides several to choose from. The login screen will look generally familiar from one system to the next there are places to 3 Notice the leading. in these file names; they are hidden files, to cut down on the clutter in your home directory. Unix has lots of these. 10 January, 2014

11 type your id and password, and (usually) some option menus. To explain any further about what s going on here, we need to take a step back and talk about the X Window System. 11 January, 2014