LINUX NETWORK SECURITY PETER G. SMITH CHARLES CHARLES RIVER MEDIA, INC. Hingham, Massachusetts
Contents Preface xvii 1 Introduction: The Need For Security 1 1.1 Introducing the Enemy 1 The Hacker Myth 3 1.2 Just Who Is at Risk? 3 1.3 The Implications of a Compromise 5 1.4 Hackers and Crackers 8 Crackers 9 Summary 10 Endnotes 11 References 11 2 Understanding the Problem 13 Part I: Attacks Against Linux 14 2.1 Exploits and Vulnerabilities 14 Weak Passwords 14 suid Binaries 16 The Buffer Overflow 18 The Basics 18 Race Conditions 23 Key Logging 28 Unauthorized X Windows Access 29 2.2 Trojans and Backdoors 30 The Sendmail Trojan 30 Modifying /etc/passwd 31 Modifying/etc/inetd.conf 32
vl Contents Creating suid Shells 33 Trojaned System Binaries 34 CGI Abuse 34 2.3 Rootkits 36 FLEA 36 TOrn 39 Adore (2.4.x kernel) 41 Adore-ng (2.6.x kernel) 46 Part II: Attacks Against the Network 46 2.4 Denial of Service (DoS) 46 Ping-Pong Attack 48 Distributed Flood Nets 48 The Smurf Attack 50 Fragmentation Attacks 53 SYN Flodding 53 Nonbandwidth-Oriented DoS Attacks 54 2.5 TCP/IP Attacks 55 ARP Spoofing 55 DNS Attacks 56 Packet Sniffing 58 Switched LAN Sniffing 61 IP Spoofing 64 Man-in-the-Middle Attacks 69 Replay Attacks 69 Injection Attacks 70 Summary 70 Endnotes 71 References 71 3 A Secure Topology 73 3.1 Network Topology 74 Switches, Hubs, and Sniffing 74
Contents vii Gateways, Routers, and Firewalls 79 Wireless Networking 81 Network Address Translation (NAT) 83 The DMZ 86 3.2 A Detour into Iptables 89 Preparation 89 Patch-O-Matic 89 Installation 89 The Life Cycle of a Packet 91 Using Iptables 93 General Syntax 94 3.3 Implementing the Three-Legged Model 103 Firewall Rulesets 103 Traffic Routing 109 3.4 Network Tuning with the / p ro c Filesystem 110 Sysctl 111 Routing Options 113 Security Settings 115 ICMP Messages 116 TCP Settings 118 3.5 Virtual Private Networks and IP Security 120 Virtual Private Networking (VPN) 120 Road Warriors 120 IPsec 121 Implementing a VPN with IPsec 125 Summary 129 Endnotes 130 References 131 4 Assessing the Network 133 4.1 Portscanning with Nmap 135 Scan Types and Options 135
viii Contents Nmap in Use 4.2 Vulnerability Auditing with Nessus Installing Nessus 4.3 Web Site Auditing with Nikto Summary Endnotes References 5 Packet Filtering with Iptables 5.1 The Components of an Iptables Rule Generic Matches TCP-Specific Matches UDP-Specific Matches ICMP-Specific Matches Matching Extensions Targets 5.2 Creating a Firewall Ruleset Protecting the Firewall Protecting the DMZ ICMP Messages TTL Rewriting Blocking Unwanted Hosts Filtering Illegal Addresses Local Packet Filtering 5.3 Firewall Management: Dealing with Dynamic IP Addresses DHCPCD Blocking and Unblocking Hosts Using GUI Management Tools Summary Endnotes References 136 146 146 153 157 158 159 161 163 163 166 168 169 169 174 178 179 182 184 185 185 186 191 196 196 198 200 202 203 203
Contents ix 6 Basic System Security Measures 205 6.1 Password Protection 206 The /etc/passwd file 207 Shadowed Passwords 208 Password Protection Algorithms 211 Login Control with /etc/login.defs 211 Password Strategies 212 Enforcing Strong Passwords 214 6.2 User Control and PAM 217 PAM Configuration 218 Password Control 222 Limiting Resources 224 The Non-PAM Way 226 Controlling su Access 226 Creating a Chroot Environment 227 Other PAM Modules 227 6.3 Services 229 Common Services 229 Starting and Stopping Services 233 6.4 Tightening User Permissions 239 World-Writable Files 239 SUID and SGID Files 240 Partitions and Mount Options 240 Ext2 Attribute 242 6.5 Delegating Root Access 243 /etc/sudoers 244 SUDO Security 247 6.6 Physical Security 253 Removing the CD-ROM and Floppy Drive 253 Case Locks 253 Location 254 Keyloggers 254
Contents The BIOS 254 Summary 257 Endnotes 258 References 259 7 Desktop Security 261 7.1 Viruses and Worms 262 Clam 262 General Antivirus Precautions 264 7.2 Safe Web Browsing 264 Scripting 264 Cookies 270 Authentication 272 Digital Certificates 278 7.3 E-Mail 280 Client-Side Mail Filtering 280 E-Mail Integrity 282 7.4 X Windows 283 Host-Based Authentication 284 Token Authentication 285 Summary 286 Endnotes 286 References 287 8 System Hardening 289 8.1 Choosing a Distribution 290 General Distributions 290 Specialized Distributions 293 8.2 chroot Environments 294 Jail Construction 295 Escaping from chroot Jails 300 8.3 Stripping Down Linux 301
Contents xl Unnecessary Binaries 301 Compilers and Interpreters 302 Other Tools 303 Placing System Utilities on CD-ROM 303 Choosing Applications During Installation 304 Post-Installation Package Management 305 8.4 Memory Protection 307 StackGuard 307 MemGuard 308 Stack-Smashing Protector 309 Bounds Checking 311 CRED 312 Libsafe 313 PaX 315 Nonexecutable Memory (NOEXEC) 315 Address Space Layout Randomization (ASLR) 316 Buffer Overflow Detection 320 Conclusion 322 8.5 Policing System Call with Systrace 323 Installation 323 Components of a Policy File 324 Policy File Creation 327 Automatic Policy Generation 327 Policy Enforcement 329 Interactive Policy Enforcement 330 Third-Party Policy Files 331 Summary 332 Endnotes 333 References 334 9 Access Control 335 9.1 Introduction to Access Conrol 336
xii Contents Discretionary Access Control (DAC) 336 Mandatory Access Control (MAC) 336 Domain Type Enforcement (DTE) 336 Linux Security Modules (LSM) 338 9.2 Role-Based Access Control with Grsecurity 339 Installation 340 A Note on Group Memberships 340 Security Level 341 Address Space Protection 341 RBAC Options 342 Filesystem Protection 342 Kernel Auditing 345 Executable Protections 346 Network Protections 347 Logging Options 349 Access Control 349 ACL Structure 350 Implementing Grsecurity 359 9.3 LIDS: Linux Intrusion Detection System (LIDS) 364 Installation 364 Lids Administration 366 Sealing the Kernel 366 LIDS-Free Sessions 367 File ACLs and Capabilities ACLs 368 Implementing LIDS 374 9.4 Other Access Control Projects 381 SELinux 381 Rule-Set Based Access Control (RSBAC) 382 DTE 382 Comparing Techniques 383 Summary 384
Contents xiii Endnotes 385 Reference 385 10 Securing Services 387 10.1 Web Services and Apache 388 Configuration 388 Version Hiding 389 Resource Limiting 391 Access Control 391 Web Scripting 398 Secure Perl-CGI Programming 399 CGIWrap 405 PHP 406 Ch rooting Apache 407 10.2 SSH 412 Configuration 412 Hiding the SSH Server Version 413 Connection Tunneling 414 10.3 NFSandNIS 415 NFS 415 10.4 DNS and BIND 423 General Precautions 423 DNS Security Extensions (DNSSEC) 432 Split Functionality Nameservers 436 10.5 E-Mail 438 Sendmail 439 Qmail 447 POP3 and IMAP 448 Stunnel 448 10.6 FTP 451 WU-FTP 451 VSFTPD 454
xlv Contents TLS (SSL) Support 455 Summary 455 Endnotes 456 References 456 11 Keeping Secure 459 11.1 Staying Up to Date 460 Application Mailing Lists 460 Security Mailing Lists 461 Up2Date 462 Patch Management with Ximian Red Carpet 462 11.2 Logging and Log Analysis 464 Protecting /var/log 465 Syslog 465 /var/log/wtmp 467 BSD Process Accounting 468 Log Analysis with Lire 470 11.3 System Integrity 471 Tripwire 471 Post-Install Configuration 475 Using Tripwire 477 Some Closing Thoughts 482 Chkrootkit 483 11.4 Intrusion Detection 485 Snort 485 11.5 Recovering from a Compromise 489 Discovering a Security Breach 489 Analyzing the System 490 Seeking Justice 490 Summary 491 References 492
Contents XV Appendix A Recompiling the Linux Kernel 493 Obtaining the Kernel Source Code 494 Configuring the Kernel 495 Compiling the Kernel 495 Installing the Kernel 496 LILO 496 GRUB 497 Endnote 498 Appendix B Kernel Configuration Options for Networking 499 Network Support -> Networking Options 500 Networking Support -> Networking Options -> TCP/IP Networking 500 Networking Support -> Networking Options -> Network Packet Filtering -> IP: Netfilter Configuration 501 Networking Support -> Networking Options -> Network Packet Filtering -> IP: Netfilter Configuration -> Connection Tracking 502 Networking Support -> Networking Options -> Network Packet Filtering -> IP: Netfilter Configuration -> Iptables Support 502 Networking Support -> Networking Options -> Network Packet Filtering -> IP: Netfilter Configuration -> ARP Tables Support 503 Appendix C NAT Firewall Script 505 Appendix D Complete Firewall Script 509 Appendix E Cryptography 517 Cryptography Basics 517 Encryption Algorithms Defined 517 Digest (Hash) Algorithms Defined 518 Attacks Against Cryptography 518 Legal Issues 518 Popular Encryption Algorithms 519
XVi Contents DES 519 Double DES and 3DES 519 AES 519 RC2 519 RC4 521 RC5 521 RC6 521 RSA 521 Blowfish 522 IDEA 522 Hash Algorithms 522 MD2 523 MD4 523 MD5 523 SHA 524 Public Key Cryptography (PKC) 524 Digital Signatures 525 PGP, PGPI, OPENPGP, and GNUPG 525 Security 526 References 526 Appendix F About the CD-ROM 527 System Reqirements 527 CD-ROM Files 528 Chapter 2 528 Chapter 3 528 Chapter 4 529 Chapter 5 529 Chapter 7 530 Chapter 8 530 Chapter 11 530 Index 531