Top Secret KVM, Lessons Learned from an ICD 503 Deployment

Transcription

1 Top Secret KVM, Lessons Learned from an ICD 503 Deployment Frank Caviggia July 30, 2014 Defense in Depth

2 Overview System Configuration - Hardware - Software Security Controls - Security Concepts - Government Standards - Hardening Scripts (STIG FIX, SSG) - DISA STIG Kickstart DVD SELinux Concepts - What is SELinux? - DAC and MAC - Polyinstantiation and Multitenancy KVM Security Features - svirt SELinux Labels on VMs - Multiple Firewall Levels - cgroups Control Groups (Limiters) More Information 2

3 System Configuration 3

4 System Configuration: Hardware Commodity Hardware Rack Servers - Dell R200s (IdM) - Dell R710s (RHN) Blade Server - HP C7000 with ProCurve 6120 Switch - 4x HP BL460C G7 Servers - HP D2200sb Storage Blade 4

5 System Configuration: Software Red Hat Software - Red Hat Enterprise Linux 6.5 Server x86_64 - Authentication (IdM) [389-DS(LDAP)/Kerberos] - Red Hat supported version of FreeIPA - Kernel Virtual Machine (KVM) - Red Hat Enterprise Virtualization (RHEV) RHEV-M (Management Console for KVM Hypervisors) - Red Hat Network (RHN) Satellite 5.6 5

6 System Configuration: RHEL, RHEV, KVM Red Hat Enterprise Linux 6: - RHEL is Common Criteria Evaluated (Certified EAL 4+) - FIPS (Level 1 Certified, Level 2 In Evaluation) - Linux Unified Key Setup (LUKS) Encryption (Data at Rest) 8 Keys - SSH uses AES256 (Counter mode) Encryption (Data in Motion) - Web portals (Apache HTTPD) will require PKI authentication to view login page Red Hat Enterprise Linux 6 EAL4+ OSPP, including Labeled Security, Advanced Audit, Advanced Management, and Virtualization Extended Modules Dell HP IBM SGI (report, target) Kernel Cryptographic API "2.0" #1901 Certified, Level 1 Disk Volume Cryptographic API "2.0" #1933 Certified, Level 1 libgcrypt "2.0" #1757 Certified, Level 1 OpenSSH Client "2.0" #1791 Certified, Level 1 OpenSSH Server "2.0" #1792 Certified, Level 1 OpenSSL "2.0" #1758 Certified, Level 1 Openswan "2.0" #1859 Certified, Level 1 NSS (Freebl) #1710 Certified, Level 1 NSS #1837 Certified, Level 1 6

7 System Configuration: RHEL, RHEV, KVM (continued) Kernel Virtual Machine (KVM): - Type 1 Hypervisor (Bare-Metal) runs as Kernel Module - Utilizes native virtualization in Intel (VT-i,VT-d) and AMD hardware - RHEL/KVM is Common Criteria Evaluated (Certified EAL 4+) - Inherits Security (SELinux, Auditing) and Performance - VM Limits (Leading Performance SPECvirt) CPUs - 4TB RAM* KVM SPECvirt 2010 Results KVM vs VMware * As of RHEL 6.5/ RHEV 3.4 and newer 7

8 System Configuration: RHEL, RHEV, KVM (continued) Red Hat Enterprise Virtualization Manager (RHEV-M): - Web-based portal access to Virtual Machines (VMs) - Role Based Access Control (RBAC) to VMs - SPICE protocol (plug-ins for IE or Firefox) or Standalone Client - Support for up to 4 independent monitors per VM - Akin to vcenter in VMware 8

9 System Configuration: RHEL, RHEV, KVM (continued) Red Hat Enterprise Linux Lifecycle: - 10 years of support (up to 13 years with just security fixes) - Common Vulnerabilities and Exposures (CVE) are fixed through Red Hat Security Advisory (RHSA) process: - Ensures IAVAs are patched - Ensures system stability and support (backporting) - There are no licenses, only subscriptions (stable budgeting) - Upgrades are included with subscription Support Lifecycle of Red Hat Enterprise Linux 6 9

10 System Configuration: RHN Satellite Red Hat Network Satellite IAVA patching and validation (patch management) - File provisioning to connected hosts (configuration management) - SCAP compliance scans (continuous monitoring) more on this later RHN Satellite: SCAP Compliance Reporting 10

11 Security Controls 11

12 Security Overview Security is like an onion the more layers you peel the more you cry Goal: Create a secure virtualization environment using a standard set of packaged scripts, configurations, and policies to be deployed across systems. Controls are implemented through the following mechanisms: - Hardening Scripts, Kickstart Installation - Discretionary Access Controls (DAC) - SELinux Policies - Mandatory Access Controls (MAC) - Network Controls (TCP_WRAPPERS, iptables, ebtables) - Process and Memory Controls (cgroups) - Administrative Controls (physical, policies, etc.) - Continuous Monitoring (SCAP, RHN Satellite) 12

13 Security Overview: Government Regulation There are multiple government standards and regulations some of which overlap: CAPP 1 FIPS RBACPP 2 LSPP 3 Cross Domain Controls NIST (USGCB) NSA SNAC DISA STIGS Common Criteria EAL 4+ System Security Controls 1 Controlled Access Protection Profile (CAPP) 2 Role-Based Access Control Protection Profile (RBACPP) 3 Labeled Security Protection Profile (LSPP) 13

14 Hardening Scripts Apply Security Best Practices to Base Operating System Hardening will be applied by shell scripts, configurations, and policies based upon several government standards and open-source projects to standardize configuration: SCAP Security Guide1 - RHEL 6 SCAP, Security Configuration NIST United States Government Configuration Baseline (USGCB)2 DISA Unix STIGs - Aqueduct Project3 - Tresys Certifiable Linux Integration Platform (CLIP)4 NSA Security Configuration Guide5 - USB blocking, configurations, and other lockdowns (Gnome) 1 SCAP Security Guide USGB Content - 3 Aqueduct Project Tresys CLIP Project NSA SNAC Guide

15 Hardening Script: Implementation The hardening shell script serves several purposes in hardening the system: - Distributes baseline system configurations and policies for authentication, auditing, accounts, and services - Modular code in folders and separate scripts allows for adaptation to meet changing system and security needs of customer code - Verifies application of hardening with logging, hardening can be re-applied in case of modification of baseline, fits in with continuous monitoring apply.sh CAT I CAT II CAT III CAT IV NIST NSA SNAC Hardening Script Function (apply.sh) gen1000.sh gen2000.sh gen1000.sh gen9999.sh gen2000.sh gen1000.sh gen9999.sh gen2000.sh gen1000.sh gen9999.sh gen2000.sh nist1000.sh gen9999.sh nist2000.sh nsa1000.sh nist9999.sh nsa2000.sh nsa9999.sh 15

16 Hardening Script: Packaging Hardening scripts were packaged in RPM for the following reasons: - Integrity verification # rpm V stig-fix-1.0.el6 Verify the integrity of an RPM - Integrated version control and configuration management - Distribute scripts via RHN Satellite Server Check out the open source project here: 16

17 Security Configuration Automation Protocol (SCAP) SCAP is implemented on Red Hat Enterprise Linux by OpenSCAP (oscap) and the SCAP Security Guide (SSG) developed with collaboration with the NSA, NIST, and DISA. RHN Satellite can run SCAP Scans against a defined security baseline to check for configuration compliance on a schedule. This helps to maintain continuous monitoring: # oscap xccdf eval --profile stig-rhel6-server-upstream --results results.xml --report report.html --cpe /usr/share/xml/scap/ssg/content/ssg-rhel6-cpe-dictonary.xml /usr/share/xml/scap/ssg/content/ssg-rhel6-xccdf.xml OpenSCAP XCCDF System Compliance Check # wget # wget # oscap xccdf eval --results results.xml --report report.html com.redhat.rhsa-all.xccdf.xml OpenSCAP XCCDF Patch (CVE) Compliance Check SCAP Terms & Definitions - XCCDF (extensible Configuration Checklist Description Format) - Creates checklist for security configuration on a target system - OVAL (Open Vulnerability and Assessment Language) - Standardized security information content - CPE (Common Platform Enumeration) Dictionary - Names and Metadata for security Evaluation - CCE (Common Configuration Enumeration) - Identifies mappings between SCAP security checks and STIG/NIST settings 17

18 Security Configuration: DISA STIG Kickstart DVD The hardening script RPM was combined with a customized Kickstart to produce a standardized installation DVD to help meet security requirements right from installation. Screenshots of DISA STIG Kickstart DVD More Information: 18

19 SELinux Concepts 19

20 20

21 SELinux: Overview Security-Enhanced Linux (SELinux) was a research project sponsored by the NSA to provide Mandatory Access Controls (MAC) to the Linux kernel SELinux mainlined into the Linux kernel in August 2003 (2.6.0-test3), it was first enabled for general use in Red Hat Enterprise Linux 4 Kernel enforcement based on security context provided by policies rather than standard permissions. Think of it like a chroot jail on steroids or watertight compartments in ship design Watertight Compartments in Ship Design 1 SELinux Policy Example 1 Picture Source: Wikipedia Bulkhead (Partition) 21

22 Concepts: Discretionary Access Control (DAC) Traditional Unix Permissions - User, Group, Others (ugo) - Read, Write, Execute (rwx) Discretionary Access Controls (DAC) Access Control Lists (ACLs) - POSIX 1 compliant ACLs standard in Linux filesystems (ext3, ext4, XFS, etc.) - Extends DAC controls to specific user(s) and group(s) 1 Portable Operating System Interface EXchange 22

23 Concepts: Mandatory Access Control (MAC) SELinux has 3 defined policy modes - Targeted (Default), Strict, and MLS. Security Context implemented through extended attributes (xattr) in filesystem and enforced by the Linux Kernel according to SELinux Policy Security Context in SELinux Unix concept of everything is a file (devices, processes, files, directories, etc.) Thus, everything is labeled with a Security Context SELinux policy defines the watertight compartments the SELinux policy control how users, services, files, and binaries interact Policy is generally developed with software vendor when possible. Otherwise, developing policy can be achieved through testing and evaluation giving least privilege to allow completion of a job function 23

24 Concepts: Type Enforcement (TE) Type Enforcement (TE) used by Targeted policy (Default) in SELinux - The Linux Kernel enforces transactions between processes and objects via domain transitions - Further control can be specified using different policy SELinux Domain Transitions Compromised Apache process cannot access /etc/shadow 24

25 Concepts: Polyinstantiation and Multitenancy Polyinstantiation1 is the process used on MLS systems to ensure data being processed by users at separate security levels do so in isolated spaces to use to prevent unauthorized access to data. Data written to these directories will be stored in an independent directory at the security level that they were written, particularly important for shared temporary directories (/tmp, /var/tmp, /dev/shm/) User will not see the redirection to a secure folder, SELinux handles the transition transparently. See the Private Tmp feature in RHEL 72 Multitenancy extends the concept of polyinstantiation with cgroups and Linux Containers (LXC) to ensure that applications are securely separated from each other through Type Enforcement (TE) and MCS (the c0.c1023 attributes of the security level) Multitenancy in OpenShift 1 IBM Developer Works Article Improve Security with Polyinstantiation

26 Kernel Virtual Machine Security 26

27 KVM Security: svirt (SELinux for KVM) Each VM has their own container via SELinux Type Enforcement (TE) and Multi-Category Security (MCS) which uses random compartments to keep the VMs separate 27

28 KVM Security: svirt (SELinux for KVM) (continued) Compromised VM containment with KVM and svirt (SELinux Labels) VS. Compromised VM uses hypervisor exploit to compromise other VMs 28

29 KVM Security: Multiple Firewall Levels 29

30 KVM Security: Multiple Firewall Levels (continued) The ebtables firewall that enables basic ethernet frame filtering on a Linux network bridge, logging, MAC (network address) NAT, and brouting. The firewalls (iptables and ebtables) will be used to complement each other. # ebtables A FORWARD i vnet+ -among-src! 54:52:00:5b:1a:cd= j DROP Prevent IP-MAC Address Spoofing from VMs # ebtables A OUTPUT i vnet+ p IPv6 j DROP Drop All Outbound IPv6 Packets What can ebtables do? Ethernet protocol filtering MAC address filtering Simple IP header filtering ARP header filtering 802.1Q VLAN filtering In/Out interface filtering (logical and physical device). MAC address NAT Logging Frame counters Ability to add, delete and insert rules; flush chains; zero counters Brouter facility Ability to atomically load a complete table, containing the rules you made, into the kernel Support for user defined chains Support for marking frames and matching marked frames 30

31 KVM Security: Control Groups cgroups (control groups) is a Linux kernel feature to limit, account and isolate resource usage (CPU, memory, disk I/O, etc.) of process groups. Resource limiting: groups can be set to not exceed a set memory limit this also includes file system cache. Prioritization: some groups may get a larger share of CPU or disk I/O throughput. Accounting: to measure how much resources certain systems use (e.g. billing purposes) Control: freezing groups or checkpointing and restarting. Dynamic Changes in Workload and Priority (e.g. Number Crunching Overnight, Web Servers during Work Hours) 31

32 KVM Security: Secured Development Litterbox! 32

33 Questions? 33

34 More Information DISA STIG Kickstart DVD: Hardening Scripts: SCAP Security Guide: VDSM Hooks for RHEVM: Classification Banner: 34