Secure to the Core: The Next Generation Secure Operating System from CyberGuard

Transcription

1 Secure to the Core: The Next Generation Secure Operating System from CyberGuard Paul A. Henry MCP+I, MCSE, CCSA, CCSE, CFSA, CFSO, CISSP, CISM, CISA Senior Vice President CyberGuard Corp A CyberGuard Corporation White Paper September 2004 Copyright 2004 CyberGuard Corporation. All rights reserved.

2 What is a Secure Firewall Operating System? While industry experts may debate which firewall architecture provides the optimum level of security, few would disagree about the critical importance of a secure firewall Operating System. Many vendors claim their network security products are built upon a hardened OS. What you will find in virtually all cases, is that the vendor simply turned off -- or removed -- unnecessary services, and then patched the OS for known vulnerabilities. Clearly, this is not a hardened OS but really a patched OS. A true hardened OS is one in which the vendor has modified the kernel source code to enforce a security perimeter between the operating system, firewall software and network stack. Correctly implemented, this eliminates the risk that a service running on the hardened OS could be exploited by a hacker to obtain root level privilege and then highjack the firewall. CyberGuard s heritage in developing secure real-time operating systems for the US Department of Defense is evident today in our Mandatory Access Control (MAC) and Multi-Level Security (MLS) operating system technologies. Together, MAC/MLS completely insulate the operating system layers used to inspect and transport packets from those that allow the firewall to be configured and managed. A user who has been authenticated and authorized for administrative access can never leave the firewall and connect to the network. Likewise, a user seeking network connectivity can never gain access to the firewall s management functions. Even of the firewall were to be compromised, no further network incursions would be possible. In this White Paper, we review the sophisticated techniques CyberGuard used to implement this same level of security in our next generation firewall product, and the resulting technical and operational benefits. The Evolution of CyberGuard s Next Generation of Security Products CyberGuard s experience building secure operating systems dates back to 1967, when the company -- then known as Datacraft -- began building mission-critical real-time simulation and control systems. In 1987, with 20 years of OS experience under its belt, CyberGuard now a part of the Harris Corporation -- and AT&T Federal Systems began joint development of an Orange Book B1 MLS/OS and LAN solution. It has been estimated that 75 man years of development time was invested in this critical government project. Development concluded in 1991 and the NCSC B1 Evaluation Cycle began. In 1992 the evaluation concluded with the award by the United States Department of Defense for the world s first TCSEC B1 OS and Network LAN certification. No other firewall vendor to date has ever been able to match this achievement. Secure to the Core: CyberGuard s Next Generation Operating Environment Page 2

3 In response to customer demands, CyberGuard decided to migrate to a more mainstream Operating System that would preserve the same levels of security while supporting a broader set of hardware platforms. CyberGuard purchased the source code rights to SCO UnixWare and ported many of the security mechanisms that enabled us to achieve our previous B1 certification. This purpose built Operating System has served our clients well since In fact not a single CERT bulletin has ever been written against our SCO UnixWare based firewall solution. The advent of 64 bit processing architectures and high performance platforms from Intel (among others), creates the foundation for CyberGuard to offer a new set of highly scalable purpose-built security solutions. This was our primary motivation in developing CG Linux ; a next generation secure Operating System based on the Linux kernel. CyberGuard s CG-Linux Operating System takes full advantage of the security controls that helped us earn our original TC SEC B1 certification, while incorporating the additional security features available when u tilizing a Linux-based kernel. Collectively, these capabilities allo w us to offer the highest level of Operating System security available in a commercial firewall product. The table below summarizes the security features built into the respective UnixWare and CG-Linux OS. Security Mechanisms Features Linux UnixWare Description Discretionary Access Control (DAC) Yes Yes Limit s a user's access to a file or directory. Based on owner/group IDs and permission bits. Multilevel Security (MLS) Yes Yes Creates a barrier between non-administrative users, processes, and data, and the corresponding set of users, processes, and data of the firewall security systems. Based on a modified Bell-LaPadula security model. Mandatory Access Control (MAC) Yes Yes Enforces mandatory system-wide policies that cannot be changed at the discretion of individual users. Based on a modified Bell-LaPadula security model. Capability (Privileges) Yes Yes Divides the super user privilege into a number of discrete privileges that can be assigned to multiple users or programs. Roles Yes Yes Organizes administrative duties in to roles that can be assigned to multiple administrative users. Used to provide separation of duties. Auth Yes No RSBAC mechanism for restricting the ID to which a program may switch (setuid). File Flags Yes No RSBAC model for providing fine-grained access control over file system objects (files, directories, symbolic links etc.). PAM User Level Authentication Yes No Enables the use of longer passwords and more granular transaction logging. Audit Yes Yes Audits security relevant events at a very granular level, enabling forensic analysis and accountability. Secure to the Core: CyberGuard s Next Generation Operating Environment Page 3

4 Discretionary Access Control Discretionary Access Control (DAC) is an access control service that enforces a security policy based on the identity of system users (or groups of users) and their respective authorizationsto access files and other system resources. There are three categories of users: 1. Owner The owner of the file 2. Group Users in the same group as the owner 3. Other Everyone else There are three kinds of authorizations: 1. Read Users may read the file or list the contents of a directory 2. Write - Users may write to the file or add a new file to the directory 3. Execute Users may execute the file or lookup a specific file DAC is used primarily to limit a user's access to a file or directory. This access is considered to be discretionary because the owner determines at his or her discretion who receives these read, write and execute access rights. Multi Level Security CyberGuard s implementation of Multi Level Security (MLS) is based on a modified version of the Bell-LaPadula security model. MLS provides the security mechanisms and enforcement systems needed to allow data with different degrees of sensitivity to be securely maintained and accessed on the same system. Essentially, MLS provides a barrier between the non-administrative users, processes, and data, and the corresponding set of users, processes, and data of the firewall security systems. A process inherits its sensitivity level from its respective use. Therefore the permissions for the process determine the level of sensitivity of the data that the process is permitted to act upon. MLS enforcement enables an administrative user to run a process that reads or modifies a firewall configuration file, while preventing a non-administrator -- running the exact same process -- from accessing or modifying the firewall configuration data. Mandatory Access Controls CyberGuard s implementation of Mandatory Access Controls (MAC) enhances and complements DAC by enforcing MLS rules within the CG-Linux kernel. MAC enforces mandatorysystem-wide policies that cannot be changed at the discretion of individual users. Most commercial Operating Systems provide support for DAC only. Secure to the Core: CyberGuard s Next Generation Operating Environment Page 4

5 Many in the security community believe that MAC is inherently more secure than DAC, because it eliminates some of the most prevalent incorrect permissions mistakes made by administrators trying to implement DAC in security systems. In addition, a DACbased OS can be exploited by a Trojan Horse program to alter the DAC security settings, thereby allowing an escalation of privileges for a malicious user. Limiting Super User Privileges In a typical UNIX or Linux operating system, the Super User -- otherwise known as the Root User -- has total control over all aspects of the operating system and the tasks and programs it is running. CyberGuard s CG-Linux divides Super User privileges across multiple users and system processes, effectively reducing the Super User s authority. This increase security by reducing the dependence on a single entity that could otherwise assert total control over all security processes. Should a malicious hacker ever achieve Super User status, they would gain very limited control over the firewall and its security-related processing. Role-Based Management CyberGuard has always provided extremely granular control over the separation of administrative duties. This makes it possible to provide selective administrative access while ensuring that no one can gain complete control over the firewall and its security processing. CyberGuard has enhanced and extended this role functionality in its next generation firewall product by incorporating these features into the CG Linux OS. This provides additional security by reducing the possibility that a firewall administrative duty can be circumvented by an operating system administrative function. Auth Module In a typical UNIX environment, a Super User can change the authority level at which a process operates within the OS. This is explicitly prevented by CG-Linux, which offers extensive controls over which privilege changes are permitted and by whom they can be applied. This eliminates the common ploy of the Privilege Escalation Attack, in which a hacker alters the au thority level of a process in order to increase their privileges within the OS and gain control over the firewall. Secure to the Core: CyberGuard s Next Generation Operating Environment Page 5

6 File Flags To further enhance OS security, CG-Linux provides additional granularity in its standard file access controls. The file flags are complementary to the standard Linux file permissions and can only be altered or changed by an authenticated security officer. Flag execute_only search_only read_only write_only secure_delete no_execute Checked for FILE, FIFO, SYMLINK DIR FILE, FIFO, SYMLINK, DIR FILE, FIFO, SYMLINK FILE FILE no_delete_or_rename FILE, FIFO, SYMLINK, DIR append_only add_inherited FILE, FIFO, SYMLINK FILE, FIFO, SYMLINK, DIR By extending the access control capabilities of Linux standard file permission, CG-Linux provides a level of granular contro l far beyond what is available in a commercial OS. Pluggable Authentication Module (PAM) PAM is a UNIX programming interface that enables third-party security methods to be used. By using PAM, multiple authentication technologies, such as RSA, DCE, Kerberos, smart card and S/Key, can be added without changing any of the login services, thereby preserving existing system environments. CyberGuard has incorporated PAM into CG-Linux, affording numerous security enhancements, including: 1. Support for longer passwords 2. Password and account expirations / verifications 3. Improved transaction logging (including information on the user and login address) 4. RSBAC restrictions to provide more granular access control The incorporation of an enhanced version of PAM into CG-Linux affords a great deal of flexibility and expandability for authentication-related servic es in current and future CyberGuard products. Audit and Alert Systems CyberGuard has always offered superior logging capabilities, and stored log files in binary format to preserve data integrity. CG-Linux provides additional security benefits Secure to the Core: CyberGuard s Next Generation Operating Environment Page 6

7 by dramatically improving the performance and flexibility of the firewall Alert and Audit Systems. Because these systems can reach deeper and wider into the CG Linux audit trail, the granularity and amount of data that can be logged is far more extensive and granular than ever before. The binary format dramatically improves search and query performance while the increased granularity and breadth of information allows you to drill down into the log file with increased precision during your queries. The Alert system has also been dramatically enhanced and now includes fully user configurable alerts. This includes OS performance data that is typically absent with firewalls built upon a commercial OS. Operating System Performance The evolutionary move to Linux allows CyberGuard to leverage new OS efficiencies that significantly improve performance and throughput. CG-Linux offers full support for 64 bit processors, for the Intel Itanium processor family, as well as non-intel-based platforms. This provides compelling opportunities to deploy CG-Linux based security solutions on embedded devices. The firewall architecture has changed from a UnixWare Stream model to a the faster and more efficient Socket model implemented in Linux Linux fully supports threads. This significantly improves performance, memory management and overall resource efficiency. Threads provide a useful programming technique for dividing work into separate pieces. Programs that correctly use threads can run on multiprocessor systems, with each thread running on a separate CPU. Any slow process running on a single-cpu system can theoretically execute on an N-way multiprocessor in 1/N of the time. In Conclusion CyberGuard historically has provided the most secure and best performing application proxy based firewalls in the industry. The legacy continues with the evolution of our next generation CG-Linux OS. Providing unparalleled performance and security, CyberGuard is well positioned to remain as the preferred solution for securing the world s most demanding networks. Secure to the Core: CyberGuard s Next Generation Operating Environment Page 7

8 CyberGuard Corporate Headquarters Quadrant Business Center 350 SW 12th Avenue Deerfield Beach, FL Phone: Fax: CyberGuard Europe Limited Asmec Centre, Eagle House The Ring, Bracknell Berkshire, RG12, 1HB United Kingdom Phone: +44 (0) Fax: +44 (0) Copyright 2004 by CyberGuard Corporation. All rights reserved. This publication is intended for use with CyberGuard Corporation products by CyberGuard's personnel, customers and end users of CyberGuard's products. It may not be reproduced in any form without the written permission of CyberGuard Corporation. CyberGuard is a registered trademark of CyberGuard Corporation. UnixWare is a registered trademark of Santa Cruz Operations, Inc. All other trademarks are the property of their respective owners.