Open Text Secure Shell

Open Text Secure Shell Securing Open Text Exceed Open Text Connectivity Solutions Group Abstract Security concerns continue to receive unprecedented focus from organizations around the globe, and the cost of security breaches has proven to be financially crippling for many organizations. The need to secure sensitive data is paramount. Industry-specific security standards and government regulations are also prompting organizations to review and improve their current security infrastructures and policies. The search for a comprehensive security solution from a trust-worthy vendor is on, and Open Text Secure Shell is that solution. Open Text Secure Shell is FIPS 140-2 validated and can be used to ensure that all your X Window sessions are encrypted and secure.

2 Securing Open Text Exceed W hite Paper Contents The Security Challenge... 3 Driving Security... 4 Structural Factors... 4 External Factors... 4 Connectivity A Definition... 5 X Window (or X11)... 5 Telnet... 5 FTP... 5 Security Risks in a Connectivity World... 6 Weak Authentication... 6 Easy Protocol Decoding... 6 Data Authenticity and Integrity Tampering... 6 Solutions for Secured Connectivity... 7 SSL... 7 Kerberos... 7 Secure Shell... 8 How to Secure Exceed... 9 Host Based Security... 9 User Based Security... 9 Open Text Secure Shell... 9 Implementation... 11 Securing X Connections... 11 X11 Forwarding... 11 X11 Forwarding In Depth... 12 XDMCP... 12 Securing Telnet... 13 SFTP File Transfer... 13 Secure Shell Management Console... 14 Certificates and Keys... 14

3 Securing Open Text Exceed W hite Paper The Security Challenge Security is a hot topic today. Although, companies have been slow to recognize the importance of security, things have changed during the last decade. Security is a top priority and there are no indications that this will end any time soon. The costs of security (or lack thereof) have now been clearly identified, and the picture does not look very good. The CSI (Computer Security Institute), in partnership with the FBI (Federal Bureau of Investigation), releases a Computer Crime and Security Survey every year. This survey is one of the longest-running surveys in the information security field and definitely one of its most relevant. The survey document is available from the CSI Website (www.gocsi.com). In 2009, Respondents reported big jumps in incidence of password sniffing, financial fraud, and malware infection. One-third of respondents' organizations were fraudulently represented as the sender of a phishing message. Average losses due to security incidents are down again this year (from $289,000 per respondent to $234,244 per respondent), though they are still above 2006 figures. Twenty-five percent of respondents felt that over 60 percent of their financial losses were due to non-malicious actions by insiders.

4 Securing Open Text Exceed W hite Paper Driving Security Beyond the potential for significant financial damages, many other factors are urging companies to investigate, purchase and deploy security technologies. Structural Factors Inability to run operations without IT infrastructures. IT framework downtime impacts revenue and profit. Integrity of information is essential to accomplish daily operations and require mutual trust. Theft of proprietary information can mean life or death situations for companies. Businesses are becoming more and more interconnected. How many transactions per day are performed through electronic data exchange? Exposure to the outside world. External Factors Our world has become more security conscious. Amid heightened concerns for national security, every individual has fully realized that safety has been redefined. Massive IT attacks (Code Red, Nimda, Sobig ) have had a worldwide impact and prompted media attention. Because we live in an interconnected world, the security of IT infrastructure is not an expert reserved domain any longer. Proliferation of standards and legislations with direct or indirect impact on Security: since 2001, numerous initiatives have been taken by the government that directly impact security policies of public and private organizations (e.g. The Patriot Act, Sarbanes-Oxley and HIPAA). Threat of legal liability by customers and/or partners. Organizations and software vendors are being held to a higher degree of accountability for security, if not in the courtroom, then by their customers. Organizations are challenged to prove they are managing security at a level that will satisfy their business partners and stakeholders. This goes beyond discussing what security products are installed, to communicating compliance and management practices of information security.

5 Securing Open Text Exceed W hite Paper Connectivity A Definition Connectivity is a domain where network communications are paramount. In its broader sense, Connectivity can be defined as the group of technologies that allow multiple systems (heterogeneous or not) to communicate. In a more popular sense, Connectivity designates TCP/IP client server technologies working with standardized protocols which allow systems to interconnect and exchange information. Some of the most popular Connectivity technologies is: X Window (or X11) X Window (or X11) is a windowing and graphic system developed at MIT. Almost all UNIX graphic applications are X Window based. One of X Window most notable properties is its ability to separate the application processing layer, the logic, from its graphic layer, the user interface, which can be deployed on a remote machine. Telnet Telnet is a protocol for remote computing on a network. It allows a computer to act as a remote terminal on another machine, anywhere on the network. The remote computer (also called the telnet server) accepts input directly from the client computer\and output for the client session is directed to the client screen. Many other protocols such as TN3270 (Mainframe) or TN5250 (AS/400) are derived from Telnet FTP File Transfer Protocol, is a protocol for exchanging files over a network. FTP is most commonly used to download a file from a server, or to upload a file to a server using a network.

6 Securing Open Text Exceed W hite Paper Security Risks in a Connectivity World Although some connectivity protocols have been in use for more than a quarter century, very few of them come with a strong security model. Inherent security flaws exist in almost every connectivity protocol and many organizations do not realize how vulnerable they are to these security issues. Weak Authentication As surprising as it may seem, many protocols, such as Telnet and FTP, send their messages in clear text over the network. Such messages include username and passwords, as well as all other information displayed to the user during the session. Widely available network sniffing tools allow any attacker to easily capture that information and use it for their own profit Easy Protocol Decoding Although X Window does not transmit user s input as a string of text to the X application, the protocol remains relatively easy to decode in order to retrieve the desired information. Keyboard inputs are transmitted in clear-text as numbers which can be easily obtained and interpreted to rebuild the original text that was entered by the user. Access to password protected X applications can be compromised by anybody with a network sniffing tool and a little bit of patience Data Authenticity and Integrity Tampering The Man in the Middle attack, or TCP hijacking, is well known attack where an attacker sniffs packets from the network modifies them and inserts them back into the network. Sensitive information can be intercepted and altered without a chance for the victim to know that their data has been tampered with. Although this attack requires a superior level of skills than those required for simply sniffing a network, some program/code sources are available on the internet for the enterprising hijacker. Because of their widespread usage throughout organizations, connectivity software is a target of choice. Their popularity does not allow companies to simply remove and replace them with more secure technologies without significant investment. The solution to this problem lies in applying robust security techniques to existing connectivity protocols.

7 Securing Open Text Exceed W hite Paper Solutions for Secured Connectivity Given that connectivity protocols are still going to be around for a significant number of years, it becomes mandatory for organizations to implement complementary security solutions that can be built on top of their existing infrastructure. Some of those solutions consist of building encryption and authentication within the network hardware. Although it sounds like an interesting way of increasing the security level of the network, such a solution is very expensive and involves significant disruption of the business operations. Another solution consists in building a security layer within the existing protocols. High effectiveness, minimal business disruption and relatively low investments are some of the characteristics that have led many organizations to choose this path. SSL SSL (Secure Sockets Layer) is a protocol which allows for the encryption of data transmitted between two computers. It was developed in the mid-90s by Netscape to facilitate the transmission of sensitive information via the Internet. Since then, it has been built into all major browsers and web servers and is the industry standard for protecting information sent over the Internet. SSL uses public and private keys. There are two strengths of SSL, 40-bit and 128-bit. The bits indicate the length of the session key the longer the session key, the harder it is to crack the code. When a client wants to connect to a server using SSL, the client and the server go through a series of requests and acknowledgements ( the SSL handshake ). SSL is used to secure a wide variety of protocols and has been adopted by a large number of organizations. As an example, SSL has become the de facto security standard for Mainframe and AS/400 connectivity through the TN3270 and TN5250 protocols. It s also widely used to secure HTTP connections and many 3rd party protocols. Kerberos Kerberos is a trusted third-party authentication mechanism. It is trusted in the sense that each of its clients believes Kerberos judgment as to the identity of each of its other clients to be accurate. Kerberos keeps a database of clients and their private keys. The private key is a large number known only to Kerberos and the client to which it belongs. In the case that the client is a user, it is an encrypted password. Network services requiring authentication register with Kerberos, as do clients wishing to use those services. Microsoft introduced operating system level support for Kerberos in Windows 2000. Although interoperability between a classic Kerberos environment and its Microsoft counterpart was somehow challenging at the beginning, the two environments can now work seamlessly together. Many companies are considering using Kerberos as their primary authentication mechanism, now that it can be used from the Windows environment. A Microsoft Windows server can easily become a

8 Securing Open Text Exceed W hite Paper Kerberos domain controller and thus serve as an authentication trusted tier for all third party authentication needs. Another factor that speaks to Kerberos is its ability to be used as part of a much wider security implementation. The Secure Shell protocol for instance, which provides authentication and encryption services, can use Kerberos as one of its authentication methods. Secure Shell The Secure Shell protocol was created in 1995 by a young Finnish student named Tatu Ylönen after he was victim of a password-sniffing attack. In 1999, OpenBSD shipped with OpenSSH, a derivative of the original free SSH 1.2.12 which also supports the Secure Shell 2 protocol. The Secure Shell protocol offers numerous answers to security issues among which: It offers strong security against cryptanalysis and protocol attacks It provides support for key and certificate management infrastructures It can work in conjunction with existing certificate infrastructure if available It is relatively easy to deploy and can be made easy-to-use It does not require in-depth security knowledge from the user and can work transparently behind-the-covers.

9 Securing Open Text Exceed W hite Paper How to Secure Exceed In the past, the main concerns regarding security revolved around the ability to display to the Exceed X server. There were 2 ways to control access: Host Based and User based. Host Based Security Either of these methods can be used with Exceed and are configured under Xconfig Security, Access Control and System Administration. With Host Access Control, you decide which UNIX hosts clients will be able to display to the Exceed X server. If a host is not in this list it will get a message that it is unable to display to Exceed. User Based Security With User based control, access is based upon a security token known as a magic cookie. In this case, the token is kept in the users UNIX home directory and is also copied to the PC running Exceed. The two are then compared and if they are not identical, display to Exceed is rejected. These security mechanisms do not take into account that any credentials and data are still sent over the network in clear text. Although password complexity rules have increased, passwords are often easily cracked. A study by The University of Wyoming says that it will take over 800 years to crack a 9 character password made up of upper and lower case letters. However anyone snooping on the network with a packet trace utility can still see it in the clear. These are some of the reasons that SSH was developed. Open Text Secure Shell Open Text Secure Shell is an add-on to a wide selection of Open Text Connectivity Solutions, including Open Text Exceed, Exceed PowerSuite, Open Text HostExplorer, HostExplorer FTP and Open Text NFS Client, as well as any third-party network applications. It can seamlessly encrypt network traffic generated by applications in compliance with industry regulations, such as FIPS 140-2. As a tightly integrated add-on component for Open Text Connectivity Solutions, it offers great transparency to end users, affecting neither their workflow nor productivity. Open Text Secure Shell provides Secure Shell 2 (SSH), Secure Sockets Layer (SSL) & TLS, LIPKEY, and Kerberos security mechanisms to ensure security for communication types such as X11, NFS, terminal emulation (Telnet), FTP and any TCP/IP protocol.

10 Securing Open Text Exceed W hite Paper Open Text Secure Shell offers a very flexible infrastructure allowing the protocol to evolve as new authentication methods are invented. Current supported authentication methods include: Username/Password Certificate and Keys Keyboard Interactive Open Text or MIT Kerberos Microsoft Kerberos (Active Directory) Multiple Authentications Regardless of what you choose you can relax knowing that all your data and credentials will be secure from intrusion. Additional features include: Extensive ability to configure the protocol settings. Choice among several strong encryption algorithms including AES. Choice among several key exchange algorithms. FIPS 140-2 validated cryptographic module. Choice of several Message Authentication Code (MAC) algorithms to ensure data integrity. Support for Agent forwarding for remote authentication. Ability to automatically or manually select the X11 port settings. Configurable tunnel parameters such as window size, connection timeout, compression values, and trace level.

11 Securing Open Text Exceed W hite Paper Implementation Open Text Secure Shell is a Secure Shell 2 client application. With it, you configure and start Secure Shell sessions that you can use to secure otherwise unsecured communications. A Secure Shell session, or tunnel, is an authenticated and encrypted SSH2 connection initiated from a Secure Shell client to a host on the network that is running a Secure Shell server. During the initial negotiation of this connection, both the server and client are authenticated. Server authentication is performed initially using public key exchange, and a number of authentication methods are supported for client authentication. Once the tunnel is established, the information channeled through it is encrypted using any of a number of supported encryption ciphers including Blowfish, 3DES, CAST128, and the U.S. Advanced Encryption Standard (AES). The following scenarios describe different implementations of Secure Shell. Securing X Connections Securing X connections are more complex than with other protocols or services. However it is important to understand how it works so that you know where to look if troubleshooting is required. Securing an X Window session requires that you create an Xstart profile that specifies Secure Shell as the startup method. The Connection parameters you specify in Xstart are used to establish a Secure Shell tunnel to the target host. With Xstart you can encrypt your login credentials or your entire session and be sure that all your data is safe from unwanted intrusions. X11 Forwarding You can secure X Window sessions initiated with HostExplorer or Xstart. When using Xstart, X11 forwarding is enabled by default, whereas with HostExplorer you need to ensure X11 is enabled in your profile. Enabling X11 forwarding within HostExplorer instructs the secure shell server to forward X applications, started from a HostExplorer session, from the host to your local machine through the secure shell tunnel created by the HostExplorer session. When you start a tunnel where X11 port forwarding is enabled, the Open Text Secure Shell engine requests X forwarding when connecting to the Secure Shell server. If the server supports X forwarding, it runs an X11 proxy on the remote host. The $DISPLAY variable is configured by the remote Secure Shell server so that initiated X applications connect to the X11 proxy, which sends the connection through the tunnel. What is displayed on your desktop is a secure session.

12 Securing Open Text Exceed W hite Paper X11 Forwarding In Depth The secure shell client sends out a request for a secure shell connection. The secure shell daemon gets the request and then spawns another sub-daemon to take care of this new secure connection. Unlike a regular telnet connection, where the user sets the DISPLAY on the command line, the ssh daemon takes care of this and sets the display. This is part of how X11 forwarding works as previously explained. In addition to setting the display it also creates an authentication token known as a magic cookie by running a program called Xauth. From a firewall perspective you only have to open outbound port 22. When the connection is requested with X11 forwarding, the ssh daemon spawns a subdaemon to handle the display. Upon the first connection, the host identifies itself with a key or fingerprint. In a UNIX environment, this is kept in the known_hosts file. With Open Text Secure Shell, it is kept in a repository. The client and the server exchange a list of supported algorithms and then agree on one. An authentication token is generated by Xauth and stored in the user s.xauthority file. The X client is displayed to this proxy X server and then everything is sent securely to Exceed for displaying on the local workstation. Since ssh is a client/server environment, both sides must have X11 forwarding enabled. Since the main purpose of Xstart is to launch X clients, this is implicitly enabled. However it is important to ensure that the ssh daemon has it enabled as well or the connection will fail. XDMCP Most UNIX users are familiar with the CDE, Gnome or KDE desktop environments; however, it is not possible to secure XDM connections. Secure Shell only secures TCP/IP connections and XDMCP uses UDP/IP. A simple way around this is to launch Xsession, dtsession, gnome-session or startkde from the Xstart command line. These are the different script files that ultimately invoke the respective desktop environments. This will provide the UNIX desktop with the added benefit of knowing that your session and data are encrypted and protected.

13 Securing Open Text Exceed W hite Paper Securing Telnet When you install Open Text Secure Shell alongside HostExplorer, you gain the option of creating Secure Shell-enabled HostExplorer VT session profiles. You can then use these session profiles to start secure VT sessions, which use the Secure Shell protocol instead of using unsecured Telnet connections. Securing a HostExplorer VT session requires that you create a VT session profile in HostExplorer that specifies one of the Secure Shell connection types. Each of the Secure Shell options also specifies an authentication type. You must provide the connection and authentication parameters necessary to establish a Secure Shell connection to the target host. For more control over tunnel settings, you also have the option of specifying a custom Open Text Secure Shell tunnel profile. The settings are then read from the tunnel profile to the HostExplorer session profile. Open Text Secure Shell tunnel profiles contain all the parameters necessary to start and configure a Secure Shell tunnel. When you create your HostExplorer profile, you can configure it to load the necessary connection parameters from the tunnel profile of your choice. You can create tunnel profiles with Open Text Secure Shell, and then select the tunnel to be used for your VT session, or you can create them when you configure your HostExplorer VT session profile. SFTP File Transfer SFTP or SSH File Transfer Protocol is a network protocol that provides rich file transfer and manipulation capabilities over an SSH2 connection. The Open Text Secure Shell implementation of the SSH2 protocol includes SFTP support for HostExplorer FTP and FTP Classic, and an scp2 command line tool. After SFTP connections are established, the files you send and retrieve move securely between the client and the host s Secure Shell server. When you configure an FTP profile to use Secure Shell, all parameters used to establish the Secure Shell tunnel are stored in the FTP profile. Instead of specifying the connection parameters explicitly when you create a Secure Shell-enabled FTP session profile, you also have the option of specifying these parameters by associating a tunnel profile with your FTP profile. Tunnel profiles contain all the connection parameters needed to establish the Secure Shell tunnel to the remote host. You can also do a combination of both. That is, in the FTP profile, you can specify a tunnel profile from which to load the necessary parameters. Then, in the same FTP profile, you can specify and save parameters that override some of those loaded from the specified tunnel profile.

14 Securing Open Text Exceed W hite Paper Secure Shell Management Console The Open Text Secure Shell Management Console is used to view, configure and establish Secure Shell tunnel connections. The Console is what provides the ability to fine-tune your secure shell settings. This is where encryption algorithms can be selected, different authentication methods can be configured and various secure tunnels to your various hosts can be created. The console assists in managing, monitoring, and troubleshooting your connections. Certificates and Keys You can use Open Text Secure Shell Management Console or the Certificates and Keys Management Console to manage keys and certificates that reside in the key stores. The console lets you manage keys and certificates for use with Secure Shell and SSL connections initiated with the various Connectivity Solutions. You can perform the following tasks; view information about your keys about your certificates, view certificate authorities, generate user keys, generate certificate requests, generate self-signed certificates, import and export keys and certificates.

15 Securing Open Text Exceed W hite Paper About Open Text Connectivity Solutions Group Open Text's leading Connectivity Solutions connect people, data and applications in mission-critical environments through a complete line of remote application access and data integration solutions. With 90 percent of Global 2000 companies relying on its award-winning solutions for over 20 years, Open Text understands the financial and operational challenges that most organizations face, whether they are multiple systems, disparate data sources, or geographically dispersed teams. About Open Text Open Text is a leader in Enterprise Content Management (ECM). With two decades of experience helping organizations overcome the challenges associated with managing and gaining the true value of their business content, Open Text stands unmatched in the market. Together with our customers and partners, we are truly The Content Experts, supporting 46,000 organizations and millions of users in 114 countries around the globe. We know how organizations work. We have a keen understanding of how content flows throughout an enterprise, and of the business challenges that organizations face today. It is this knowledge that gives us our unique ability to develop the richest array of tailored content management applications and solutions in the industry. Our unique and collaborative approach helps us provide guidance so that our customers can effectively address business challenges and leverage content to drive growth, mitigate risk, increase brand equity, automate processes, manage compliance, and generate competitive advantage. Organizations can trust the management of their vital business content to Open Text, The Content Experts. connectivity.opentext.com Sales: Support: [connsales@opentext.com] [+1-905-762-6400 Worldwide] [1-877-359-4866 North America] [connsupport@opentext.com] [+1-905-762-6400 Worldwide] [1-800-486-0095 North America] www. o p e n text.com For more information about Open Text products and services, visit www.opentext.com. Open Text is a publicly traded company on both NASDAQ (OTEX) and the TSX (OTC). Copyright 2009 by Open Text Corporation. Open Text and The Content Experts are trademarks or registered trademarks of Open Text Corporation. This list is not exhaustive. All other trademarks or registered trademarks are the property of their respective owners. All rights reserved.