CERTIFICATE BASED SSO FOR MYDOCUMENTUM OUTLOOK WITH IBM TAM WEBSEAL

Transcription

1 CERTIFICATE BASED SSO FOR MYDOCUMENTUM OUTLOOK WITH IBM TAM WEBSEAL ABSTRACT This white paper provides information on configuring My Documentum client for outlook for web SEAL client side certificate authentication and systematic approach on solving various problems encountered in Web SEAL certificate authentication mechanism.

2 To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller, visit or explore and compare products in the EMC Store Copyright 2015 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware and <insert other VMware marks in alphabetical order; remove sentence if no VMware marks needed. Remove highlight and brackets> are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number - H14316

3 Table of Contents Executive summary... 5 Audience... 5 Terminology... 5 Introduction to TAM and WebSEAL... 6 Tivoli Access Manager Components... 6 Tivoli Access Manager Runtime... 6 Tivoli access manager Java runtime... 6 Tivoli Access Manager Web portal manager... 7 Overview of web SEAL... 8 Web SEAL client side certificate authentication... 8 Web SEAL junctions... 9 Web SEAL certificate authentication integration with My Documentum client for outlook... 9 Configuring MDO application for webseal certificate Authentication Certificate Creation for testing Creating self-signed certificates using IBM ikeyman utility Creating self signed certificate for web SEAL server Creating self signed certificate for MDO Client Creating self-signed certificate for App-Server Exporting certificates Exporting webseal server certificates Exporting client certificates Exporting Appserver and web SEAL server certificates to JKS file Configuration at client Installing client certificates to windows keystore Turn off default certificate policy of.net on using self signed certificates Configuration in MDMO server Configuring JBOSS/TomCat application server for SSL Enable JXWS WebSEAL handler... 25

4 Configuring DFC client for DFC privileged authentication Configuration at WebSEAL Server Enable client certificate authentication at web SEAL server Create user name in LDAP registry Creating Junction to the backend application server Creating TCP junction Creating 2-way SSL or mutually authenticated junction Creating 1-way SSL or mutually authenticated junction Testing the setup Troubleshooting Miscellaneous Conclusion References... 33

5 Executive summary This white paper explains how to configure desktop applications like My Documentum client for outlook for Web SEAL client side certificate authentication to leverage single sign on capabilities. This document also briefs about authentication flow at various components of the Tivoli access manager (TAM) and Documentum stack. Audience This white paper is intended for customers, partners and consultants who may want to configure My Documentum client for outlook for WebSEAL client side certificate authentication. Terminology TAM (Tivoli Access manager for e-business) provides robust, policy-based security to a corporate Web environment. TAMeb provides authentication of users, control of access privileges, auditing, single sign-on, high availability, and logging. DMZ ( Demilitarized Zone) is a physical or logical sub network that contains and exposes an organization's external services to a larger untrusted network. CMS Certificate management system is X.509 certificate storage database used by WebSEAL JKS Java key stores provide a convenient mechanism for storing and deploying X.509 certificates and private keys for JAVA application servers

6 Introduction to TAM and WebSEAL Tivoli Access Manager Components Tivoli Access Manager is an authentication and authorization solution for corporate Web, client/server, and existing applications. Tivoli Access Manager allows you to control user access to protected information and resources. By providing a centralized, flexible, and scalable access control solution, Tivoli Access Manager allows you to build secure and easy-to manage network-based applications and e-business infrastructure. Tivoli Access Manager supports authentication, authorization, data security, and resource management capabilities. Although many components make up Tivoli Access Manager for e-business here are the 5 you need to get started using simple single sign-on: Policy server Policy server maintains the master authorization database for the management domain as well as the policy databases associated with other secure domains that you might decide to create. This server is key to the processing of access control, authentication, and authorization requests. It also updates authorization database replicas and maintains location information about other Tivoli Access Manager servers. User Registry User Registry is LDAP registry stores user information. WebSEAL server is IBM Tivoli Access Manager s securing policy management solution. WebSEAL is a reverse proxy server that sits in the DMZ between the client and the resource begin requested. The WebSEAL server brokers the exchange of information between the client and the back-end resource. The Tivoli Access Manager Policy server stores your organization security policy. The security policies are stored in the master authorization database. WebSEAL keeps a local copy of this database. Tivoli Access Manager Runtime contains runtime libraries and supporting files that applications can use to access Tivoli Access Manager Servers. Once you have installed the runtime you will be able to manage your Tivoli Access Manager environment from any machine where the runtime is installed. Tivoli access manager Java runtime is optional it is required if you intend to use the Web portal manager which is the graphical user interface for TAM administration.

7 Tivoli Access Manager Web portal manager is a Web-based graphical user interface (GUI) used for Tivoli Access Manager Administration. The GUI counterpart to the pdadmin command line interface, Web Portal Manager provides management of users, groups, roles, permissions, policies, and other Tivoli Access Manager tasks. A key advantage of using Web Portal Manager is that you can perform these tasks remotely, without requiring any special network configuration. Figure 1: Tivoli access manager components and authentication flow 1. The user makes a request for a Tivoli Access Manager protected resource. The policy enforcer (in this case WebSEAL) intercepts that request and collects the appropriate information from the user to verify his or her identity. 2. Once the user ID and password is verified against the user registry a second call is made to the user registry to create the user credential. 3. Once the user credential is created it is returned to the policy enforcer for authorization. 4. Authorization is performed using the authorization database replica that resides on the policy enforcer.

8 Overview of web SEAL IBM Tivoli Access Manager WebSEAL is the resource manager responsible for managing and protecting Web-based information and resources. WebSEAL is a high performance, multithreaded Web server that applies fine-grained security policy to resources in the Tivoli Access Manager protected Web object space. WebSEAL can provide single signon solutions and incorporate back-end Web application server resources into its security policy. WebSEAL is a reverse proxy server that sits in the DMZ between the client and the resource begin requested. The WebSEAL server brokers the exchange of information between the client and the back-end resource Following are the common authentication mechanism provided by WebSEAL a. Forms Authentication User is prompted for login name and password b. Basic Authentication User is prompted for login name and password c. Client side certificate authentication Client certificate is used for user authentication Web SEAL client side certificate authentication Client-side certificate authentication enables a user to use a client-side digital certificate to request an authenticated identity for use within a Tivoli Access Manager secure domain. When authentication is successful, WebSEAL obtains a Tivoli Access Manager Identity that is used to build a credential for the user. The credential specifies the permissions and authorities to be granted to the user. WebSEAL supports client-side certificate authentication in three different modes. a. Required certificate authentication mode WebSEAL always requires a client certificate for every client request made. If the client does not present a certificate, the SSL connection with the client is closed and client-side certificate authentication is not attempted. b. Optional certificate authentication mode WebSEAL prompts for client certificate for every request made. If client presents the certificate, web seal uses it for creating 2-way SSL connection otherwise it establishes SSL connection with just the PD-S-SESSION-ID submitted with the request. c. Delayed certificate authentication mode

9 It prompts for client certificate only when client tries to access the protected resource. Web SEAL junctions A WebSEAL junction is an HTTP or HTTPS connection between a front-end WebSEAL server and a back-end Web application server. Junctions logically combine the Web space of the back-end server with the Web space of the WebSEAL server, resulting in a unified view of the entire Web object space. A junction allows WebSEAL to provide protective services on behalf of the back-end server. WebSEAL performs authentication and authorization checks on all requests for resources before passing those requests across a junction to the back-end server. You can create WebSEAL junctions with the pdadmin command-line utility or with the Web Portal Manager. Web SEAL certificate authentication integration with My Documentum client for outlook MDO uses optional certificate authentication to leverage SSO capabilities. There are two authentication flows when using web SEAL. First, Client authenticates itself to WebSEAL proxy using client certificate and gets back the valid session cookie. Second, content server authenticates user against Documentum repository using DFC privileged authentication mechanism. DFC privileged authentication establishes trusted login between DFS and Documentum content server. Following diagram depicts authentication flow in Documentum stack components:

10 Figure 2: Certificate Authentication flow in My Documentum outlook 1. Client access the protected resource using WebSEAL junction. 2. WebSEAL challenges the user for client certificate. 3. Reads the certificate from windows keystore. 4. Client sends the certificate to the WebSEAL server. 5. WebSEAL checks client certificate against the certification revocation list. 6. WebSEAL matches the certificate subject with the users LDAP DN. Authentication is successful if they match each other. 7. WebSEAL makes calls to the policy server using authorization API which checks user is authrorized to access the resource. 8. If the authorization is successful it passes the request to the MDO server. 9. MDO passes the login info to the content server which authenticates user to access repository resources. 10. On successful authentication CS returns the valid session ID. 11. DCO server responds to the WebSEAL with the JSESSION ID. 12. WebSEAL returns the WebSEAL session cookie PD-S-SESSION-ID and the JSESSIONID to the client. Later these session Ids are used by the client to make further requests.

11 Configuring MDO application for webseal certificate Authentication Certificate Creation for testing When developing a production application, you might want to wait until testing is complete before purchasing a true digital certificate. With ikeyman, you can create a self-signed digital certificate to use for testing purposes. A self-signed digital certificate is a temporary digital certificate issued to you, with yourself as the CA. Creating self-signed certificates using IBM ikeyman utility. The ikeyman utility is a tool you can use to manage your digital certificates. With ikeyman, you can: 1. Create a new key database 2. Create a test digital certificate, 3. Add CA roots to your database, 4. Copy certificates from one database to another, 5. Request and receive a digital certificate from a CA, 6. Set default keys, 7. Change passwords. The ikeyman utility is automatically installed with the SSL Toolkit. Creating self signed certificate for web SEAL server 1. Double click on the IBM GSKit ikeyman application located at C:\Program Files\ibm\gsk7\bin 2. Select Personal Certificates from the pull down list.

12 3. Open key database file 4. Click on Key Database File->Open 5. Select key database type as CMS. Browse through the pdsrv.pdb file and then click OK. During installation sample key database pdsrv.kdb is created for webseal. It is located at C:\Program Files\Tivoli\PDWeb\wwwdefault\certs\. You can either use the same database file for new self-signed certificates or you can create new one.

13 6. All KDB files are password protected. Enter password to when it is prompted for. Password for pdsrv.kdb file is pdsrv. Click OK. It opens the certificates already present in the database file.

14 7. Select Create->New Self -Signed Certificate. Which opens new certificate dialog 8. Enter Key Label as server-cert Common Name will be WebSEAL server machine name Organization, Where Organization is the suffix provided while creating LDAP entry. Enter Validity Period of the certificate. Click OK.

15 9. It will prompt you to set the key as default key in database. Select YES. Creating self signed certificate for MDO Client 1. Select Create->New Self -Signed Certificate. Which opens new certificate dialog 2. Enter Key Label. Client user name as Common Name. Organization, Organization unit etc. Where Organization is the suffix provided while creating LDAP entry Enter Validity Period of the certificate. Click OK.

16 3. It will prompt you to set the key as default key in database. Select No. Creating self-signed certificate for App-Server 1. Select Create->New Self -Signed Certificate. Which opens new certificate dialog 2. Enter Key Label. App server machine name as Common Name. Organization, Organization unit etc. Where Organization is the suffix provided while creating LDAP entry Enter Validity Period of the certificate. Click OK.

17 3. It will prompt you to set the key as default key in database. Select No. Exporting certificates Once the certificates are created they must be exported to install it on client machine and app server machine. Exporting webseal server certificates 1. Select the server certificate to be exported. Click on Extract Certificate button on right pane. On the Extract Certificate to a file, select Data type to Binary DER data WebSEAL certificate. Select the location to export. This creates Server-Cert.der in c:\testcerts.

18 Exporting client certificates 2. Select client certificate to be exported. Click on Export/Import button on right pane. On the Export/Import key dialog, select Export Key, Key file type as PKCS12 for client certificate. Select the location to export. This creates Client-Cert.der in c:\testcerts Exporting Appserver and web SEAL server certificates to JKS file 3. Create new key database file to hold Server certificates. Select Key Database File -> New from the menu Select Key database type as JKS. For Example CA-Certs.jks Enter the name and location to be created. Click Ok.

19 4. Provide password to protect file. 5. Create one more JKS file to hold Application server certificate. Select Key Database File -> New from the menu Select Key database type as JKS. For Example Appserver.jks Enter the name and location to be created. Click Ok. 6. Provide password to protect file.

20 7. Open key database file Click on Key Database File->Open Select key database type as CMS. Browse through the pdsrv.pdb file and then click OK. During installation sample key database pdsrv.kdb is created for webseal. It is located at C:\Program Files\Tivoli\PDWeb\wwwdefault\certs\. 8. Export both WebSEAL server and Application server certificates to CA-Certs.jks and AppServer.jks. Refer following screenshots for same.

21

22 Configuration at client Installing client certificates to windows keystore Using Microsoft management console 1. Open Microsoft Management Console from Run by typing mmc. The console window opens. 2. Go to File -> Add/Remove Snap-In and click Add button. 3. In the Add Standalone Snap-in dialog box, select Certificates and click Add. 4. In the Certificates snap-in dialog box, select My User Account and click Finish. 5. Right-click the personal Certificate folder > All Tasks > Import 6. Certificate Import wizard appears. Click Next 7. Select the location of the certificate to be imported to personal folder and Click Next.

23 8. Enter the password for private key

24 9. Select the certificate store as Personal 10. Click Finish on completing certificate wizard. 11. To install CA certificates Right-click the Trusted Root Certification Authorities > All Tasks > Import. 12. Select location of server-cert.der. 13. Select Trusted CertificationRoot Authorities as the certificate store. 14. Click Finish on Import completion. Turn off default certificate policy of.net on using self signed certificates Set "InvokeCustomCertficatePolicy" to true to override.net framework default certificate policy in outlook.exe.config. Location on 32bit client machine Outlook.config.exe is located at \\Program Files\Microsoft Office\<office folder*> Location on 64bit client machine Outlook.config.exe is located at \\Program Files(x86)\Microsoft Office\<office folder*>

25 * OFFICE11 for Outlook 2003 * OFFICE12 for Outlook 2007 * OFFICE14 for Outlook 2010 * OFFICE15 for Outlook 2013 Configuration in MDMO server Configuring JBOSS/TomCat application server for SSL 1. Copy the JKS key store file to MDMO server. 2. Modify server.xml to enable SSL port and to add the Application server and WebSeal server certificate. For JBOSS server.xml file is located at: - C:\jboss\server\default\deploy\jboss-web.deployer For Tomcat server.xml file is located at: - C:\apache-tomcat\conf <Connector port="8443" protocol="http/1.1" SSLEnabled="true" maxthreads="150" scheme="https" secure="true" keystorefile="c:\webseal_certs\appserver.jks" keystorepass="password" truststorefile="c:\webseal_certs\ca-certs.jks" truststorepass="password" clientauth="true" sslprotocol="tls" /> Enable JXWS WebSEAL handler Modify authorized-service-handler.xml file of EAR\WAR file to add DFCprivilegedHandler. Include the following handler to the authorized-service-handler-chain.xml <handler> <handler-name>webseal Privileged DFC authentication</handler-name> <handler-class> com.invisiblecrm.smartfolder.dfs.utils.privilegeddfcjaxwshandler </handler-class> </handler> Configuring DFC client for DFC privileged authentication Using Documentum Administrator client to enable the trusted login and trusted server privileges for MDMO server DFS client 1. Login to DA as Administrator and click on Privilege Clients node

26 2. Click on File > Manage Clients 3. Search for the DFC for your DFC instance (Eg. dfc_websealapp) and using the arrow key, click on OK 4. On the added DFC, Right click using context menu, select Properties. 5. Check Enable Trusted Login and Trusted Login privileges. 6. Click OK.

27 Configuration at WebSEAL Server Enable client certificate authentication at web SEAL server Modify the following properties of webseal server configuration file WebSEALddefault.conf located at C:\Program Files\Tivoli\PDWeb\etc 1. Turn off basic authentication and forms authentication if they are enabled. Under [forms] stanza set forms-auth = none Under [ba] stanza set ba-auth = none 2. Set the following properties to turn on the certificate authenticate mechanism. [authentication-mechanisms] cert-ssl = C:/Program Files/Tivoli/PDWebRTE/bin/sslauthn.dll & -cfgfile [C:/Program Files/Tivoli/PDWeb/etc/WebSEALd-default.conf] [Certificate] accept-client-certs = optional [SSL] WebSEAL-cert-keyfile = C:/Program Files/Tivoli/PDWeb/wwwdefault/

28 certs/pdsrv.kdb WebSEAL-cert-keyfile-stash = C:/Program Files/Tivoli/PDWeb/wwwdefault/ certs/pdsrv.sth WebSEAL-cert-keyfile-label = Server-Cert Enter the webseal server certificate name as mentioned during certificate creation 3. Restart webseal server Create user name in LDAP registry Tivoli Access Manager provides graphical user interface Web Portal Manager and command line utility pdadmin to create, list, and delete junctions. The pdadmin is installed as part of the Tivoli Access Manager Runtime package. 1. Login to pdadmin using following command >pdadmin a sec_master p password Where sec_master is the Administrator and password is password for sec_master account. 2. Create user testuser pdadmin>user create [ gsouser] [ no-password-policy] <user_name> dn cn sn password -gsouser option enables global sign-on capabilities. -no-password-policy option allows the administrator to create the user with an initial password that is not checked by the existing global password policies. dn(distiniguished name) Specifies the registry identifier assigned to the user being created. cn (common name) Specifies the common name assigned to the user being created sn (sir name) Specifies the family name of the user being created. For example: pdadin>user create gsouser no-password-policy testuser cn=testuser, o=dco, c=in testuser tu password 3. Enable specified user account. A user cannot log in with a disabled account. pdadmin>user modify user_name account-valid {yes no} For example:

29 pdadmin>user modify testuser account-valid yes Creating Junction to the backend application server 1. Login to pdadmin using following command >pdadmin a sec_master p password Add the location of pdadmin utility to system path if it unable to find it. (C:\Program Files\Tivoli\PolicyDirector\bin in our case) 2. Note down the webseal server instance using following command >pdadmin> server list Creating TCP junction Issue following command to create TCP junction pdadmin> server task instance_name-webseald-host-name create -t tcp -s -j -e utf8_uri -c iv_user -h host_name [-p port] jct_point For example: pdadmin>server task default-webseald-webseal create -t tcp -s -j -e utf8_uri -c iv_user -p h /junctiontcpdco Creating 2-way SSL or mutually authenticated junction Issue following command to create SSL junction pdadmin>server task instance_name-webseald-host-name create f t ssl D dn K servercert-label -s -j -e utf8_uri -c iv_user_l [-p port] jct_point For Example: pdadmin>server task default-webseald-webseal create -f -t ssl D "CN=CONFIG595VM2.dco.com,O=DCO,C=IN" -K "Server-Cert" -s -j -e utf8_uri -c iv_user_l -p h /ssljunc Creating 1-way SSL or mutually authenticated junction Issue following command to create SSL junction

30 pdadmin>server task instance_name-webseald-host-name create f t ssl D dn K servercert-label -s -j -e utf8_uri -c iv_user_l [-p port] jct_point For Example: pdadmin>server task default-webseald-webseal create -f -t ssl D "CN=CONFIG595VM2.dco.com,O=DCO,C=IN" -K "Server-Cert" -s -j e utf8_uri -c iv_user_l -p h /ssljunc default-webseald-webseal = Will be the default WebSeal server instance dn = Fully qualified Application server machine name -h = IP address of Application server Testing the setup Once the configuration is done the setup and be tested by accessing MDMO server resource using web SEAL junction. 1. On client machine, Open Internet explorer and provide URL to access webseal protected resource using web SEAL junction. The access mode should be HTTPS when certificate authentication is used For Example: of WebSEAl-server>/<junction>/dco/smartfolder/build.html IP ADDRESS>/ssljunc/dco/smartfolder/build.html 2. Select the right client certificate in the certificate selection dialog. 3. Build.html is downloaded on successful authentication. Troubleshooting Import fails with error message collision message The underlying connection was closed. Could not establish trust relationship for this SSL/TLS secure channel. Check if WebSEAL server URL is reachable via Internet explorer Check if Client and Server certificate are added to local machine certificate store. Remove and re-add the existing client and server certificate to local machine certificate store Change the WebSEAL URL to include hostname of WebSeal machine instead of IP address of WebSeal machine Restart the Outlook and retry the import operation

31 MSG: failed to initialize trust manager Check if privileged DFC instances for DFS server deployment correctly using DA Create a new properties file named local-dfs-runtime.properties under mdmo.war\web-inf\classes\ and add the following information to the file dfc.session.allow_trusted_login = true 403 Forbidden Error messages at MDO client Following are the checklist if you run through forbidden error messages. Configuration errors at WebSEAL server. Make sure WebSEAL is started successfully in service control manager Run through WebSEAL.conf file and check all the attributes are configured for enabling client-side certificate authentication. Check user account is created at user registry and account is enabled. Make sure client certificate is not expired. Missing WebSEAL cookies(pd-s-session-id) in SOAP request header Missing WebSEAL cookies (PD-S-SESSION-ID) in UCF requests Check for client logs, extra logs and sso logs for more information Check for valid name is been obtained when user certificate is selected Check if the headed returned by WebSeal server contain PD_S_SESSION_ID cookie e.g. Set-Cookie: PD-S-SESSION- ID=2_1_pHMhHgvVLaM5VtXtBSz0bRDAPoo7HKh5+Ym-ZP2WTsBYOJAx; Path=/; Secure; HttpOnly Error during repository login. Documentum Authentication Failure. WebSEAL handler is not configured at DFS. Configure the web SEAL handler in the authorized-service-handler.xml file Privileged DFC client instance not approved Login to DA and check DFC client of DFS server is enabled for trusted login Timestamp difference between app server and content server Make sure that DFS server and the content server are having same timestamp 501 Third party server is not responding This means the backend application server is not responding. Check the web SEAL junction is configured for right app server with right port. In case of SSL junction make sure that server.xml file has the entries for both ca certificates and app server certificate JKS files. 501 Internal server errors on content transfer This is seen with TAM 6.1 which doesn t support HTTP 1.1 chunking. This issue is not seen in TAM which supports HTTP chunking.

32 Enable the chunking at the UCF client and server to resolve this error during content transfer. You must configure ucf.server.config.xml located in /WEB-INF/classes to disable HTTP 11 chunking and use UCF alternative chunking, edit the following options: <option name="http11.chunked.transfer.encoding"> <value>disabled</value> </option> <option name="alternative.chunking.buffer.size"> <value>1m</value> </option> Configuring WebSeal Load Balancer to use the cookies MDMO client fill only the following cookies with every subsequent request sent to webseal server. o "PD-S-SESSION-ID" o "PD_ID" o "PD_STATEFUL" This is seen with TAM 6.1 which doesn t support HTTP 1.1 chunking For performance tuning, You can change the default value for UCF alternative chunking buffer size to 2M. Note: The value must be an integer; with units in bytes, kilobytes (K), or megabytes (M). For example 1M or 512K (no space) Miscellaneous Useful commands

33 Conclusion This white paper talks about configuring WebSEAL client-side authentication for My Documentum outlook. It covers detailed configuration at web SEAL server, MDO client and DFS server to leverage SSO capabilities with web SEAL client side certificate authentication. References IBM Tivoli Access Manager for e-business: Installation Guide IBM Tivoli Access Manager for e-business: Administration Guide IBM Tivoli Access Manager for e-business: WebSEAL Administration Guide IBM Tivoli Access Manager for e-business: Troubleshooting Guide doc/welcome.htm