CERTIFICATE-BASED SINGLE SIGN-ON FOR EMC MY DOCUMENTUM FOR MICROSOFT OUTLOOK USING CA SITEMINDER

Transcription

1 White Paper CERTIFICATE-BASED SINGLE SIGN-ON FOR EMC MY DOCUMENTUM FOR MICROSOFT OUTLOOK USING CA SITEMINDER Abstract This white paper explains the process of integrating CA SiteMinder with My Documentum for Microsoft Outlook to authenticate users using certificate-based authentication. December 2011

2 Copyright 2011 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Part Number H8858 2

3 Table of Contents Executive Summary... 5 Audience...5 Certificate-based Authentication and My Documentum for Microsoft Outlook... 5 CA SiteMinder Overview...5 SiteMinder components...6 Authenticating using certificates...6 My Documentum for Microsoft Outlook and CA SiteMinder...7 Creating and installing the certificate... 8 Creating certificates using OpenSSL...8 To create a certificate authority:...8 To create certificate for server machine and sign with CA:...9 To create an X.509 certificate for a user or client and sign it with a private key:...9 Installing certificates in the Windows Key Store...10 Configuring CA SiteMinder for My Documentum for Microsoft Outlook Policy Server and Directory Server...12 To create the Agent:...12 To create the agent configuration object:...13 To create Host Conf objects:...14 To configure user directory properties:...15 To create the authentication scheme for certificate-based authentication:...16 To configure a domain:...17 To create a rule for the realm:...19 To create a Response:...20 To create a Policy:...22 Specifying the Policy Server Certificate mapping...23 Web agent and web server...25 Configuring the Apache web server...25 To configure the web agent:...26 Configurations in Documentum Content Server and My Documentum for Microsoft Outlook server...30 Configuring Content Server...30 To enable SiteMinder SSO on Content Server:...30 Configuring the My Documentum for Microsoft Outlook server...31 Testing the setup...31 Troubleshooting Error in apache proxy...32 Error when using req command in OpenSSL...32 Conclusion

4 References

5 Executive Summary This white paper explains how a desktop-based application such as My Documentum for Microsoft Outlook can work in a certificate-based mode of authentication provided by CA SiteMinder. This paper covers the configurations that must be performed on the SiteMinder Policy Server, Web Agent, Web Server and the My Documentum for Microsoft Outlook server to enable authentication using a sample set of self-signed certificates created using OpenSSL. This paper also covers troubleshooting setup and other known issues. Audience This paper is intended for those who are responsible for integrating My Documentum for Microsoft Outlook with CA SiteMinder for certificate-based mode of authentication. Certificate-based Authentication and My Documentum for Microsoft Outlook CA SiteMinder Overview SiteMinder provides centralized security management an enterprise needs to authenticate users and control access to web applications and portals. This ensures protection of high-value applications using stronger authentication methods, while lower-value applications may be protected using simple user name and password approaches. CA SiteMinder provides access management support to many authentication systems including passwords, tokens, X.509 certificates, smartcards, custom forms, biometrics and combinations of authentication methods. Figure 1 illustrates the major components of SiteMinder. 5

6 Figure 1. SiteMinder components SiteMinder components Policy Server: The Policy Server acts as a decision point and validates user credentials against stored access control policies. It then communicates status information with the Web Agent. Web Agent: The Web Agent intercepts the request to access a resource and checks the Policy Server to determine whether the resource is protected. If the resource is protected, the Web Agent communicates with the Policy Server to authenticate and authorize the user. It caches information about authenticated users to allow quicker access. Policy Store: The Policy Store stores all policy related objects including resources that SiteMinder protects, users or groups that cannot access those resources, and actions to take when users are granted or denied access. User Store: The User Store represents an existing user directory for an organization. It contains user and group information, passwords, and attributes. The Policy Server uses the User Store to authenticate users. Authenticating using certificates Certificate-based authentication is one of the authentication schemes supported by SiteMinder. This scheme of authentication considers the X.509 client certificate as a proof of the user s identity. The X.509 client certificate is unique for each user and contains the following information: Name or Distinguished Name (DN) Public key Name of Certificate Authority (CA) who issued the certificate The X.509 server certificate is installed on the web server where secure sockets layer (SSL) is enabled. The certificates must be issued by a valid and trusted Certificate 6

7 Authority and must not yet have expired. The public key of the issuing CA must validate the issuer s digital signature, and the user s public key must validate the user s digital signature. The first step in this authentication scheme is to establish an SSL connection with the web server or proxy on which the web agent is installed. When the SSL connection is successfully established, the details in the certificate are sent to the Policy Server for verification against the information in the user store. SiteMinder uses certificate mapping to determine how to compare a user's certificate with the information stored in the user directory. Certificate mapping defines how data in the certificate is mapped to form a user Distinguished Name (DN). The Policy Server uses this user DN to authenticate the user. If certificates are stored in an LDAP directory, a certificate mapping can direct the Policy Server to verify that the certificate provided by the user matches the certificate associated with the user DN in the LDAP directory. My Documentum for Microsoft Outlook and CA SiteMinder My Documentum for Microsoft Outlook supports only certificate-based authentication in SiteMinder. It uses the client certificates installed in the Windows Keystore to encrypt the request. The server can decrypt the request and verify it s validity using the CA public key installed on it. The certificate details are passed to the Policy Server that verifies user credentials in LDAP. After successful authentication, an SMSESSION cookie is created and the request is passed to the My Documentum for Microsoft Outlook server, for processing. The My Documentum for Microsoft Outlook application server passes the cookie to Content Server. Content Server verifies the user credentials with the Policy Server using the cookie value and authenticates the user to the repository. The certificate user name and the repository username must be the same for the authentication to be successful on Content Server. 7

8 Figure 2. My Documentum for Microsoft Outlook and SiteMinder interaction Creating and installing the certificate OpenSSL is an open source implementation of SSL and TLS. The sample certificates created with this tool will be used further for the integration of My Documentum for Microsoft Outlook with SiteMinder. Creating certificates using OpenSSL The X.509 certificate contains the public key and binds it with the holder s identity. To create a certificate authority: 1. Create an RSA private key as follows: > openssl genrsa -des3 -out private/ca.key 1024 The genrsa command generates an RSA private key. -des3 : This option encrypts the private key with Triple DES cipher. -out : The output file name : gives the size of the private key to be generated. The user is prompted to specify a passphrase or password. The ca.key is placed in the private folder. 2. Create an X.509 certificate and sign using a private key as follows: > openssl req -new -x509 -key private/ca.key -out public/ca.crt -days

9 The req command primarily creates and processes certificate requests in PKCS#10 format. -new : This option generates a new certificate request. -key : This specifies the file to read the private key from. -out : This specifies the output filename to write to or standard output by default. The user is prompted to enter details such as country name and organization. The Common Name or CN and the identify of the user must be unique. The ca.crt CA certificate is created. To create certificate for server machine and sign with CA: 1. Create an RSA private key for server as follows: > openssl genrsa -des3 -out private/server.key Create the Certificate Signing Request, > openssl req -new -key private/server.key -out server.csr 3. Sign the certificate with the CA s private key, > openssl x509 -req -days 360 -in server.csr -CA public/ca.crt - CAkey private/ca.key - CAcreateserial -out public/server.crt When the x509 utility is used to sign certificates and requests, the utility behaves like a mini Certifying Authority. -req: Requires a certificate request as input. -days: Denotes the number of days for which the certificate is valid. -in: Specifies the input filename from which a request is read. A request is read only if the creation options (-new and -newkey) are not specified. -CA: Specifies the CA certificate to use for signing. The CA signs this input file using this option. Its issuer name is set to the subject name of the CA, and it is digitally signed using the private key of the CA. -CAkey: Sets the CA private key with which a certificate is signed. -CAcreateserial: Creates the CA serial number file if it does not exist. -out: Specifies the output filename to write The Common Name for the sever certificate must be a Fully Qualified Domain Name of the server machine. To create an X.509 certificate for a user or client and sign it with a private key: 1. Create a client private key and generate a request as follows: > openssl req -new -newkey rsa:1024 -nodes -out client/client.req - keyout client/client.key 2. Create an X.509 certificate and sign it using CA as follows: 9

10 > openssl x509 -CA public/ca.crt -CAkey private/ca.key -CAserial public/ca.srl -req -in client/client.req -out client/client.pem -days 100 The output is a.pem file that is converted to the pkcs12 format. 3. Convert the.pem file to the pkcs12 format as follows: > openssl pkcs12 -export -clcerts -in client/client.pem -inkey client/client.key -out client/client.p12 -name <your_certificate_name> The pkcs12 command creates and parses PKCS#12 files (sometimes referred to as PFX files). -export: Specifies that a PKCS#12 file is created and not parsed. -in: Specifies the filename from which the certificates and private keys are read. Specifies the standard input, by default. -inkey: Specifies the file from which the private key is read. -out: Specifies the filename of the file in to which certificates and private keys are written. -name: Specifies the ``friendly name'' of the certificate and private key. This name is typically displayed in list boxes by the software that imports the file. The client.p12 is the client certificate in the pkcs12 format. It stores the private key and public key of the client. Figure 3. client.p12 Client Certificate Structure Installing certificates in the Windows Key Store The client certificate in the pkcs12 format (client.p12) must be installed on the client machine (see Figure 2) for My Documentum for Microsoft Outlook to successfully authenticate to the server. You can install the certification in one of the following ways: 10

11 Using Internet Explorer a. Double-click the relevant.p12 file. Windows opens the Certificate Import wizard. b. Click Next. You are prompted to provide the private key password required to import the certificate. c. Retain all other default selections and click Finish. The client certificate is imported into the Personal folder in the Windows Keystore. d. Open the Certificates dialog box in Internet Explorer by selecting Tools > Options and clicking the Content tab and clicking Certificates. The certificate is listed in the Personal certificates tab. Figure 4. Certificates dialog box Using Microsoft Management Console a. Select Start > Run, and type mmc to open the Microsoft Management Console. The console window appears. b. Select File -> Add/Remove Snap-In and click Add. The the Add Standalone Snap-in dialog box appears. c. Click Add. The Certificates snap-in dialog box appears. d. Select My User Account. e. Click Finish. 11

12 f. Click OK to close the dialog boxes. The client certificate you imported is listed in the Personal folder. Configuring CA SiteMinder for My Documentum for Microsoft Outlook Install the following software to configure SiteMinder to work with My Documentum for Microsoft Outlook: SiteMinder Policy Server (this example uses smps win32.zip) SiteMinder web agent (smwa-6qmr4-cr008-win32.zip) ServletExec_ISAPI_50013.exe Sun LDAP directory server (ds[1].5.2.p4.windows.full.zip) Apache server (apache_ win32-x86-openssl-0.9.7m.msi) After installing the software and performing all initial configurations, verify whether all components start without errors. Policy Server and Directory Server Perform the tasks listed in this section, to configure the Policy Server. To create the Agent: 1. Select Start > Programs > SiteMinder > Policy Server User Interface. 2. Click Administer Policy Server. Log in to the Policy Server using the SiteMinder and password credentials. 3. In the left pane, right-click the Agent node, and select Create Agent. The SiteMinder Agent dialog box appears. 12

13 Figure 5. SiteMinder Agent dialog box 4. Enter the name of the agent and select the Support 4.x agents check box. 5. In the IP Address or Host Name field, enter the IP address of the host where you want to install the web agent. 6. In the Secret field enter a password. This password must be the same as the web agent password. To create the agent configuration object: 1. In the Agent Conf Objects node, right-click the ApacheDefaultSettings agent and select Duplicate configuration object. The SiteMinder Agent Configuration Object dialog box appears. 2. Enter a valid name. 3. Double-click #DefaultAgentName to edit the value. The Edit Parameter dialog box appears. 4. In the Parameter Name field, remove the # character from the parameter name to uncomment it. 5. In the Value field, enter the name of the new agent. 13

14 Figure 6. Agent Configuration Object dialog box To create Host Conf objects: 1. Select the Host Conf Objects node in the System tab in the left pane. Right-click DefaultHostSettings, and select Duplicate configuration object. The Host Configuration Object dialog box appears. 2. Enter a valid name. 3. In the Configuration Values list, double-click the #PolicyServer value. The Edit Parameter dialog box appears. 4. In the Parameter Name field, remove the #, <, and > characters from the parameter name to uncomment it, and enter the IP address of the Policy Server in the Value field. 14

15 Figure 7. Host Configuration Object dialog box To configure user directory properties: 1. Right-click the User Directory node in the System tab and select Create User Directory. 2. Enter a valid name. 3. In the NameSpace list, select LDAP. 4. In the Server field, enter the LDAP server IP address. 5. In the Root field, enter the domain controllers separated by commas. 6. In the Start field, enter uid= and in the End field, enter the rest of the DN lookup used in the LDAP for the user preceded by a comma. Ensure that the Example field contains a valid DN separated by commas, and maps to the DN in the LDAP. 7. Ensure there are no white spaces in the Root or the End fields because white spaces result in errors when the Policy Server attempts to map values in the LDAP. 15

16 Figure 8. User Directory dialog box 8. Click View Contents to verify whether the LDAP objects were created successfully. To create the authentication scheme for certificate-based authentication: 1. Right-click the Authentication Schemes and select Create Authentication Scheme. 2. Enter the name and select X509 Client Cert Template as the Authentication Scheme Type. 3. Specify the Fully Qualified Domain Name of the server hosting the Web Agent in the Server Name field. 16

17 Figure 9. Authentication Scheme dialog box To configure a domain: 1. Right-click the Domains node in the System tab and select Create Domain. 2. In the Name field, enter the name of the domain (Example: dco.com) 3. In the User Directories tab select the new user directory, and click Add to include the directory to the User Directories list. 17

18 Figure 10. Domain dialog box 4. In the Administrators tab select the SiteMinder administrator in the Create list, and click Add to include the administrator to the Administrators tab. 5. In the Realms tab click Create. The SiteMinder Realm dialog box appears. 6. In the Name field, enter a valid name for the realm. A realm represents a protected resource. 7. In the Agent field, enter the name of the new agent, or click Lookup to select the required agent. 8. In the Resource Filter field, enter /dco to indicate that all requests to the appsever under the url /dco will be protected. 9. In the Authentication Scheme list, select the new certificate-based scheme. 10. The Protected option is selected as the Default Resource Protection. 18

19 Figure 11. Realm dialog box 11. Click OK. The new realm is saved and listed in the Realms node in the left pane. 12. Click OK to close the Domain dialog box. To create a rule for the realm: 1. Click the Domains tab, and expand the Domains node. 2. In the Realms node, select the new realm. 3. Right-click the new realm and select Create Rule under Realm. The SiteMinder Rule dialog box appears. 4. Specify a name for the rule. 5. In the Resource text box enter /* so all resources under /dco are protected. This indicates that all URLs from My Documentum for Microsoft Outlook to the application server must be authenticated. 6. In the Action section select Get and Post. 7. Select Allow access and the Enabled checkbox. 19

20 Figure 12. Rule dialog box To create a Response: 1. In the Domains tab expand the domain. 2. Right-click Responses and select Create Response. 3. Specify a valid response Name. 4. Select Web Agent in the Agent Type list. 5. Click Create. 6. Specify WebAgent-Http-Header-Variable in the Attribute field. 7. Specify SM_USER in the Variable Name field. 8. Click the Advanced tab and copy the following content to the Script field: SM_USER=<%userattr="uid"%> 9. Click OK. The script populates the SM_USER attribute in the HTTP response to include the uid of the user. 20

21 Figure 13. Response Attribute Editor dialog box Add attribute fields Figure 14. Response Attribute Editor dialog box Add a script for a response attribute 21

22 To create a Policy: 1. In the Domains tab expand the new domain. 2. Right-click the Policies node and select Create Policies. The SiteMinder Policy Dialog box appears. 3. Enter a valid name. 4. In the Users tab click Add/Remove. The User/Groups dialog box appears. 5. Select the required items in the Available Members to Current Members list. Figure 15. Users/Groups dialog box 6. In the Rules tab click Add/Remove to add the rule and realm created for My Documentum for Microsoft Outlook. 7. After adding the My Documentum for Microsoft Outlook rule, select the row to activate the Set Response button. 8. Click Set Response and select the new response. 22

23 Figure 16. Policy dialog box Select rule, realm, and response Specifying the Policy Server Certificate mapping Certificate mapping is an important aspect of setting up certificate-based authentication. The attribute from the client certificate is mapped to the LDAP user linking the client certificate to a user. 1. In the SiteMinder Admin Console open Certificate Mappings from the Advanced menu. 2. Double-click the Current Mappings item to edit it. 3. Add the Issuer DN details of the Certificate to the Issuer DN field. The format of is as follows: E=SM_Admin@emc.com, CN=SM_Admin, OU=IIG, O=EMC, L=BANGALORE, ST=Karnataka, C=IN Obtain these details from the Details tab of the certificate. 23

24 Figure 17 Certificate dialog box 4. In the Mapping section, select a unique attribute that is available in the Subject DN of the client certificate whose value maps to the username available in the LDAP server. The following example illustrates how the Common Name(CN) of the certificate is mapped to the username in LDAP Store. 24

25 Figure 18 Certificate Mapping dialog box Web agent and web server This section provides steps to configure an Apache web server, and the Web Agent installed on the Apache web server. Configuring the Apache web server After installing the Apache web server, configure it with a 2-way Secure Socket Layer (SSL) by modifying the httpd.conf file available in the <Apache2 home>\conf directory. Open the httpd.conf file and uncomment the following lines so that the modules are loaded: LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_connect_module modules/mod_proxy_connect.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_ftp_module modules/mod_proxy_ftp.so LoadModule ssl_module modules/mod_ssl.so 25

26 Add the Listener port to enable SSL: Listen 443 Turn the SSL Engine by adding the following lines to the httpd.conf file: SSLEngine on SSLCertificateFile <path to server.crt file including the file name> SSLCertificateKeyFile <path to server.key file including name> SSLOptions +StdEnvVars +CompatEnvVars SSLVerifyClient optional SSLVerifyDepth 10 SSLCACertificateFile <path to CA.crt file including name> Note: You can add these lines separately in the <Apache2 home>\conf\ssl.conf file. Add the following lines to configure reverse proxy for the My Documentum for Microsoft Outlook application server: ProxyRequests Off ProxyPass /dco address>:<port>/dco To configure the web agent: 1. Run the web agent installer or if it is already installed, select Start > SiteMinder > Web Agent Configuration Wizard. The Host Registration dialog box appears. 2. Select Yes, I would like to do Host Registration now.. 3. Click Next. 4. Specify the SiteMinder admin user name and password details. 5. Click Next. 26

27 Figure 19. Enter SiteMinder administrator user name and password 6. In the Trusted Host Name field specify the fully qualified name of the machine on which the web agent is installed. 7. In the Host Configuration Object field, specify the name of the host configuration object created in the Policy Server. This field is case-sensitive. Ensure that the name matches the Host Configuration Object created in Policy Server as this field is case-sensitive. 8. Click Next. 27

28 Figure 20. Specify Host Name and Host Configuration Object 9. Enter the IP address of the Policy Server machine and click Add. 10. Click Next. 11. Change the Host configuration file location or retain the default values. 12. Click Next. A list of web servers is displayed. 13. Select the Apache server that must be configured with the web agent. 14. Click Next. 28

29 Figure 21. Specify Agent Configuration Object 15. Enter the name of the Agent Configuration Object created in the Policy Server. 16. Click Next. The SSL authentication dialog box appears. Figure 22. Select the authentication scheme 29

30 17. Select the X509 Client Certificate or Form configuration. My Documentum for Microsoft Outlook does not work with Form-based authentication. 18. Click Next. The Self Registration dialog box appears. 19. Select No, I don t want to configure Self Registration. 20. Click Next. After the web agent is installed, verify the http.conf file to ensure it contains the following entries: LoadModule sm_module "<web agent home under program files>/ bin /mod_sm20.dll" SmInitFile "<apache home>/conf/webagent.conf" Ensure that the WebAgent.conf file is created in the <apache home> directory. Open the WebAgent.conf file and add the following line: PreservePostData="NO" Add this line to disable the Preserve Post Data feature that shows an alternate page on the browser to hold the credential data when redirected. Since My Documentum for Microsoft Outlook does not have a browser interface, set the value of the PreservePostData property to No. Enable the web agent in the WebAgent.conf file by modifying the EnableWebAgent property as follows: EnableWebAgent="YES" The client works in a 2-way SSL mode with the web server if the web agent is disabled. Configurations in Documentum Content Server and My Documentum for Microsoft Outlook server Configuring Content Server Documentum Content Server is installed with SiteMinder plug-ins. To enable SiteMinder SSO on Content Server: 1. Copy the following dlls from $DM_HOME/install/external_apps/auth_plugins/Netegrity to $DOCUMENTUM/dba/auth: Windows dm_netegrity_auth.dll smagentapi.dll smerrlog.dll Solaris/AIX/LINUX dm_netegrity_auth.so 30

31 For HPUX libsmagentapi.so libsmerrlog.so libsmcommonutil.so dm_netegrity_auth.sl libsmagentapi.sl libsmerrlog.sl 2. Edit $DOCUMENTUM/dba/auth/dm_netegrity_auth.ini to include the following information: agent_name = <name of Agent Object created in Policy Server> shared_secret = <password of the Agent in Policy Server> policy_server_ip = <IP of Policy Server> resource=/dco Content Server uses this information to verify whether the Policy Server has authenticated the user. 3. Restart the repository to ensure the changes are applied. Verify the repository log file to check if errors occurred while the plug-in was loaded. Configuring the My Documentum for Microsoft Outlook server Modify the dfs-sso-config.properties file available in emc-dfs-rt.jar in the WEB_INF/lib folder with SSO type information. SiteMinder properties are as follows: sso.type = dm_netegrity user.header.name = SM_USER password.cookie.name = SMSESSION After performing the required modifications, repackage the updated emc-dfs-rt.jar file in the My Documentum for Microsoft Outlook EAR file. Testing the setup After performing the setup, you are recommended to verify client authentication using a browser before accessing it through the MS Outlook client as follows: Verify whether the SSL mode works without enabling the web agent on the proxy or web server using a browser to access the My Documentum for Microsoft Outlook URL. If this test passes, then the server and client certificates are correct. Otherwise, check if the certificates are valid and whether they are installed or referenced in the appropriate locations on both the client and server. After enabling the web agent in the WebAgent.conf file, using a browser verify whether the Policy Server authenticates the user with the client certificate. If it fails to authenticate the user, check the Policy Server and web agent configurations, especially the Certificate Mapping on the Policy Server. 31

32 Specify the web server URL in the My Documentum for Microsoft Outlook client and try to connect to the URL. If the connection fails, check for errors in the %appdata%/ssocomponent/sso.log file in the My Documentum for Microsoft Outlook client. After the initial test passes in My Documentum for Microsoft Outlook, the repository login dialog box appears. The user name and password fields are disabled. Verify whether the username is correct and click OK. If the login fails, enable authentication trace logs on Content Server to obtain additional information. Further, verify whether the dfs-sso-config.properties file in the My Documentum for Microsoft Outlook server contains the correct values. Troubleshooting Error in apache proxy The following error indicates that the passphrase must be removed from the server certificate: [error] Init: SSLPassPhraseDialog builtin is not supported on Win32 (key file <some keyfile name>) Remove the passphrase from the server certificate using the following command: openssl rsa -in server.key -out server.key Where server.key is the private key of the server machine. Error when using req command in OpenSSL The following error occurs during the creation of X.509 certificates: Unable to load config info from /usr/local/ssl/openssl.cnf error in req Run the req command after appending the following content: config < path to openssl.cfg > The openssl.cfg file is available in the OpenSSL/bin folder. Conclusion This white paper provides details about the authentication mechanism to enable certificate-based authentication by a desktop client, such as My Documentum for Microsoft Outlook. The white paper also provides a brief overview of creating certificates using X.509 and the configuration changes necessary for each component to ensure My Documentum for Microsoft Outlook works seamlessly in a certificate-based SiteMinder setup. 32

33 References CA SiteMinder Guide ENU/Bookshelf_Files/HTML/index.htm?toc.htm? html 33