ID205 IBM Lotus inotes High Availability Customer Case Study and Successful Web Deployment Best Practices Rahul A. Garg Advisory Software Engineer IBM Fredrik Söderquist Consultant Infoware Solutions Svenska AB
Agenda Stockholms IBM Läns Landsting Case Study Lotus Domino Server Configurations Reverse Proxy Configuration for Lotus inotes Examples High Availability Configuration Q&A 2
Who am I? Fredrik Söderquist fredrik.soderquist@infoware.se Represents a Swedish Premium Business Partner: Presenting the Case Study: IBM Lotus inotes deployment at Stockholm County Council 3
What is the Stockholm County Council? Stockholm County Council is English for Stockholm Läns Landstings Know in Sweden for its abbreviation SLL SLL's mandate is to ensure that its 2 million residents have access to health care and public transport as well as preventing health problems. SLL is one of Sweden's largest employer in Sweden 45 000 employees 96% work in health care 4
Why inotes? SLL started in 2002 Each organization within SLL had their own mail solution or multiple solutions Even though mail solutions were present, there was a lot of drawbacks: Basically every available product on the market was in use due to the large amount of different solutions About 30% of all SLL employees didn't have access to mail or where using group accounts There where no or very little security in these solutions Availability was delivered on a best effort basis And there were no or very little protection against virus and spam Mail was an area with potential for improvements SLL IT Council Committee initiated a project to establish and deploy a centralized mail solution based on IBM Lotus Domino It was voluntary for the organizations within SLL to join the central mail solution The central solution had to show compelling business values in order to make the organizations move 5
Why inotes? (continued) The new central mail environment showed these business values: Lower TCO Higher availability More performance Added functionality Easier administration Why was inotes chosen in preference of Notes? The organizations within SLL determined that inotes was: Cheaper and easier to roll out Cheaper and easier to maintain Required less user training All in all a lower TCO 6
How did SLL build their inotes solution? Combining high availability, high performance and low costs isn't easy SLL defined two mindsets to help them keep costs down: Automation instead of administration Centralized operations, decentralized administration SLL first focused on the foundation, the back-end architecture 7
Back-end architecture The back-end architecture was placed in two geographically separated sites and placed within the core network of SLLnet SLLnet is SLL's WAN Highly available High performing 8
Back-end architecture (continued) 9
Clustering All server roles that needed to provide its services in a high availability configuration were clustered, mostly in a very common mirrored cluster configuration However for the Mail Servers SLL chose a different clustering method, a method that could be explained as a multiple non-mirrored cluster. It's actually a single Domino cluster with eight server members. But the mailboxes aren't mirrored between the servers, instead they are randomly and asymmetrically distributed between all eight servers. 10
Clustering (continued) Randomly selected home server mail/user1.nsf Randomly selected replica server on the opposite site 11 mail/user1.nsf
Automation & archiving To help keep costs down SLL focused on automation A product called User Management Tool (from Infoware Solutions) together with custom administrative applications created an automated user management process Many additional custom administrative and products were user for additional automation processes and application used for self administration/service desks. Server-based archiving A product called Mailpak Archiving Tool (from Infoware Solutions) was deployed to help keep storage costs down Low performing (cheap) storage was used for the online archive The mail servers got added performance due to less data to manage High performing storage was used for the mail servers 12
SLL's inotes solution Solid back-end sorted out, only thing needed was a front-end SLL wanted transparent failover and load balancing of inotes Many different products were looked at and discarded WebSphere Edge Components were chosen Front-end Products: WebSphere Edge Dispatcher (load balancers) WebSphere Edge Caching Proxy WebSphere Edge Content Based Router WebSphere Application Server (LDAP authentication) Customization: Login application Web service to CLDBDIR.nsf to get user's mail servers for load balancing Proxy plugin for the web service look up 13
SLL's inotes solution (continued) Every HTTP request is load balanced Failover is completely transparent to users Since forms85.nsf is located on all eight mail servers, all mail servers are involved in presenting inotes data. Since users have two mailboxes, both mail servers holding their mailbox is involved in load balancing and failover of the content in the mailbox Proxies do URL-masking, user always sees mail.sll.se Firewall between the front-end and back-end ensures that only the proxies can communicate with mail servers over HTTP SLL is terminated by the proxies which makes the mail servers gain performance Only one SSL certificate is needed which help keep cost down Proxies caches requests for added performance 14
15
Lessons learned & best practices 1 Sort out the back-end first 2 You need a front-end 3 Maintenance work during office hours is possible with a front-end 4 With a front-end you only have one place for SSO integration 5 Load balancing utilizes HW better, make sure you don't overscale 6 Notes and inotes users can co-exist 7 inotes demands less user training than Notes (depending on your users skills) 8 Automation is a primary key to cut TCO 16
What's next for SLL? 8.5.2 upgrade Virtualization SLL is currently in the middle of an upgrade to 8.5.2 that is planned to be completed by Q1 2011 With this upgrade they are looking into implementing DAOS to further reduce the storage costs Even more cost reducing actions are made with further virtualization of their infrastructure Sametime 8.5 Sametime is being deployed to provide online meetings, chat and awareness. And of course Sametime will be integrated with inotes 17
Agenda Stockholms IBM Läns Landsting Case Study Lotus Domino Server Configurations Reverse Proxy Configuration for Lotus inotes Examples High Availability Configuration Q&A 18
IBM Lotus Domino Server Configuration Requirements Replication Enable Multi-Server Single Sign On Eliminates the need for users to re-authenticate when accessing multiple servers Create Web SSO configuration Add an LTPA (Light-Weight Third Party Authentication) Token Set HTTP Authentication Method Lotus inotes Redirector Cluster replication is required for load balancing Requires Domino Web Configuration (domcfg.nsf IBM Lotus Sametime Integration 19
IBM Lotus Domino Server Replication Configuration Enable Cluster Replication Select Domino servers to be added to the cluster Create a new cluster 20
IBM Lotus Domino Server SSO Configuration Create Web SSO Configuration All Server Docs View Click on Web and create Web SSO Configuration 21
IBM Lotus Domino Server SSO Configuration Create Web SSO Configuration (continued) Set Config name DNS Domain name For dual-directory configurations Domain Servers 22
IBM Lotus Domino Server SSO Configuration Create Web SSO Configuration (continued) create a SSO Key 23
IBM Lotus Domino Server SSO Configuration Set HTTP Authentication Method Internet Protocols --> Domino Web Engine Add Web SSO Configuration Enable Domino Servlet Manager for Sametime Integration 24
IBM Lotus inotes Redirect Overview Domino application based on the IWAREDIR.NTF template Allows Lotus inotes users to access their mail file and mail server using only the name of the inotes Redirect server Uses Domino authentication methods to redirect a user's browser to their mail file based on their username and password Create Lotus inotes Redirect application using IWAREDIR.NTF template Configure Redirect application This can be done through the Notes client. 25
IBM Lotus inotes Redirect Server Settings Server Settings How to look up mail file and mail server Fixed Force redirection to the specified url, eg user comes from domino.acme.com and setting is set to mail.acme.com, the URL will become mail.acme.com Dynamic The url is built from the incoming URL request. eg. mail1.acme.com MailServer Rewrites the URL to be the home mail server. 26
IBM Lotus inotes Redirect Server Settings (continued) Reverse Proxy field can be used to add name of reverse proxy server that will be used as junction name in the redirection URL NOTE: This does NOT provide reverse proxy functionality; you still need a reverse proxy server 27
IBM Lotus inotes Redirect Server Settings (continued) In general, when working with reverse proxies, use the Fixed mode Leave the server name field blank 28
IBM Lotus inotes Redirect Server Settings (continued) To assist load balancers working in a clustered environment, a form called ServersLookup (new to 8.5.2) has been made available and will reside in the Redirect database When requested by the load balancer The ServersLookup form will return one of two HTTP response headers in the format X-Domino-xxxxx, each containing a comma separated list of servers. X-Domino-ReplicaServers is returned when the service finds the relevant path within its own cluster X-Domino-ClusterServers is returned only when the mail servers are part of a different cluster. 29
IBM Lotus inotes Redirect Server Settings (continued) Configure SSL options SSL used only on authentication SSL used for whole session 30
IBM Lotus inotes Redirect Server Settings (continued) Coming in 8.5.3 Omit http protocol from redirect URL Default is No, which means the redirect URL will include the http(s) protocol This is useful in scenarios where external users will be using https as provided by a reverse proxy and internal users will be using http against the back-end Domino server 31
IBM Lotus inotes Redirect Server Settings (continued) Coming in 8.5.3 Use home mail server for Domino Directory lookups Default is No, which means lookups will be done against current server hosting the Redirect server This is useful when supporting multiple Domino domains, each with its own Directory, and using one Redirect server. Previously, lookups would only be done against current server which could result in improperly constructed redirect URLs. Allowing lookups against the user's home mail server will ensure the redirect URL is properly built. Note that this will need the Redirect server to be added to the Trusted Servers field on the remote server 32
IBM Lotus inotes Redirect UI Settings UI Setup Personal Options allow users to specify which inotes area they want to start in, eg. Mail, Calendar, etc Login Options allow users to choose between the different inotes modes Enable the shared or public computer mode that disables attachments Username Cookie: Adds a Remember Me option to stores user name. 33
IBM Lotus inotes Redirect Ultra-light Settings Ultra-light / Mobile Settings Enable Radio Button, creates a Ultra-Light mode button when using Firefox Mobile Devices, detects the device and creates a login page formatted for mobile screens that are listed. 34
IBM Lotus inotes Redirect Application Settings ACL Configuration Sets up Access Control for the the redirector so that users can use it 35
IBM Lotus inotes Redirect Implementation Add the inotes Redirector to the HTTP Configuration in the Server Doc Internet Protocols -->HTTP inotes Redirector DB name 36
IBM Lotus inotes Redirect Implementation (continued) DWALoginForm needs Anonymous connections enabled Otherwise, Login page will have a broken image link If Anonymous connections are not allowed Use the NOTES.INI setting, HTTPPublicUrls Available starting in 8.5.2 Allows you to add public urls for images e.g. HTTPPublicUrls=/iwaredir.nsf/* 37
IBM Lotus inotes Redirect Login page Using the DWALoginForm Create and open Domino Web Server Configuration application (domcfg.nsf) Click Add Mapping Change the Target Database to your Lotus inotes Redirect application Change the Target Form to DWALoginForm Save and Close Default Login page will look like this 38
IBM Lotus Sametime Integration with reverse proxies Requirements when deploying with a Reverse Proxy server Edit stlinks.js (located on both Sametime and Domino inotes servers) var ll_rproxyname= https://reverseproxyhost.acme.com ; var ll_affinityid= st8 ; Note: The stlinks directory which contains stlinks.js may get overwritten during a Domino server update. It's recommended to backup stlinks.js and hostinfo.js, which may have been customized. Edit stconfig.nsf --> Meeting Services (located on the Sametime server) Reverse Proxy Enabled: True Reverse Proxy Alias: st8 Reverse Proxy example settings Proxy /st8/communitycbr/* http://sts.acme.com:8082/communitycbr/* Proxy /st8/* http://sts.acme.com/* ReversePass http://sts.acme.com/* https://reverseproxyhost.acme.com/* Deploying in a high-availability configuration Sametime clusters for failover Sametime stand-alone mux servers if expected volume is high 39
IBM Lotus Sametime Proxy 8.5 configuration Requirements for WebSphere Caching Proxy Server Enable PUT and DELETE method directives as required by the Sametime Proxy server Add rewrite and mapping rules for inotes and Sametime with junction points in order for requests to be sent to the proper server Lastly, in order to keep access secure, it's recommended to configure SSL between the client browser and the reverse proxy server. 40
IBM Lotus Sametime Proxy 8.5 configuration (continued) Domino Server Configuration The Sametime Proxy server used by inotes users is defined on the Domino server by setting a notes.ini var. There are two possible server settings, inotes_wa_sametimeproxyserver and inotes_wa_sametimeproxyserverssl. We recommend using both settings as follows: For intranet/internal users, set inotes_wa_sametimeproxyserver to the URL of the actual Sametime Proxy server. inotes_wa_sametimeproxyserver=http://stproxyserver.company.com:9080 In the case where intranet/internal users are required to make a secure connection inotes_wa_sametimeproxyserver=https://stproxyserver.company.com:9443 For internet users, set inotes_wa_sametimeproxyserverssl to the URL of the reverse proxy inotes_wa_sametimeproxyserverssl=https://revproxy.company.com 41
Reverse Proxy Configuration for Lotus inotes Software Based Solution Apache Easy to setup Caching Low cost IBM WebSphere Edge Robust Easy Ideal for mid size deployments Hardware Based Solution F5 Networks BIG-IP Load Traffic Manager (LTM) Highly Scalable Advanced scripting support Ideal for complex deployments 42
Reverse Proxy Configuration for Lotus inotes using Apache Apache Reverse Proxy: Apache2 Vhosts can be configured Scalable Configuring the Reverse Proxy server Modules mod proxy, rewrite, proxy-balancer (Loadmodule.conf) Configuration Files httpd.conf, ssl-global.conf, loadmodule.conf, listen.conf ProxyRequests Off Mapping rules: set directives for inotes Example: ProxyPass / http://dom1.acme.com/ ProxyPassReverse / http://dom1.acme.com/ 43
High Availability Configuration for Lotus inotes using Apache Apache Reverse Proxy: Load Balancing Example: ProxyPass /balancer-manager! ProxyPass / balancer://inotescluster/ stickysession=jsessionid nofailover=on ProxyPassReverse / http://dom1.acme.com/ ProxyPassReverse / http://dom2.acme.com/ <Proxy balancer://inotescluster> BalancerMember http://dom1.acme.com route=dcmail1 loadfactor=50 BalancerMember http://dom2.acme.com route=dcmail2 loadfactor=50 ProxySet lbmethod=bytraffic (or byrequests) </Proxy> Enable SSL use openssl to generate ssl keys and certificates to support SSL traffic 44
Case Study Reverse Proxy using Apache Central Bank of India 7000 users provisioned. 4500 active users 35% users use inotes as the only mode of mail access Majority of inotes users access mails over internet. Some branch location users connect via low speed dial up links Solution Domino servers in cluster Users mail files clustered across all servers Apache Reverse Proxy used for inotes users from Internet Apache on SUSE linux Domino ICM used internally SAN storage 45
Central Bank of India's Deployment 46
Reverse Proxy Configuration for Lotus inotes Software Based Solution Apache Easy to setup Caching Low cost IBM WebSphere Edge Robust Easy Ideal for mid size deployments Hardware Based Solution F5 Networks BIG-IP Load Traffic Manager (LTM) Highly Scalable Advanced scripting support Ideal for complex deployments 47
Reverse Proxy and High Availability Configuration for Lotus inotes Using F5 F5 BIG-IP LTM Hardware-based Advanced Application Delivery Controller Improves Application Performance Intelligent compression Optimized TCP/IP stack (TCP Express) improves performance on WAN/LAN Secure the Applications and Data Selective, hardware-based encryption Offload Tasks from the Application Servers OneConnect, minimizes connections on the server side Fast Cache, caches server content on the BIG-IP SSL Offload, hardware optimized for SSL encryption Compression Total Application Control irules allow you to customize how you intercept, inspect, transform and control application traffic 48
Case Study Reverse Proxy using BIG-IP LTM Tests at Lotus Performance Lab Domino Configuration BIG-IP LTM 2 Domino Servers Mirrored Configuration 4000 Concurrent Users / Server GZip HTTP Caching SSL Termination Results CPU Reduction Per Domino Server. Faster Response time from Server 49
Case Study Reverse Proxy using BIG-IP LTM 4000 Mail Files on Domino 8.5.1 4000 Mail Files on Domino 8.5.1 Gzip SSL HTTP Caching Load Balancing 50
BIG-IP LTM and Domino Performance Results CPU by 28% Per Domino Server CPU by 28% Per Domino Server Reponse Time by 75% 51
F5 Networks & Lotus inotes Collaboration Working with F5 we have developed a simpler way to deploy inotes securely to the Cloud The irule that was developed find the user mail file replicas across clusters and routes the user to the least loaded servers We have added changes in IWAredir.nsf return x-header with cluster, and replica server lists. irule Configuration Requirements. This can be used by any High Availability / Reverse Proxy that can parse response headers. BIG-IP LTM Configuration Add All Domino Servers into a Pool Configure HTTP Profile for Best performance. Domino Configuration Create iwaredir.nsf on all server All domino servers should be in the same domain. E.G @ACME This is available on inotes Wiki, and DeveloperWorks. 52
IBM's Deployment of the irule mail/user2.nsf Domino Cluster 1 Domino Cluster 2 mail/user1.nsf All available mail servers share the load on forms.nsf and the other data Load balancing, failover, caching, GZIP and terminating SSL URL masking Domino Cluster 3 Domino Cluster 4 Spraying mail/user1.nsf mail/user2.nsf 53
Conclusion Key take away points Improved Scalability More Flexibility Better Performance High Availability Enhanced Security You have options ADC???? 54
Related Information and Resources Achieving high availability with IBM Lotus inotes http://www.ibm.com/developerworks/lotus/library/inotes-avail/index.html IBM Websphere Edge Components Infocenter F5 http://publib.boulder.ibm.com/infocenter/wasinfo/v7r0/index.jsp? topic=/com.ibm.websphere.edge.doc/welcome.html Networks http://www.f5.com/products/big-ip/ Deploying the F5 BIG-IP LTM with IBM Lotus inotes http://www.f5.com/pdf/deployment-guides/f5-ibm-inotes-dg.pdf Apache Reverse Proxy Tutorial http://www.apachetutor.org/admin/reverseproxies Module lists and descriptions http://httpd.apache.org/docs/2.2/mod/ Open SSL configuration steps http://www.vanemery.com/linux/apache/apache-ssl.html 55
Lotus inotes References Lotus inotes http://www.ibm.com/lotus/inotes Feature matrix, product overview and collateral Lotus inotes area within Notes & Domino Wiki http://www-10.lotus.com/ldd/dominowiki.nsf Lotus Developer Domain http://www.lotus.com/ldd Lotus inotes 8.5 Articles http://www.ibm.com/developerworks/lotus/library/inotes-full/ Lotus Domino Web Access Performance papers http://www.ibm.com/developerworks/lotus/library/domino85-inotes/ Support IBM & Fix Central Tech Notes -- http://www.ibm.com/developerworks/lotus/support/ Fix Central -- http://www-933.ibm.com/support/fixcentral/ Lotus Greenhouse http://greenhouse.lotus.com Showcase for WPLC products and technologies Includes live community and feedback forum 56
Related Sessions AD108: IBM Lotus inotes Customization: Make Lotus inotes Your Own! ID204: What's New in IBM Lotus inotes 8.5.2 - and Beyond SHOW108: Extending IBM Lotus Notes, Lotus inotes, Lotus Symphony, and Lotus Sametime Connect with Widgets, Policies, Plug-ins and APIs ID201: What's New in IBM Lotus Notes 8.5.2 - and Beyond ID102: Best Practices for Upgrading to IBM Lotus Notes and Domino Servers to 8.5.x - Real World Analysis 57
Q&A Thank You! 58
Legal Disclaimer IBM Corporation 2011. All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. Apache is a registered trademark of The Apache Software Foundation in the United States, other countries, or both. Big-IP, Load Traffic Manager are trademarks or registered trademarks of F5 Networks or its subsidiaries in the United States and other countries. Infoware Solutions is a registered trademark of Infoware Solutions in the Sweden, other countries, or both. 59