Jeff Schertz MVP, MCITP, MCTS, MCP, MCSE

Jeff Schertz MVP, MCITP, MCTS, MCP, MCSE A comprehensive excerpt from Jeff Schertz s Lync Server MVP Blog Lync Web Services Load Balancing with KEMP VLM

This article addresses a standard DNS Load Balanced scenario utilizing a Hardware Load Balancer (HLB) for web server requests only. What is unique about this setup though is that the HLB is not actually a hardware solution, as the KEMP VLM is a virtualized service. Although operation is identical between each, the Hyper-V version (also available as a VMware instance) was used for this deployment. KEMP Technologies publishes and maintains a detailed deployment guide covering various Lync topologies (Front End, Director, Edge) but as highlighted at the end of Section 3 that guide does not cover the DNS Load Balancing scenarios where the HLB is only used for web traffic. It is only intended to be used for full load balancing scenarios where the LB handles all Lync traffic and not just web server requests. Environment A completely new Lync server environment was deployed and utilized for this article as all previous articles on this site have utilized a Standard Edition deployment (mslync.net). This new environment consists of a single Active Directory domain and split-brain DNS configuration (jds.net) with two Enterprise Edition Lync 2010 Servers. When using DNS Load Balancing for Lync Front End pools it is not required to actually have a separate load balancer to handle the web traffic, although that would not provide any level of resiliency for the web services. The topology configuration will require the definition of a new FQDN which will be passed to Lync client during on-band provisioning for all web services URLs that is different that the standard pool name. This Override FQDN is then either pointed to just one of the Front End servers (when no load balancing solution is present) or the virtual IP of a separate load balancer configured to forward web service requests to any server in the Front End pool. Using the Lync Server Topology Builder expand the Enterprise Edition Front End Pool and view the properties on the pool object. Under Web Services the Internal web services Override FQDN will be displayed (e.g. poolweb.jds.net).

To easily identify if DNS Load Balancing or a Hardware Load Balancer is being used in an environment simply look at the DNS Host records defined for the Lync pool. In this example the Override FQDN is pointed to 192.168.1.41 which is also the same IP address defined for one of the Pool FQDN Host records (e.g. pool.jds.net) as well as the Server FQDN Host record (e.g. lyncfe1.jds.net). If a load balancer was used then the Override FQDN would instead be pointed to a unique IP address which would traditionally be an HLB s virtual IP. As there is no actual load balancer for this Front End Pool then one will now be added. VLM Setup Virtual LoadMaster version 6.0-28a was used for this article in conjunction with version 1.3 of the deployment guide as a reference, but not that some of the steps in this article are not currently documented by KEMP (primarily the SSL configuration at the end of this article). The Hyper-V package includes a ReadMe file which explains how to import the virtual machine into Hyper-V. But after importing the virtual machine an error will be displayed which can be verified in host server s event log. Browse to Applications and Services Logs > Microsoft > Windows > Hyper-V-VMMS > Admin event log and look for the Event IDs of 18330 which indicate that the virtual network adapters in Hyper-V are not the same adapters which where available to the guest when it was last shutdown and packaged for distribution.

View the settings on the new virtual machine and select the desired network for each of the two adapters to resolve this issue prior to starting the guest. Start the virtual machine and then connect to the console to view the running status and assigned IP address of the system. It will receive a DHCP address by default if one is available on the network (e.g. 192.168.1.110). Using a web browser go to the assigned IP address over HTTPS and then log-in with the default username (bal) and password (1fourall) as described in the KEMP LoadMaster Quick Start Guide. https://192.168.1.110 Enter the license key and define a new password for the administrator account.

Navigate to System Configuration > Interfaces > eth0 and enter a static IP address in the Interfaces Address field in the format shown below. Click Set Address and confirm the change. The browser should be redirected to the new IP address. 192.168.1.43/24 Select System Configuration > Local DNS Configuration > Hostname Configuration and enter the desired short hostname (e.g. VLM1000). Select System Configuration > Local DNS Configuration > DNS Configuration and enter the desired DNS server(s) for the environment. Select System Configuration > Route Management > Default Gateway and enter the associated router IP address for the local subnet (e.g. 192.168.1.1). VLM Configuration Much of the configuration steps are already included in the official KEMP deployment documentation for Lync integration, but some key steps are not explained very clearly so those will be broken-out individually below. (Version 1.3 which was published in March 2012 is used for this article so future updates to the KEMP documentation may change the section numbers and/or order of steps detailed below.) Open the KEMP LoadMaster Deployment Guide for Microsoft Lync 2010 from their Lync Solutions page and go to section 5 to complete the General Configuration steps. Portions of Section 6 will be used to define the new virtual service which handles load balancing client requests directed to the Lync Front End web services. SKip directly to Section 6.5 as the initial steps are used for balancing SIP, conferencing, and other traffic which is not applicable to this configuration. In Step 3 (6.5.3) the directions state to enter the virtual IP address of the Lync Server Internal Base Webservices URL but this IP address must first be defined (the guide assumes this has already been done). Since the Lync topology is currently using one of the pool servers (e.g. 192.168.1.41) as the target for the internal Override FQDN then this configuration must be modified to select a new IP address for the load balancer to use. Verify the currently defined Internal Web Services Override FQDN in the Lync Topology (e.g. poolweb.jds. net) and then update the existing DNS Host (A) record for that FQDN with a new, unassigned IP address (e.g. 192.168.1.44)

Complete the steps in Section 6.5 using the new IP address as the Virtual Address for the new virtual service. A couple of items in Section 6.5 do not match the version 6.0 interface (like the Netmask setting) but once the steps in Section 6.5 are complete the virtual service configuration should match the settings shown below. Once the virtual service configuration is complete then browse to the Virtual Services > View/Modify Services menu to verify that the Status is reported as Up.

Create another new virtual service port HTTP request over port 80 (primarily used by Lync Phone Edition clients for Root certificate download during initial provisioning). Note that the Server Check URL is different as the /abs/handler string used in the HTTPS rule is not a valid check for HTTP traffic. The connection would fail over port 80 with a 403 server error causing the load balancer to see the services as offline. Pointing to the blank.html file stored under the meet directory is just one possible way of testing a successful HTTP connection to the server. Once the virtual service configuration is complete then browse to the Virtual Services > View/Modify Services menu to verify that the Status is Up for both virtual services.

If a reverse proxy is deployed in the environment then follow the steps in Section 6.6 to create another virtual service for each external web services (ports 8080 and 4443), applying the same guidance as above by defining another unallocated IP address (which the reverse proxy rule would be pointed to). SSL Acceleration In the past the currently detailed configuration would be sufficient for a supported Lync load balancer deployment, but since the addition of the Mobility services in Cumulative Update 4 there has been a shift in best practices regarding the use of SSL with load balancers. At the time of the initial Lync 2010 release this was not supported and when improperly configured could cause issues with some client functionality (e.g. Lync Phone Edition firmware updates). Note: If Lync Mobility services will not be used in a deployment (no requirement to support Lync Mobile clients) then this additional SSL configuration can be ignored as the existing HTTP/HTTPS configuration will be sufficient for the Front End pool. But with the inclusion of the Mobility web service (Mcx) it is now required (and thus best practice) to implement SSL decryption on at least the Front End pool web services. The reasoning for this is to provide proper session affinity from the mobile client to the same Front End server throughout the client session. That affinity is handled via cookies on the client and the load balancer must be able to decrypt the HTTPS client session, inspect the traffic to read the cookie and decide which Front End server the session began with, and then re-encrypt the traffic. This is not possible without enabling SSL decryption as the load balancer would be unable to read the traffic and would simply forward it on to any Front End server. When the mobile client has not been used for some time and the user re-invokes the session the client requests may be seen as a new connection by the load balancer, yet the previous session is still stored in the memory of the specific Front End server which originally handled that client s sign-in request. So for proper mobile client operation it is important that the client is always reconnected back to the same Front End server as the individual servers in a pool do not replicate or share session state data with each other. Note that this functionality is commonly referred to as SSL Acceleration or Offloading as traditionally offloading indicates that the SSL processing load is actually offloaded from the real servers to the load balancer for performance advantages. But in this case the client SSL connection will be broken down and terminated at the load balancer to inspect the cookie data and then re-encrypted again for the remainder of the trip back to the Front End server to the HTTP 443 service. Typically Offloading would be client-to-load balancer over 443 and load balancer-to-server over 80. Certificate Preparation Prior to enabling SSL on the load balancer a pair of certificate files are needed. The VLM does not support importing an entire chain from a single file so the existing root certificate and a new server certificate for the virtual service itself will need to be provided in separate files. The root certificate is simple to locate and install, but for the virtual service certificate it will either need to be created or one of the existing certificates issued to a Lync Front End server can be used. Although the simplest approach is just to use one of the certificates already created for a Lync Front End server in the pool it is a better practice to issue a dedicated certificate to the real server containing only the Subject Names which are applicable to the configured virtual service. Although technically there is no harm in providing additional names which are not needed, having the Common Name field of the certificate use an FQDN which has nothing to do with the load balancer is not a solid practice (e.g. labfe1.jds.net). Thus a unique certificate will be generated for the virtual service containing on the FQDNs applicable to web services (e.g. poolweb, meet, dialin). Request New Certificate On one of the Lync Front End servers launch the Lync Server Deployment Wizard and re-run the Request, Install, or Assign Certificates step. (The Lync certificate wizard will be used to request this certificate but it will not be assigned to any Lync Services.)

Under the Default certificate item select only the Web service internal usage and then click Request. Select the default option to Send the request immediately to an online certification authority and then select the desired internal Windows Enterprise CA if there is more than one. Make sure to use the same certificate authority chain which was used to issue the existing certificates to the Lync servers. On the Name and Security Settings window enter a descriptive Friendly Name for the certificate, leave the default option of a 2KB length and then make sure to enable the Mark the certificate s private key as exportable option. As this certificate will need to later be exported from the Windows Server and imported into the load balancer the private key must be allowed to be exported with the public key, otherwise the certificate will be useless to the load balancer. Without the private key the host using the certificate would be unable to decrypt and traffic signed using its public key.

On the Subject Name / Subject Alternative Names window notice that the desired FQDNs should already be provided since only the internal web services usage was selected for this request. Complete the certificate request and when the Online Certificate Requests Status window appears make sure to deselect the Assign this certificate to the Lync server certificate usage option. The certificate should not be applied to the Lync Server as the wizard was only used as a simple way to request a SSL certificate with the desired SN and SAN configuration. Finish the wizard and verify that none of the certificate usages on the Lync Server have been modified. Export Certificates Now that the new certificate has been created it must be exported to a file so it can be loaded into the load balancer along with the root certificate. On the same Lync Front End server where the new certificate was just requested launch mmc.exe and add the Certificates snap-in, making sure to select Computer account and Local Computer. Expand Certificates (Local Computer) > Personal > Certificates and locate the newly created certificate in the list.

Right-click the new certificate and select All Tasks > Export to launch the Certificate Export Wizard. On the Export Private Key window make sure to select Yes, export the private key. On the Export File Format window leave the default options and do not enable any of the checkboxes. The certificate chain should not be provided as that will be handled through a different process and this export must contain only the new certificate s public and private key, nothing more.

Provide a password to protect the.pfx file (e.g. password) and then select a filename and location to save the exported package (e.g. c:\temp\vlmcert.pfx). Now that the load balancer certificate is exported the next step is to locate and export the root certificate for the issuing certification authority. In the same MMC Console window open the new certificate and then click on the Certification Path tab. Highlight the root certificate in the path and then click on the View Certificate button. In the root certificate window select the Details tab and then click the Copy to File button.

The Certificate Export Wizard will be launched again and in the Export File Format window select the option Base-64 encoded X.509 (.CER) and then complete the wizard providing a path to save the exported root certificate to (e.g. c:\temp\rootcert.cer). (Do not select DER encoded binary as this is the wrong format and will not work with the Kemp VLM installation process.) Import Certificates into VLM At this point there should be two separate export files (e.g. vlmcert.pfx, rootcert.cer) which need to be imported in the the load balancer. The root certificate will be installed first. Using Windows Explorer on the same Lync Front End server where the certificate were just exported browse to the root certificate file (e.g. rootcert.cer) and open this file with Notepad (right-click, Open With). Select all of the text and copy it to the clipboard

In the KEMP LoadMaster web management console select the Certificates > Intermediate Certs menu option and then click the Add New button. Paste the contents of the clipboard into the Intermediate Certificate field and then create a descriptive name for the new file which will be saved in the load balancer file system (e.g. JDSRoot.pem). The file name does not have to match the name of the exported file, it simply must be a unique name for this certificate file as it is stored on the VLM filesystem. If there are no other intermediate certificate files yet loaded into the VLM then any name can be chosen. Click Add Certificate and if the installation is successful the new certificate should be listed as shown below. Now that the root certificate is installed in the VLM then the next step is to import the server certificate. Select the Certificates > SSL Certificates menu option and then create a descriptive name for the new certificate to be installed into the VLM (e.g. LyncWebPool) and then click the Import Certificate button.

Ignoring the first two large text fields advance to the bottom of the page and click the Browse button to locate and select the exported server certificate file on the local computer (e.g. vlmcert.pfx). Enter the same password used during the initial export process of the PFX (e.g. password) and then click Store. A message should appear reporting the successful installation of the new certificate and then the console should refresh showing the new certificate as installed, but not yet assigned to any virtual services. Enable SSL Acceleration The final step is to enable SSL on the previously configuration HTTPS virtual service in the load balancer. Select the Virtual Service > View/Modify Services menu option and click Modify on the FE Pool HTTPS virtual service configured for HTTPS/443. (Do not select the HTTP/80 virtual service.) Expand the SSL Properties menu item and select the Enabled checkbox. A message will appear stating that there is no certificate file available for this service so a self-signed certificate will temporarily be assigned to the virtual service. Return to the SSL Properties and then click the Add New button in the Certificates row. In the VS To Add drop-down menu select the desired IP:Port (e.g. 192.168.1.44:443) and then click the Add VS button.

At this point the configuration should update and show that the new certificate is successfully bound to the HTTP virtual service. To complete the configuration browse to the HTTPS virtual service properties again and select the following options. Under Standard Options change the Persistence Option of Mode to Server Cookie and leave the default options for the other settings. (A different Cookie Name then Set_Me can be entered; the functionality is not impacted by the name but when searching through system logs a more descriptive name may be desirable.) Under the SSL Properties enable the Reencrypt option to ensure that the traffic forwarded on to the Lync Front end servers will still be directed to the proper 443 port for internal web service client connections.

At this point the HTTPS listener should be complete and the Status reported as Up. To read this review online on Jeff s Blog, visit this link: http://blog.schertz.name/2012/06/lync-web-services-load-balancing-with-kemp-vlm/ About Jeff: Jeff is currently a Microsoft Solutions Architect for Polycom, focused on helping customers integrate our video conferencing solutions and UC endpoints with Microsoft OCS and Lync environments. As a Microsoft MVP awardee in Communications Server since 2008 his contributions include TechNet forum participation and moderation, writing technical blog articles and white papers, presenting on UC topics to virtual users groups online, and representing Microsoft at TechEd and other public conferences. Other recent duties include working on various content development tasks for product documentation, testing and training materials as an external consultant. As a consultant in Microsoft infrastructure technologies Jeff has over 15 years of experience in the IT industry, most recently as a Senior Consultant for PointBridge Solutions on projects ranging from targeted global messaging migrations to enterprise Active Directory, Exchange Server, and Communications Server architecture and deployment. In the past he has held several system administration positions for school districts, software developers, consumer product manufacturers and data/voice service providers. Jeff currently lives in Chicago and enjoys playing guitar, photography, snowboarding, traveling, and motorcycle racing. If you'd like to contact Jeff, Email address: jeff@schertz.name