Citrix Server Monitoring Without the Hassles and Cost ProactiveWatch Monitoring For the Rest of Us Written by: Douglas. Brown, MVP, CTP President & Chief Technology Officer DBCC, Inc. www.dabcc.com Page 0
Citrix Server Monitoring Without the Hassles and Cost ProactiveWatch Monitoring For the Rest of Us Executive Summary re you responsible for a small to medium size Citrix Server deployment? re you a consultant or VR responsible for customer Citrix deployments? If so then you must read on! few weeks ago I attended Citrix iforum and as many of you know, I m a huge fan and proponent of performance monitoring and management solutions. This being said, I must admit I was taken back a bit that at iforum, Citrix managed to mention their new EdgeSight end user application performance monitoring solution at least once in every breakout session. Now I understand it is a new solution for Citrix and that they want to get their message out but it made me think, what is the customer problem that Citrix is trying to solve with EdgeSight? If we look at the Citrix customer base, they have around 160,000 customers and over 5,000 VR partners worldwide. EdgeSight is designed to be a big boy application performance monitoring tool that requires a bit of knowledge to setup, configure and manage, not to mention some additional hardware. I also think that a good chunk of those 160,000 customers are not big boy enterprises and they don t want more infrastructure and more hardware. This being said, for how many of those customers is EdgeSight ultimately the correct solution? I think the answer is that over time Citrix EdgeSight will become a top shelf end user application monitoring tool for large and sophisticated Citrix deployments that are supported and managed by a large and sophisticated internal IT staff. If EdgeSight can ultimately live up to this promise that Citrix laid out at iforum, then this would be huge for Citrix, as it would solve the problem of Citrix applications performance being more difficult to monitor than on other leading edge platforms (like a web server, coupled with a J2EE (Weblogic or WebSphere) applications server, and a back end Oracle database). So, if one day Citrix EdgeSight solves the large and sophisticated customer problem, what is left? The answer is that I believe most of the Citrix deployments in the world are small (less than 10 Citrix Servers, and less than 30 total Windows servers). These deployments are managed in many cases by an IT staff that is spread too thin, with too many tasks and operating on a limited budget with no ability to put in, much less maintain an onsite application performance monitoring and management solution. In fact the overwhelming majority of the Citrix deployments in the world are not managed by the internal IT staff at all, but by consultants who work for the VR who installed and built the Citrix servers. If this sound like you then you must read on Page 1
1 Introducing ProactiveWatch Last year, I got my first exposure to ProactiveWatch, a company that has completely taken a different approach to monitoring Citrix environments and the servers and networks that surround them. Before I explain what ProactiveWatch is, let me tell you what it is not: 1. ProactiveWatch is not hard to install. I was able to get it up and running on the DBCC.com web servers in minutes with no configuration and no need for any backend components that I would be forced to deploy and manage. 2. ProactiveWatch is not hard to configure. It comes with templates that are preconfigured and work out of the box 3. You don t have to put in any extra infrastructure for ProactiveWatch because it is delivered on a software as a service basis (No additional servers or SQL required). You just install the ProactiveWatch agent on your servers and point them to the Proactive Watch gateway (a Windows Service running on a non-dedicated server) and then install the Console (the ProactiveWatch Explorer is a rich client.net application) on the PC or a server if you want to publish it via Citrix. You do not need to buy or install a back end database or applications servers because ProactiveWatch hosts those back end pieces for you. 4. Even though the ProactiveWatch agents at your site talk over the Internet to the hosted ProactiveWatch back-end pieces, you do not need to poke holes in firewalls to make all of this work. Just install the Gateway Service on a box that can open an outbound connection on 443 and you are good to go. 5. It does NOT COST LOT OF MONEY! ProactiveWatch is sold through the Citrix VR community (more on this later), and pricing starts at $30 per managed server per month. 6. For all of the above reasons, ProactiveWatch does not require a lot of time. The whole thing only takes minutes to install and works right out of the box. Page 2
2 Some Cool Things about ProactiveWatch Now that you know what ProactiveWatch is NOT and how it is different from all of the other monitoring products out there, let me tell you some things that are really unique and cool about it: 1. Despite the fact that it is inexpensive and easy to set up, it understands what a Citrix server is and what happens on Citrix Servers. It collects its data once every 10 seconds, and collects it on a per process instance basis. So, it not only tells you that Outlook is causing a problem, but whose instance (what user ID) of Outlook it is. 2. When a problem occurs it snap-shots the state of the server. Then it captures the detail of which individual processes are causing the problem. It also captures the entire System Profile, and set of installed pplications, Security Updates and Hot Fixes that were in place at the time of the issue (see a screen shot of the diagnostics that come with an alarm below). Page 3
3. ProactiveWatch monitors networks at the applications level of the TCP/IP protocol. Most monitoring products use ICMP Ping to measure latency between two end points on a TCP/IP network. The issue with Ping is that a blue-screened server with a smart network card will quickly respond to a Ping when in fact the server is useless from the perspective of doing any real work for users. ProactiveWatch establishes a socket between any two computers where you want to monitor network latency, which means that it is measuring the performance of the network from the perspective of the applications that are using that network. 4. ProactiveWatch has extremely cool capacity and load analysis features that are actually useful in a Citrix or MSTS environment. The ProactiveWatch agent collects the number of active sessions on the server every 10 seconds and also collects 9 key utilization metrics (like CPU, Physical Memory, Virtual Memory, etc.). The system keeps the maximum levels for each of these metrics for the last 24 hours, 3 days and 7 days for each level of active concurrent user. So, you get a real picture of maximum resource utilization as the real load grows on your servers. In the screen shot below, you can see that the server is at 80% of Physical Memory once it gets to around 21 or 22 concurrent users. My guess is that once load gets above 23 or so concurrent users, this server will not be delivering a good experience to its users. 5. ProactiveWatch couples the capacity analysis with true analysis of load over the course of the day. The graph below shows that the server spends most of its day above 20 concurrent users. (Poor users! I m feeling sorry for the admin too.) Page 4
6. I also really like the ability on the part of ProactiveWatch to compare servers (or workstations) to each other. Let s first take a look at the load graphs for two servers that are supposed to be the same, CTX01 and CTX04 (same hardware, software, published apps, etc.). But as you can see they are reacting very differently to the load. CTX01 is running out of memory and CTX04 is not. Why? The reason why is that different software has crept onto these two supposedly identical servers..net and Java are both on the server that is running out of memory (imagine that), and not on the other one. If you are a VR you can even do this comparison across servers at many different customer sites to ensure consistency of servers. The comparison below shows the differences in applications, security updates, and hot fixes that are installed across CTX01 and CTX04. These comparisons can be done across any number of servers, and can even compare servers at the partner s site, to servers at the customer s site. Page 5
ProactiveWatch also allows you to compare hardware profiles and even published applications across any set of servers (again even across servers that are installed at different customer sites). n example of a hardware profile compared across four different Citrix Servers in a farm is below. Notice that the hardware for CTX01 and CTX04 is the same, which points us back to.net and Java as the cause for the memory issues on CTX01. 3 The Monitoring Details So what exactly does ProactiveWatch monitor? Well contrary to most of the monitoring products out there today that try to take a variety of PerfMon Counters, collect them and make sense of them, ProactiveWatch works by collecting the information that is inherently useful in the first place. So, other than the core CPU, Memory, Disk, etc. metrics that it collects frequently along with per process instance granularity, most of the effort expended by ProactiveWatch is upon uniquely valuable information. Here is a list of the various monitors, the data they are based on, and how they work: 1. Connectivity this is a socket between the agents at the customer site and the ProactiveWatch back end. If you want to know if a computer or a site has dropped off of the Internet, this monitor will tell you. The diagnostics for this monitor run a tracert from the ProactiveWatch back end back to the agent that dropped off, so you can follow the trail of where the connection died. 2. Latency this is a round trip test packet over a socket connection between the agents and the ProactiveWatch back end. gain this is what a browser would experience trying to connect to a web server, not an ICMP Ping. Page 6
3. Client2Server Latency this is a round trip test packet over a socket between any two sets of computers. So this can measure applications level latency between a remote site and the Web ccess Gateway, between Web ccess Gateway and the Citrix Farm, and then between the Citrix Farm and the back end servers. 4. CPU the server is running out of CPU, or a single process is using all of the CPUs on the server. 5. Memory the server is running out of either Physical Memory or Virtual Memory. 6. Disk the server is running out of disk space on a monitored drive. 7. Boot the server has rebooted. If this happens during the nightly reboot window it is not a problem, but not if it happens in the middle of the day. 8. pps an application has been installed or uninstalled. 9. Profile the system profile has changed. This can be a change in IP address, a change in memory, etc. 10. Registry the Run or RunOnce sections of the Registry have been changed. This detects software that has been installed without going through add/remove programs. 11. Crash a process has crashed. The diagnostics include the Dr. Watson log entry for the module that caused the crash. 12. Service a monitored service has gone down. ProactiveWatch can monitor manually specified services, or all automatically started services. 13. Process a monitored process has gone down. 14. Log a monitored event log entry has occurred. 15. Port either a desired port (1494) is not listening, or an undesirable port (like an FTP server on a Citrix server) has shown up where it should not be. 16. gent Types ProactiveWatch does a great job with Citrix and Microsoft Terminal Servers, with any other type of Windows server, and also has agents for several different types of Linux. Some of the partners working with ProactiveWatch have worked with their development team to port the Linux agent into the firewalls that the partner sells. Page 7
4 Very Cool ProactiveWatch Things for VRs or Citrix Partners ProactiveWatch has two unique features that make it absolutely outstanding for VRs or Partners that want to use it to monitor all of the servers at all of their customer sites. The first of these is a unique communications architecture that is both one-way and Internet friendly. When you install the software at a site, you only need one non-dedicated server that can open an outbound port (443) to the public Internet. The ProactiveWatch Gateway is installed on this server its job is to collect the data from the agents at that site and send it to the ProactiveWatch back end. ProactiveWatch does not require or use any inbound connections, so you do not need to open any inbound ports to make it work. You can easily install a gateway and a couple of agents at different customer sites without having to do any firewall or router work. The diagram below shows how ProactiveWatch uses only outbound Internet friendly communications to work. Remote Site Citrix ccess Gateway VR Explorer dmin Console ProactiveWatch Gateway Port Open Outbound Only (29443) HTTPS (443) Port Open Outbound Only Internet Firewall Web Servers Firewall Customer s Other Windows or Unix Servers ProactiveWatch Backend Server Farm HTTPS (443) Port Open Outbound Only Port Open Outbound Only (29443) Customer Explorer (Optional) ProactiveWatch Gateway Internal Users Port Open Outbound Only (29080) Customer s Citrix Presentation Server Farm The next feature is that the system by default views things on the basis of Customer Location ComputerName. So the idea of one VR or Partner being able to see multiple customers systems on one pane of glass is a core built in design assumption of ProactiveWatch. The system also supports the idea of an dmin at a customer site having his own console that allows him to see only his environment, while the VR is seeing this dmin s environment as well rolled up into a view of all of the VR s customers. For VRs and Partners, ProactiveWatch therefore opens up the following opportunities: 1. The VR can resell ProactiveWatch on a subscription basis to any of his customers who have an dmin, and can then do collaborative support of the customer s issues with the dmin based upon the same set of diagnostic data. The idea of the consultant at the VR working with the dmin at the customer site on the same problem, off of the same set of data, and in real time is an absolute breakthrough. 2. The VR can make his support of his smallest customers for whom the VR is the IT department much more valuable, more nimble, and less expensive to deliver. Page 8
3. The VR can see the servers at the customer site change. s inconsistencies creep into the environment (user installed toolbars in IE, or hot fixes are applied only to some servers) the VR gets notified, and can do instant comparisons across servers. This includes the ability to compare servers at the customer site against a reference server at the VRs site! n example of this See all of your customers on one pane of glass view is below The dmin at the CMSI customer site would see this: Page 9
5 Conclusions I have seen a lot of monitoring products over the years. This is the very first one that does a great job for the small to medium sized Citrix deployments and for the dmin at the customer site at a low cost in terms of effort and money. This is also the very first monitoring product that is designed for this size of business and deployment. I have seen that it can effectively scale down to a 5 or a 10-server environment while providing such a rich level of functionality. Heck, I run it on my DBCC.com web servers and it provides me the information I to need to know what is happening and the best part is that I did next to nothing for this valuable information. Finally, I have never seen a product that is as specifically designed for the partner community as this one is. If you are a customer, then contact your reseller. If you are a Citrix partner who manages Citrix customer deployments, then this tool is perfect for you! From my first hand experiences as a consultant who was responsible for a slew of Citrix servers across many different companies, it is refreshing to know that now there is a solution that solves my problem, as ultimately my problem is the customer problem times the amount of customers I have! If you want to learn more please visit www.proactivewatch.com If you would like to speak to someone who will answer your questions, please send an email to sales@proactivewatch.com Page 10