Best Practices in Scheduling Patch Installation for Minimal User Impact Greg Shields 1. 8 0 0. 8 1 3. 6 4 1 5 w w w. s c r i p t l o g i c. c o m / s m b I T
2012 ScriptLogic Corporation ALL RIGHTS RESERVED. ScriptLogic, the ScriptLogic logo and Point,Click,Done! are trademarks and registered trademarks of ScriptLogic Corporation in the United States of America and other countries. All other trademarks and registered trademarks are property of their respective owners. 2 Best Practices for Scheduling Patch Installation for Minimal User Impact
We IT administrators spend so much time worrying about patches themselves that sometimes we forget about our users. Indeed getting those monthly Microsoft patches installed is important, but when that installation causes users to miss their deadlines we re actually hurting our business. That s why working with patches and a patch management solution requires more than just clicking buttons. It requires more than just a priority on keeping things secure. It requires a look at how our users behave, and how we can fulfill our requirements for security while minimizing our impact on everything else. What you need is a patch management schedule. How Not to Do Patching As an IT analyst and occasional consultant, I get the regular opportunity to pop into other people s networks and poke around. That opportunity puts me into the networks of the very smallest all the way to the very largest IT organizations. While every network is different, you d be amazed at how common some tactics are. Figure 1 WSUS Patch Configuration in Group Policy Management Console Take, for example, the usual settings many administrators configure for deploying WSUS patches. These settings are strikingly common, even as they completely obliterate user productivity on patch deployment day. Even worse, a casual read through the blogosphere would have you think that they re actually a good idea. Here are the Group Policy settings found in Computer Configuration Administrative Templates Windows Components Windows Update that define how you shouldn t schedule patching: Configure automatic updates. Under Configure automatic updating: 4 Auto download and schedule the install. No auto-restart with logged on users for scheduled automatic updates installations. Disabled. Delay Restart for scheduled installations. Enabled, with any configured value. 3 Best Practices for Scheduling Patch Installation for Minimal User Impact
Reschedule Automatic Updates scheduled installations. Wait after system startup, with any configured value. Let s take a look at what this combination of Group Policy settings actually accomplishes. Its first setting configures automatic updates to download and install once they ve been approved and the start time arrives. Configure that start time for the middle of the night, and most of your patchwork gets done when no one s around. This approach might have worked well back in the days when most computers never left the office. But today, most computers aren t desktops anymore. They re laptops, and they re not necessarily guaranteed to be powered on when the start time kicks off. Here s where the rest of these settings actually cause more problems than help. Computers that are powered off during their time for patching will, with this configuration, start that patch installation process once they power back on. That period is usually when the user needs the computer the most, right when they re starting work for the day. The collection of bad settings here may give the user a postponement and even a delay to partially insulate them from the pains of the post-installation patch reboot. But if that user powers on their laptop, starts working on a document, and then walks away for a while, they re going to be in for a world of hurtful rebooting. Figure 2 Configuring Automatic Updates Introducing the Patch-and-Hold Technique Many administrators configure these less-than-optimal settings because of a long-held fear of not immediately rebooting a computer after a patch installation. That fear is admittedly warranted. Back in Windows early days, not rebooting after a patch installation could cause real problems. Half-installed patches lingering around a system drive in those days sometimes caused BSODs or worse. Today s Windows Update service is far smarter than what it was in the old days. The Windows Update service has for a long time been intelligent enough to pre-stage patches that require reboot. Once pre-staged, the computer can continue to operate without problems. Patches awaiting a reboot will get installed later during that reboot. 4 Best Practices for Scheduling Patch Installation for Minimal User Impact
This added intelligence grants the patching administrator the ability to patch systems, but hold off on the reboot until a more appropriate time in the future. One could still, for example, set the Configure Automatic Updates policy setting to Auto download and schedule the install for some period during the workday. Your next step is to prevent the reboot from happening. You could accomplish this by setting the No auto-restart with logged on users for scheduled automatic updates installations policy setting to Enabled. Doing the combination of these two settings gets patches installed while users and their laptops are in the office, but prevents the reboot until the next time they actually reboot the computer. You can speed this process even more for patches that don t require a reboot for their installation. Those that don t can get automatically installed by setting the Allow Automatic Updates immediate installation policy setting to Enabled. Figure 3 No Auto-Restart Configuration How to Handle Reboots: The Scheduled Reboot Window This combination is pretty powerful stuff, and it s functionality that you can get right out of the box with Group Policy. Yet there s something that s absolutely missing that keeps this from being a complete solution: Reboots. While the settings in the last section are great for getting rid of the unpredictable reboots that happen as a result of patch installation, they do nothing for actually getting those reboots programmatically implemented across the board. Indeed, you can just tell your users, Hey, you should probably reboot from time to time. But we all know that users invariably do the wrong thing when we give them the right to choose. WSUS and its Group Policies alone don t address the reboot problem very well (a point that I ve already made in the beginning of this article). As a result, solving the reboot problem requires some tools that exist outside WSUS itself. The solution that works the best across nearly every implementation is what I ll simply call a scheduled reboot window. 5 Best Practices for Scheduling Patch Installation for Minimal User Impact
A scheduled reboot window is a period of time that you and your users negotiate wherein you re allowed to reboot every desktop in your organization. Perhaps that window is from 2:00PM to 4:00PM on Wednesdays and Saturdays. Maybe it s slightly past the end of workday. Notwithstanding when you agree upon your reboot window, it is a set time period every week that you re given carte blanche to reboot everything. Having that reboot window means being able to Patch-and-Hold users computers. Those with desktops will get rebooted because they re on the network. Those with laptops will get rebooted when they unplug at the end of the day and head home. System, patched; system rebooted. Scheduling that reboot window to occur just past the end of Patch Tuesday also ensures you get patches installed and systems rebooted quickly after patches are announced. Now, your next question probably asks, So, how does one script the reboot? The possibilities are numerous. You could create a VBScript or PowerShell script to accomplish the task, scheduling either perhaps via a Scheduled Task. Doing so via a scheduled script obviously solves the task, but requires some first-hand knowledge of scripting. You ve also got to keep an eye on your Scheduled Task, ensuring that it runs every night like it s supposed to. Here s a quick script that can reboot a bunch of computers, all at once. You ll need a file called computers.txt that contains computer names, one per line, in the same folder as the script. RebootFile = "computers.txt LogFile = "results.txt" Set fso = CreateObject("Scripting.FileSystemObject") Set f = fso.opentextfile(rebootfile, 1, True) On Error resume next Do While f.atendofline <> True strcomputer = f.readline Set objwmiservice = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & strcomputer & "\root\cimv2") If Err.Number <> 0 Then Err.Clear Else Set coloperatingsystems = objwmiservice.execquery("select * from Win32_OperatingSystem") For Each objoperatingsystem in coloperatingsystems ObjOperatingSystem.Reboot() Next End If Loop This script is admittedly very basic. It s also VBScript, which every IT pundit in the world will tell you is a language soon to go away. Accomplishing the same task in Windows PowerShell is remarkably simpler: Restart-Computer -computername (Get-Content computers.txt) 6 Best Practices for Scheduling Patch Installation for Minimal User Impact
The Pain of Reboot Scripts Reboot scripts are ridiculously powerful, when you ve got the time to code them correctly. The script above very obviously reboots computers, but it does nothing for letting you know which ones actually responded. It doesn t let you know when computers have problems, or when they re just not behaving correctly. A desktop management solution can give you better control over this whole reboot process. Such a solution might introduce the kinds of logging that alerts you when computers misbehave. Scripts alone are also challenging for those special cases, like when you want to reboot everybody except for the vocal few, like the executives and those annoying developers who run jobs at night. Scripts run via Scheduled Tasks also do little for laptop users who rather than shut down their computers as they leave for the night, instead choose to just put them to sleep. Some laptop users in the Scheduled Tasks situation might go days or even weeks without rebooting, thanks to Windows built-in hibernation features. So the moral of this story is: Get yourself off the WSUS settings that benefit you but cause problems for your users. Get yourself on a scheduled reboot window, to ensure your patches complete their installation in a timely manner. And get yourself a desktop management solution, one that s primed to assist when those special cases become problematic. 7 Best Practices for Scheduling Patch Installation for Minimal User Impact