System Center 2012 R2 Service Management Automation

System Center 2012 R2 Service Management Automation WHITEPAPER Version 1.0.4 Author Michael Rüefli, Microsoft MVP for CDM, Cloud Architect at itnetx gmbh, Switzerland Twitter:@drmiru Blog: http://www.miru.ch Reviewers Michel Lüscher, Datacenter Architect, CoE, Microsoft Twitter: @MichelLuescher Blog: http://www.server-talk.eu Kristian Nese, Microsoft MVP for CDM, Infrastructure Ranger, CTO at Lumagate, Norway Twitter:@KristianNese Blog: http://kristiannese.blogspot.ch/ Aleksandar Nikolic, PowerShell MVP, the co-founder and editor of PowerShell Magazine Twitter:@Alexandair Blog: http://www.powershellmagazine.com/

Document History Version Author Changes / Comments 1.0.0 Michael Rüefli Initial version after review 1.0.1 Michael Rüefli Minor corrections to referenced blog posts according to input by Jim Britt 1.0.2 Michael Rüefli Added some more info about Remoting and variables within workflows 1.0.3 Michael Rüefli Corrections to chapter 5.7 (linking SMA Runbooks with VM Clouds). Additions to chapter 5.5 (using external editors) Added some links to chapter 7 1.0.4 Michael Rüefli Added comments / hints by Aleksandar Nikolic. Corrected typos and errors in examples. Added some FAQ Added topics to debugging / logging Added topics to Workflow Checkpoints Acknowledgements I want to shout out a big thank you to Michel, Kristian and Aleksandar, the reviewers, for their time invested to review this whitepaper. Also I d like to thank Jim Britt (MSFT) for his ad-hoc help and feedbacks. This whitepaper wouldn't be possible without your help guys! Special thanks go to my wife Eva and my family always supporting me doing all the community stuff beside the long work days.

Contents Version... 1 Author... 1 Reviewers... 1 Document History... 2 Acknowledgements... 2 1. Introduction... 6 1.1 What can you expect from reading this paper... 6 2. SMA Architecture... 7 2.1 Components of SMA Framework... 9 2.1.1 SMA Database... 9 2.1.2 OData REST Web Service... 9 2.1.3 Service Management Portal... 9 2.1.4 Runbook Worker... 9 2.1.5 Runbooks... 9 2.1.6 Jobs... 9 3. Installing SMA... 10 3.1 Prerequisites... 10 3.2 Installing the SMA Web Service... 11 3.3 Installing the SMA Runbook Worker... 15 3.4 Installing the SMA PowerShell Module... 18 3.5 Registering SMA Endpoint... 19 4. Scaling Out a SMA Deployment... 20 4.1 Installing Additional Runbook Workers... 20 4.2 Stop All Runbook Worker Services... 20 4.2.1 Add a New Runbook Worker Deployment to SMA... 21 4.2.2 Start the Service on All Runbook Workers... 21 5. Creating and Using Runbooks... 22 5.1 Modules and Consoles... 22 5.1.1 Local Execution... 22 5.1.2 Remote Execution (PowerShell Remoting)... 22 5.2 Creating Runbooks... 23 5.2.1 Anatomy of a PowerShell Workflow... 23 5.2.2 Creating the Runbook... 24 5.2.3 Note on Testing Runbook... 26

5.3 Creating Schedules... 27 5.4 Adding Connections, Credentials and other Assets... 29 5.4.1 Adding Connections... 30 5.4.2 Referencing a Connection in a Runbook... 31 5.4.3 Adding Credential Objects... 33 5.4.4 Referencing a Credential Object in a Runbook... 33 5.4.5 Adding Variables... 34 5.4.6 Referencing a SMA Variable in a Runbook... 35 5.5 Using External Development Environments... 35 5.6 Nesting Runbooks/Publishing Data... 35 5.7 Linking Runbooks to VM Cloud Action Events... 37 5.7.1 Getting Runbook Parameters and Resource Objects... 40 5.7.2 Getting Runbook Parameters and Resource Objects via OData... 41 5.8 Calling an Orchestrator Runbook from SMA... 43 5.9 Calling an SMA Runbook from Orchestrator... 46 5.10 Importing and Exporting Runbooks... 48 5.10.1 Exporting a Runbook... 49 5.11 Converting Orchestrator to SMA Runbooks... 50 5.12 Extending SMA with Additional Modules... 52 5.13 Adding/Importing Modules... 52 5.13.1 Extending SMA with custom Modules... 54 5.13.2 Portable Modules... 54 5.14 Checkpoints... 56 5.15 Resuming a Workflow... 56 5.16 Debugging and Logging... 57 5.16.1 Best Practice for Logging in SMA Runbooks... 59 5.16.2 Logging behavior inside an InlineScript... 59 5.16.3 Getting Output and Debug Information via PowerShell... 59 5.16.4 Event Log... 60 6. FAQ... 61 7. Useful Resources and Links... 63 8. Appendix A (Limitations of PowerShell Workflows)... 64 9. Appendix B (sample code)... 66 9.1 SMA Runbook (Cleanup-VMCheckpoint)... 66 9.2 SMA (Get Job Parameter from linked SPF Event)... 67 9.3 SMA (Call SC Orchestrator Runbook from SMA)... 67

Disclaimer... 68 Feedback... 68

1. Introduction This whitepaper covers basic architecture, installation, and use of System Center 2012 R2 Service Management Automation (SMA). Basic examples how to create, link, and integrate Runbooks from SMA to Orchestrator and vice versa are included as well. A FAQ section includes the most common questions and answers around SMA. Sample Runbook code used and referred in this whitepaper is located in the appendix section. 1.1 What can you expect from reading this paper After reading this whitepaper, you will be familiar with the following topics: Concepts and architecture of System Center 2012 R2 Service Management Automation Prerequisites to deploy SMA in your organization Installing and configuring the various SMA components Basics on PowerShell Workflow scripting Creating and authoring SMA Runbooks Linking Runbooks to SPF/VM Cloud action events Interoperability with System Center Orchestrator Runbooks Converting Orchestrator Runbooks to SMA Scaling an SMA infrastructure Extending SMA with custom modules Additionally you ll find a collection of useful links on SMA, PowerShell Workflow concepts and Windows Azure Pack. I hope this document can help you with a jump start on Service Management Automation. Happy reading! Cheers Michael

2. SMA Architecture Service Management Automation integrates with Windows Azure Pack and the Service Provider Foundation. The primary target using SMA is to automate your private/hybrid cloud services such as Virtual Machine, Clouds, Web Sites, SQL and MySQL database deployments. However, as SMA relies on PowerShell, everything you can do inside a PowerShell Workflow can be automated Figure 1: SMA Architecture (source: Microsoft)

The following image illustrates how SMA integrates with other System Center components and 3 rd party systems. The pluggable module architecture of SMA allows integration of almost any management infrastructure which ships with PowerShell modules. Figure 2: Component Integration (source: Microsoft)

2.1 Components of SMA Framework 2.1.1 SMA Database This is the so called heart of SMA, hosting Runbooks, their state, and configuration. It stores schedules and mappings as well. By default, the name of the database object is SMA. 2.1.2 OData REST Web Service As other components of Windows Azure Pack, SMA uses a REST-based API and web service for Management Portal and orchestration. Its primary job is to display information on the SMA Management Portal and to submit requests to the SMA database. The database returns Runbook and job states back to the web service. 2.1.3 Service Management Portal The SOAP-based portal is used to author and manage Runbooks, assets and schedules. It also displays job results as well as some statistical data. 2.1.4 Runbook Worker As the name says, the Runbook Worker start or execute SMA Runbooks. As you can have multiple of them, the solution scales very well. A Runbook Worker picks up queued jobs from the SMA database, executes them and reports back to the database. Having multiple Runbook Worker by default has a kind of randomization effect. The workers pick up tasks randomly to spread the load and jobs to be executed. Quote from MSFT: Service Management Automation uses static queue partitions to load balance work among Runbook Workers. Because of this, queues are never changed on the fly to respond to worker additions or workers going offline. If a worker in the current deployment goes offline, it is still allocated jobs but cannot process them. Similarly, if a new worker is started but is not specifically defined in the current Runbook Worker deployment, it does not pick up any jobs because it is not being assigned any jobs 2.1.5 Runbooks SMA Runbooks are similar to Orchestrator Runbooks regarding their scope with one big difference. SMA Runbooks are PowerShell Workflows, exclusively. This basically means that there is no graphical user interface to design Runbooks at this time. 2.1.6 Jobs When a Runbook is executed, a new instance, so called Job is started. The details output and results can be found in the Job Log for each Runbook.

3. Installing SMA 3.1 Prerequisites Service Management Automation is a solution that ships with System Center Orchestrator 2012 R2 and requires SPF (Service Provider Foundation) and WAP (Windows Azure Pack) as a prerequisite. As SMA relies on Windows Azure Pack Admin Portal Web Sites components and APIs, installing Windows Azure Pack in a basic configuration is NOT a mandatory step to take before installing SMA. However, to manage and operate Runbooks from a web gui, WAP Admin components have to be deployed. For more information on how to deploy Windows Azure Pack see the TechNet documentation. http://technet.microsoft.com/en-us/library/dn296432.aspx See here on how to install SPF as part of the integration between WAP, SMA, and your VM Cloud resources: http://technet.microsoft.com/en-us/library/jj642895.aspx It s a best practice to deploy Windows Azure Pack as a distributed architecture. Therefore the SMA web service as well as the Runbook Worker should be deployed to dedicated servers. The SMA database can be deployed to the same SQL instance as WAP and SPF. It is a best practice to put the Runbook Worker service on the same hosts as the SMA Web service. Other Prerequisites Server 2012 R2 IIS 7.5 or higher IIS Security Basic Authentication IIS Security Windows Authentication IIS URL Authorization IIS Application Development ASP.NET 4.5.NET Features 4.5 WCF Services HTTP Activation Install-WindowsFeature Web-Server,Web-Common-Http,Web-Basic-Auth,Web-Windows- Auth,Web-URL-Auth,Web-Net-Ext45,Web-Asp-Net45,Web-Mgmt-Console,Web-Mgmt- Console,NET-WCF-HTTP-Activation45

3.2 Installing the SMA Web Service First we kick off the SMA Web Service Setup. You can find the binaries on System Center Orchestrator 2012 R2 media. Note: At this point we assume we have an existing Windows Azure Pack deployment in place. As an absolute minimum, you need the Admin API, Admin Portal and the Tenand API of WAP to manage SMA via web browser. SMA Portal and API installation

Choose a local or remote SQL instance to host the SMA database

It s best practice to use a domain user for the App pool service account Although you can use a selfsigned certificate for PoC environments, it s a best practice to use a valid webserver SSL certificate for production environments

After choosing CEIP and Update Options, click INSTALL to kick off the installation

3.3 Installing the SMA Runbook Worker This will install the Automation Runbook Worker Service. As a best practice, Runbook Workers are deployed to dedicated machines together with the SMA Web Service.

See chapter 6 for questions about SMA licensing. Choose the SQL instance hosting the SMA database deployed before.

Choose the same user account as for the SMA web service After choosing CEIP and Update Options, click INSTALL to kick off the installation

3.4 Installing the SMA PowerShell Module Installing the SMA PowerShell module to a Runbook Worker or any other machine can be useful because SMA comes with cmdlets to automate the SMA deployment configuration. Additionally you can import and export SMA Runbooks, gather Runbook and job information, and start/stop/suspend SMA Runbooks using the cmdlets. There s nothing more than choosing to install the Module. A small MSI Setup will kick in and deploy the SMA PowerShell Module: Microsoft.SystemCenter.Service ManagementAutomation

3.5 Registering SMA Endpoint Open Azure Pack Admin Portal: https://wapadminsite.demonet.c h:30091 Switch to Automation Pane Choose to register a new SMA Endpoint. This links the SMA web service and API to Azure Pack. Provide the SMA web service endpoint and the credentials to register the endpoint. The account used here must be a member of the smaadmingroup local group on the SMA web service machine.

4. Scaling Out a SMA Deployment The different components can be made highly available. Especially the Runbook Workers, taking most of the SMA load, can be scaled out to as many simultaneous workers as needed. The API and Portal components would usually be made highly available and load-balanced using NLB or any supported 3 rd -party load balancer. The next few chapters focus on how to scale out the Runbook Workers 4.1 Installing Additional Runbook Workers To distribute the Runbook workloads, additional Runbook Workers need to be deployed. The secondary installation procedure is exactly the same as for the primary worker. After the installation you have to reconfigure the SMA environment to recognize the additional worker(s). http://technet.microsoft.com/en-us/library/dn530618(v=sc.20).aspx Note: If you reference external files, scripts, log files or modules in your Runbooks, make sure they exist on each of the Runbook Workers or use InlineScript blocks to connect to remote machines with the appropriate modules and consoles installed. 4.2 Stop All Runbook Worker Services Start a PowerShell console with elevated rights. First, let s configure the drain time, meaning the time we give the existing Runbook Worker to finish running jobs. After hitting the drain time value, they won t pick up new jobs until restarted. Set-SmaAdminConfiguration -DrainTimeInSeconds 300 We ll wait here for 5 minutes and get ourselves a coffee. Note: The maximum drain time allowed by the workers is 20 minutes While we wait for the drain time to kick in, we can gather some nice info about SMA Jobs. To show all running jobs you can use the following cmdlet: Get-SmaJob -WebServiceEndpoint https://sma.demonet.ch where {$_.JobStatus -eq 'Running'} To display an underlying Runbook behind a job, note the job ID and pass it to the Get-SmaRunbook cmdlet: Get-SmaRunbook -WebServiceEndpoint https://sma.demonet.ch -ID "fb47c86e-1bd9-4b78-b623-175b8d62f5f4" Then we finally stop the Runbook service on all current configured workers: Get-SmaRunbookWorkerDeployment -WebServiceEndpoint https://sma.demonet.ch Foreach-Object {Invoke-Command $_.Computername {Stop-Service rbsvc}}

4.2.1 Add a New Runbook Worker Deployment to SMA You have to be careful here. As the New-SmaRunbookWorkerDeployment cmdlet overwrites the Runbook servers registered to SMA with the new values. Therefore we create an array of the existing servers and add our new Runbook server to the array before calling the cmdlet. $newworkerhost = "SRVRBWRK02" $webservice = "https://sma.demonet.ch" $workers = @((Get-SmaRunbookWorkerDeployment -WebServiceEndpoint $webservice).computername) $workers += $newworkerhost New-SmaRunbookWorkerDeployment -WebServiceEndpoint $webservice -ComputerName $workers After this we can review the currently configured Runbook workers in our SMA deployment. Get-SmaRunbookWorkerDeployment -WebServiceEndpoint https://sma.demonet.ch This should now show all your Runbook Worke Runbook Worker. 4.2.2 Start the Service on All Runbook Workers In this last step, we are going to start or restart the rbsvc (Runbook Service) on every deployed Runbook Worker: Get-SmaRunbookWorkerDeployment -WebServiceEndpoint https://sma.demonet.ch ForEach-Object { Set-Service rbsvc -ComputerName $_.Computername Status running }

5. Creating and Using Runbooks 5.1 Modules and Consoles 5.1.1 Local Execution As you already know, Runbooks are executed by one or more Runbook Worker. So be sure you install all required management consoles, PowerShell modules and other files you need to reference inside your Runbooks on all of the Runbook Workers. As an example, if you want to automate SCVMM, SCCM, SCOM or any other solution which is delivered with appropriate PowerShell modules, you have to install the management components on each of the Runbook Workers. If we take SCVMM as an example here, you have to deploy the SCVMM console to each Worker. 5.1.2 Remote Execution (PowerShell Remoting) If you are using remote PowerShell sessions, be sure to use persistent sessions for multiple commands to the same remote machine. Also ensure that the required modules are present on the remote machine. Persistent PowerShell sessions have huge advantages over non-persistent aka adhoc sessions. Persistent PowerShell sessions can be referenced for multiple sequential commands or script blocks being executed remotely on the target machines while leveraging a single WinRM connection. Additionally they are fault tolerant against network outages during execution. A persistent PowerShell remote session tries to reconnect itself to a disconnected endpoint while the remote jobs are continued to be executed on the remote machine. If you are unfamiliar with PowerShell Remoting and persistent sessions refer to these link: http://powershell.com/cs/media/p/7257.aspx Note: As a best practice use PSComputerName and PSCredential parameters when using inline scripts instead of multiple Invoke-Command calls.

5.2 Creating Runbooks A SMA Runbook is basically a PowerShell Workflow. Therefore almost anything within a regular PowerShell script can be used in SMA Runbooks. However, keep in mind that PowerShell Workflows are slightly different to regular PowerShell scripts. PowerShell Workflows are translated on execution time into a Windows Workflow Foundation compatible code. The WWF then executes the compiled PowerShell code. Workflows have some limitations and syntactical differences to regular scripts. Before we look at how to create a new Runbook inside SMA, there is some need to know about PowerShell Workflows. 5.2.1 Anatomy of a PowerShell Workflow Windows PowerShell Workflows are designed for scenarios where these attributes are required: Long-running activities Repeatable activities Frequently executed activities Running activities in parallel across one or more machines Interruptible activities that can be stopped and restarted, which includes surviving a reboot of the system against which the workflow is executing Here is an example of a PowerShell Workflow executed on a SCVMM remotely, cleaning up VM checkpoints, where the creation date is older than a defined amount of days. I ll refer to this example a few times as we look more under the hood of the code below. Additionally you can find all the sample code under appendix B (chapter 9). Some side notes on the example here: In this example I m using a PowerShell remote session towards VMM, therefore I had to include the code into an InlineScript block. To use the variables and parameters defined on the root scope of the workflow within the InlineScript block, I had to refer to them via the Using scope modifier. While workflows have some additional capabilities they also have some restrictions. Refer to chapter 8 for more information about these restrictions.

5.2.2 Creating the Runbook But for now, we ll leave the PowerShell Workflow theory and dig into the SMA Portal to create the first SMA Runbook. As an example, we ll create a Runbook to cleanup VM checkpoints (delete or commit them if older than a defined amount of days). Open the Azure Pack Admin Portal: https://wapadminsite.demonet.c h:30091 Switch to Automation Pane Give the Runbook a nice name. You should consider to use the official PowerShell naming convention: Use this as an Example: Verb-SingularNoun Allowed are: letters, numbers, dashes, underscores. Has to begin with a letter. E.g. cleanup-vmcheckpoint Note: It s a best practice to use PowerShell approved nouns and verbs. Also keep the in mind that the Runbook name can t be different to the workflow name inside the Runbook code! So choose your names wisely. We will leave the TAGS empty for now. We ll have a deeper look at the TAGS later in this paper

Check if the Runbook has been created successfully Click on the created Runbook to edit it. Choose AUTHOR / DRAFT Place your code here. The editor provides basic syntax highlighting and autocompletion. However, for complex workflows you might use an external editor like PowerShell ISE, which is part of Windows, or any other professional development environment. Refer to Chapter 5.5 for details on how to use external editors.

1. Choose SAVE to keep the changes. 1 3 2 2. Test the Runbook. The Status should switch from QUEUED to STARTING within a few seconds as it gets picked up by a Runbook Worker. The Output pane shows errors and messages you pump out using the Write-Output cmdlet. Refer to chapter 5.14 for information about debugging and logging 3. Choose Publish to make the Runbook available. While you continue to edit and author your Runbook, you always have two versions of it available, the published one and the draft one. Testing the Runbook always targets the draft version. 5.2.3 Note on Testing Runbook System Center Orchestrator ships with a tool called Runbook Tester, where you can test your Runbook and step through the Runbook actions. These tests are always executed as the user account which started the Runbook Designer tool. SMA does not have a dedicated testing tool yet, so even if you choose to invoke a test run of your Runbook, it ll be picked up by a Runbook Worker and executed as the Runbook Service user account.

5.3 Creating Schedules Choose a Runbook and create a new schedule. You can also link an existing schedule to a Runbook. For our example we want the Runbook to be executed daily. Name the schedule. Spaces are allowed, but consider to use underscores instead of dashes, to keep you free from escaping characters when modifying a schedule using PowerShell

This screen appears if the scheduled Runbook has parameters. Parameters defined as mandatory have to be submitted on execution, such declared as optional can be left empty. So you have to define the parameter values within the schedule object. You can of course create multiple schedules for the same Runbook to have it executed with different parameters. So, for example, we want to delete checkpoint from productive VMs after 2 days, while we keep checkpoints for dev and test VMs for up to 30 days.

5.4 Adding Connections, Credentials and other Assets Global assets inside SMA are: Runbooks Connections (to other systems) Credentials Schedules Variables These type of objects share more or less the same idea of configurations and variables in Orchestrator. They can be used to keep your workflow code clean and free of hardcoded values. On the Automation pane you can add other types of SMA objects such as connections for SCVMM, SC Configuration Manager, OpsManager, etc., as well as credentials and variables to be reference inside Runbooks. In the example above I used a Connection Object for SCVMM to keep the workflow code clean of server names and hardcoded credentials.

5.4.1 Adding Connections An Automation Connection contains the information required to connect to an external system. External means outside SMA. You can also link a SMA instance with another by creating an SMA connection object. Connection objects typically store server/port connection information as well as credentials to connect to external systems. By using Connection Objects we can keep out Runbook-code clean from hardcoded server names and credentials. If a server name or credential changes, we just have to edit the Connection Object instead of touching all affected Runbooks. (Automation / Assets / Add Settings) For our demo script we need a VMM Connection Object.

The Server, Username and Password to connect to VMM. Note: There are two ways connecting to VMM in this case. Either you install the VMM console on each Runbook Worker, or you create a remote PowerShell session within an InlineScript block in your Runbook. Or via PowerShell $websvc = "https://sma.demonet.ch"new-smaconnection -Name VMM_LAB - ConnectionTypeName VirtualMachineManager ConnectionFieldValues @{"ComputerName"="vmm.demonet.ch";"UserName"="DEMONET\sma_vmm_admin";"Password"="So mesecurepassword"} -Description "Connection to Virtual Machine Manager" - WebServiceEndpoint $websvc 5.4.2 Referencing a Connection in a Runbook An Automation Connection contains the information required to connect to an external system. External means outside SMA. You can also link a SMA instance with another by creating an SMA connection object. Connection objects typically store server / port connection information as well as credentials to connect to external systems. $con = Get-AutomationConnection Name VMM_LAB $securepw = ConvertTo-SecureString AsPlainText String $con.password Force $credential = New-Object TypeName System.Management.Automation.PSCredential ArgumentList $con.username, $securepw Get-SCVMMServer Computername $con.computername Credential $credential Unfortunately the connection object does not store credentials as a PSCredential object. Therefore we have to generate one first. We need a PSCredential as the VMM connection will run within an InlineScript block within the Runbook. Note: The activity Get- AutomationConnection along with all other activities beginning with Get-Automation* do only exist inside an SMA Runbook To use them with an external IDE, refer to chapter: 5.5 In this example we are going to connect a VMM instance

And here is a code extract from our Runbook example: Note: Explained again here. Using InlineScript activity saves us from deploying SCVMM Console binaries and modules to the Runbook Workers, as we are remoting directly to SCVMM.

5.4.3 Adding Credential Objects Credential objects are mostly used when connecting to remote computers within a Runbook and an appropriate Connection Type for a Connection Object is not available. Add a new credential object (Automation / Assets / Add Settings) Or via PowerShell $websvc = "https://sma.demonet.ch" $cred = Get-Credential Set-SmaCredential Name "Host_Management_Account" -WebServiceEndpoint $websvc Value $creds 5.4.4 Referencing a Credential Object in a Runbook $cred = Get-AutomationPSCredential Name "Host_Management_Account This is pretty straightforward by issuing the Get-SmaCredential

5.4.5 Adding Variables Add a new SMA variable (Automation / Assets / Add Settings) and define the data type The default setting is not to encrypt the variable value. You might want to choose to encrypt it, if it contains sensitive information. But doing so makes your Runbooks less portable as the information can t be exported. Or via PowerShell $websvc = "https://sma.demonet.ch" $value = "$ENV:SYSTEMDRIVE\Logs\SMA" Set-SmaVariable -Name "GlobalLogPath" -Value $value -Description "a variable we can use in multiple Runbooks" -WebServiceEndpoint $websvc

5.4.6 Referencing a SMA Variable in a Runbook $logpath = Get-AutomationVariable -Name "GlobalLogPath" This goes as well pretty straightforward by issuing Get-AutomationVariable inside a Runbook 5.5 Using External Development Environments As already mentioned, the built-in web editor to create and edit SMA workflows is not really powerful. You might want to use PowerShell_ISE which is part of Windows, SAPIEN PowerShell Studio or any other PowerShell aware development environment. To make external code transportable you have to import some PowerShell modules, which are available to the SMA editor, but not to external environments. Given an example that you re using Connection and Credential assets as described in the chapters before, by default the required cmdlets are not available outside the SMA web editor. Thankfully, we can download a module called EmulatedAutomationActivities from TechNet, so we get those cmdlets back in our ISE external editor. See this link for download and how to install the module: http://gallery.technet.microsoft.com/service-management-d4edfbf4 5.6 Nesting Runbooks/Publishing Data As we all know, Orchestrator has a data bus to which you can publish data being available to further actions or Runbooks. SMA doesn t have this kind of store for passing output data to the next Runbook. But hey, as it s all PowerShell Workflow-based, of course we can nest Runbooks, meaning call another Runbook inside a Runbook and pass output from the first to subsequent Runbooks using parameters. The big thing here is we have to respect the variable scopes in PowerShell Workflows. Especially when using remoting or InlineScript activity, there are some limitations. Variables within InlineScript blocks are not visible to the workflow. Also remember that using scope modifier is not available to pass local variables to a remote PowerShell session within a workflow. Using scope modifier is reserved to pass variables from the parent workflow scope to an InlineScript activity. For more information on how to pass local variables into a remote session inside an InlineScript block refer to chapter 8

Workflow Get-Parameter { param( [Parameter(Mandatory=$true)] [DateTime]$DateFromOtherWF ) Write-Output $DateFromOtherWF } Workflow Test-Parameter { $result = InlineScript } { $date = Get-Date Return $date } Get-Parameter -DateFromOtherWF $result In the example above we return a variable explicitly back from the InlineScript block to the parent workflow. Afterwards we call another workflow, where we use the returned value as a parameter. Chris Sanders (MSFT) has a great blog post about nested Runbooks: http://blogs.technet.com/b/orchestrator/archive/2014/01/10/sma-capabilities-in-depth- Runbook-input-output-and-nested-Runbooks.aspx

5.7 Linking Runbooks to VM Cloud Action Events SMA Runbooks can be linked to VM Cloud events. The following table represents the different events, which can be used within a VM Cloud to link Runbooks with. As an example, you might want to kick off a workflow, each time a virtual machine gets stopped, created or deleted, or maybe perform some additional network script actions each time a tenant creates a new network. The following table illustrates the different VM Cloud action-triggered events, where you can link SMA Runbooks being invoked upon one or more of those events. Refer to the following TechNet article for more information: http://technet.microsoft.com/en-us/library/dn457800.aspx

Before continuing you have to register SMA with your VM Cloud within WAP Portal. See here for a how to: http://technet.microsoft.com/en-us/library/dn457801.aspx To link an existing Runbook to a VM Cloud action/event, the Runbook has first to be tagged. The corresponding tag is SPF SPF stands for the Service Provider Foundation which is, as you already know, part of the Windows Azure Pack deployment.

In this example we choose the event creation of a standalone VM and link our Runbook to the event Note! The host running SPF must trust the SSL certificate bound to the SMA web service endpoint to fire up SMA Runbook based on SPF-triggered events.

5.7.1 Getting Runbook Parameters and Resource Objects What if we want to get the job parameters from a Runbook which has been invoked by SPF, because we linked a VM Cloud action to a SMA Runbook? Linking a Runbook for an SPF action invokes the Runbook with tons of parameters and information about the original object the Runbook was targeted. But how to access those parameters? SMA jobs have a variable called $PSPrivateMetaData. ( Special thanks to Jim Britt, MSFT for this hint). Usually this parameter includes private job data like JobID and JobContextID. You can easily output the content of this variable. Here s an example: workflow Get-VMCreatedInfo { #Get Job Parameters and object information $objectname = $PSPrivateMetaData.Name $sourceobj = $PSPrivateMetaData.resourceObject $action = $PSPrivateMetaData.Action $params = $PSPrivateMetaData.params $VMMJobID = $PSPrivateMetaData.VMMJobId #Format the parameters and create a hash table with keys and values because we get it as a string first #We use an InlineScript activity because (add) method to hash table is not supported natively in workflows $params = $params -split (',') -replace ('"','') -replace ('{','') -replace ('}','') $sourceobj = $sourceobj -split (',') -replace ('"','') -replace ('{','') - replace ('}','') $params = InlineScript { $paramhashtable = @{} foreach ($p in $using:params) { $key = ($p -split (':'))[0] $value = ($p -split (':'))[1] $paramhashtable.add($key,$value) } } #Return the hash table back to the workflow return $paramhashtable } Write-Output $params

5.7.2 Getting Runbook Parameters and Resource Objects via OData Thanks to the OData model of SMA web service, you can also browse the SMA s OData structure using a browser or via Invoke-RestMethod cmdlet. The good news, you can browse the OData model of SMA via web browser: https://sma.demonet.ch:9090/0 0000000-0000-0000-0000- 000000000000 To view the structure in XML format you have to disable the Feed Reading View in IE s content tab To view a specific object type browse to it using the following schema: https://sma.demonet.ch:9090/0 0000000-0000-0000-0000- 000000000000/<objecttype> But of course you don t want to use the browser to gather information about Runbooks, jobs, or their parameters. PowerShell has an Invoke-RestMethod cmdlet which lets you browser an ODatabased REST web service or invoke REST requests as GET/PUT etc. The following sample script gathers all running jobs of a specific SMA Runbook, searches for referenced job execution context and displays job parameters. Additionally it gets the referenced VMM Job ID as well as the GUID of

the virtual machine being created on VMM. Refer to chapter 5.7 for more information on how to link SMA Runbooks based on VM Cloud action events.

5.8 Calling an Orchestrator Runbook from SMA This can be useful whether you want to integrate with an existing Orchestrator deployment or just want to leave some logic in Orchestrator. Given the example that you have an SC Orchestrator Runbook which performs action on VMs when they are created, updated, or deleted. SMA allows you to simply trigger a SMA Runbook based on VM Cloud action events, as seen earlier in this whitepaper. So you could use SMA to trigger and fire up a small Runbook in SMA which then kicks of the Orchestrator Runbook. 2 First we need to define a connection object for the Orchestrator Web Service Connection. 1 Open the Azure Pack Admin Portal and navigate to Automation. Choose the Assets tab and click on Add Setting 3 Choose Add Connection

Choose an Orchestrator Connection Object Name it and optionally add a description Add the Server name, Port, Username and Password to connect. Remember that this user has to be granted to execute Runbooks on the Orchestrator server.

Create a new SMA Runbook and edit it. http://sco.demonet.ch:81/orchestrator2012/orchestrator.sv c/runbooks To get the Runbook ID from your Orchestrator you can browse the Orchestrator Web Service

5.9 Calling an SMA Runbook from Orchestrator For this task we need a few things as well A Runbook (workflow) in SMA with or without parameters depending on the requirements SMA PowerShell module Installed from System Center Orchestrator R2 media on all Orchestrator Runbook servers An Orchestrator Runbook with a PowerShell/Script execution activity to kick off the SMA Runbook (Optional but recommended: Orchestrator Integration Pack for PowerShell Script Execution V1.2 http://gallery.technet.microsoft.com/orchestrator-integration-438f9ece) Install the SMA PowerShell module on all System Center Orchestrator Runbook servers

Create a SMA Runbook. In this example we use again the Runbook to delete any Hyper-V VM checkpoints (snapshot) where its creation date is older than a defined amount of dates. The parameter $gracedays has to be submitted. We will submit it when calling the Runbook from an Orchestrator activity. Save and publish the Runbook (Get-SmaRunbook -Name vmm-cleanup-vmcheckpoints - WebServiceEndpoint https://sma.demonet.ch).runbookid Get the Runbook ID using SMA PowerShell module. Node the ID, run this command in a PowerShell console, no evaluated rights are required for this command We now switch to the SC Orchestrator Console. Navigate to Global Settings Variables to define the SMA web service endpoint. Now this variable can be easily used in all Orchestrator Runbooks without knowing the full URL Create Execute PowerShell Script Runbook Activity in Orchestrator. The Parameter value is of type hash table. So if your SMA Runbook requires more than one parameter, define them by using a simple hash table e.g. -Parameters @{"parameter1"=abc;"parameter2 =999}

5.10 Importing and Exporting Runbooks While the default SMA PowerShell cmdlets allow to import Runbooks, they do not allow exporting them. Here s where SMART comes into play. SMA Runbook Toolkit is a PowerShell-based solution which helps with these tasks. Beside importing and exporting Runbooks together with their properties, SMART also comes with a conversion helper for Orchestrator Runbooks. See chapter 5.11 for more information on how to convert an Orchestrator Runbook to SMA. You can download the SMART solution under the following link: Automation Service Management Automation SMA Runbook Toolkit Spotlight SMART for Runbook Import and Export After downloading the solution you have to deploy it to your SMA instance. Make sure you unblock all PS1 files after download. The installation script returns success on completion: After this you should see 4 new Runbooks inside SMA Portal

5.10.1 Exporting a Runbook While the default SMA PowerShell cmdlets allow to import Runbooks, they do not allow exporting $WebServiceEndpoint = "https://sma.demonet.ch" $ExportDirectory = "C:\temp\RunbookExport" $RunbookName = "VMM-add-VMRolePermission" # Find the RunbookRunbook named Runbook-Example $Runbook = Get-SmaRunbook -WebServiceEndpoint $WebServiceEndpoint Where-Object -Match -Property RunbookName "$RunbookName" E:\SMA\Tools\SMART\Export-SMARunbooktoXML.ps1 - RunbookName $Runbook.RunbookName -ExportDirectory $ExportDirectory ` -WebServiceEndpoint $WebServiceEndpoint EnableScriptOutput $True ` -ExportPS1 $True ExportVars $False ExportCreds $False ExportSchedules $False ` -ExportAssets $False This little example exports a specific Runbook to a local folder. The Runbook Metadata and the plain PowerShell code are exported separately For more examples and information about SMART refer to the original blog post: http://blogs.technet.com/b/privatecloud/archive/2013/10/23/automation-service-managementautomation-sma-runbook-toolkit-spotlight-smart-for-runbook-import-and-export.aspx

5.11 Converting Orchestrator to SMA Runbooks SMART has another great tool helping to perform the transition from Orchestrator to SMA, the SMART Runbook Conversion Helper (SMART RCH). For details and the download link click here: Automation Service Management Automation SMA Runbook Toolkit Spotlight SMART Runbook Conversion Helper Keep in mind that the SMART solution is not officially supported, and you use it at your own risk. Once downloaded, unblock the file and run it with PowerShell. The tool tries to connect to the local server trying to find a local SQL instance with Orchestrator database deployed. You can change to a remote host of course by supplying the required parameters..\sma-runbookconversionhelper.ps1 -DBServer sco.demonet.ch -DBPort 1433 -DBName Orchestrator The tool lists the Orchestrator Runbook Hierarchy From here you can select a Runbook to convert by selecting the specific Runbook and clicking on Export.

By default the Runbook will be converted to a PowerShell Workflow and displayed in a PowerShell_ISE window. The next steps would be to check the code, clean it up and import the Runbook using SMART into SMA, and here s your first converted Orchestrator Runbook in SMA. For more detailed information and examples refer to the original blog post: http://blogs.technet.com/b/privatecloud/archive/2013/11/25/automation-service-managementautomation-sma-runbook-toolkit-spotlight-smart-runbook-conversion-helper.aspx

5.12 Extending SMA with Additional Modules Although SMA comes with a huge set of included PowerShell modules, you might like to import additional modules into the framework. In fact, modules replace Orchestrator Integration Packs. Creating and importing custom modules has a huge advantage. Imported modules are distributed automatically among all Runbook Workers, additionally their cmdlets are transparent to the web editor auto-completion feature. 5.13 Adding/Importing Modules In this example, we re going to import the Active Directory (ADDS) PowerShell module into SMA. Adding or importing a module into the SMA portal makes the exported cmdlets available to the SMA portal. Note! Modules you d like to import, have to be zipped and have all referenced assemblies and other external code to fit a single folder. If the external module can t fit a single folder for all references, consider to create a so called Portable Module. Refer to next chapter for more information about Portable Modules. Note! There is no need to manually distribute modules among the Runbook Workers, as soon as they are imported into SMA. Install-WindowsFeature RSAT-AD-PowerShell C:\Windows\System32\WindowsPowerShell\v1.0\Modules\A ctivedirectory First you have to install the Active Directory RSAT PowerShell module on all Runbook Workers Create a ZIP file containing the module folder. Copy the zipped module to the SMA web module directory

Switch to the SMA Admin Portal and go to Automation / Assets Choose Import Module Browse and select the ZIP file we created before Import was successful

5.13.1 Extending SMA with custom Modules SMA ships with a default set of Powershell modules already integrated into SMA. Additional modules such as your own custom modules can be imported into SMA. Importing custom modules makes them available to all Runbook Workers, without the need to distribute them using other deployment technologies. Additionally, the activities (cmdlets) are available to the SMA web editor. Joe Levy from MSFT has a great blog post on how to create and import your own custom modules into SMA. http://blogs.technet.com/b/orchestrator/archive/2014/06/12/authoring-integration-modules-forsma.aspx 5.13.2 Portable Modules Portable modules can be compared to DNS Stub Zones. As DNS Stub Zones, portable modules have all authoritative information inside the module code, but they do not contain any functionality. A portable module contains all functions and cmdlets of its original parent module, but not the plain code, just a stub link to it. The execution is "proxied" to the original module. But hey! What are they used for? As mentioned earlier, SMA requires imported modules to be packaged within a single zipped folder. System Center Components like SCVMM and OpsMgr, require additional DLL references outside of the plain module, so they can t be directly imported into SMA. Therefore they are used as portable modules. Trying to call a cmdlet from a portable module directly in the SMA Runbook will result in the following error: <your worker> does not have the Azure module installed. Either this cmdlet must be called from an inlinescript against a machine with the Azure module installed, or you must install the Azure module on <your worker> (<your worker> does not have the Azure module installed. Either this cmdlet must be called from an inlinescript against a machine with the Azure module installed, or you must install the Azure module on <your worker>) So what does us this error message want to tell? Exactly what it seems. As the cmdlets exposed in the portable module are currently not available on the Runbook Worker, which executed the Runbook. We have either the possibility to use a InlineScript activity and call the cmdlets on a remote machine, or we install the required native modules or console components on all the Runbook Workers.

5.14 Checkpoints As SMA Runbooks are PowerShell Workflows, they support Checkpoints. Checkpoints within Workflows are used to mark the progress of a workflow. So if at any point, the workflow gets suspended because of an error or exception, it can be resumed from the last successful checkpoint. By Default, Workflows get a checkpoint at launch and another one at the end of the workflow. Additional checkpoints can be set as follows: After each successful executed cmdlet (valid for all cmdlets supporting the PSPersist parameter Workflow test-wfcheckpoint { get-process cmd -PSPersist $true }..or manually at any time or if the cmdlet does not support PSPersist parameter Workflow test-wfcheckpoint { Param( [Parameter(Mandatory=$true)] [STRING]$PoolName ) } Get-StoragePool $PoolName Checkpoint-Workflow Note! InlineScripts do not support Workflow checkpoints. A checkpoint has to be set either before or after the execution of an inlinescript. 5.15 Resuming a Workflow To resume a Workflow just use the SMA Web Gui..or the SMA Powershell cmdlet Resume-SMAJob

5.16 Debugging and Logging Somewhere, sometime things go wrong. So logging and debugging SMA Runbooks can become quite important. That being said, it might be a valuable way to develop outside the SMA portal, complete testing and import the Runbook into SMA. But sometimes this is just not possible. You can t link an external Runbook code to a VM Cloud action triggered event. SMA has three logging and debugging options for each Runbook. By default all three options are disabled, meaning set to False. You can of course set these options via PowerShell: $websvc = "https://sma.demonet.ch" Set-SmaRunbookConfiguration RunbookName "cleanup-vmcheckpoint" LogDebug $true LogVerbose $true LogProgress $true After Runbook execution you can find the logging records on the history tab of the referenced job context within the SMA portal.

Before enable all logging (Only default output available where you explicitly used Write-Output) And after enabling all logging options (Even verbose cmdlet output)

5.16.1 Best Practice for Logging in SMA Runbooks As a best practice you should use Write-Verbose as your default logging method inside the SMA Runbooks. To have verbose messages appearing in draft version of Runbooks while testing, use the parameter $VerbosePreference = "Continue" To have verbose messages appearing in published version, activate the "Log verbose records" option on the Runbook. Note! You should comment out the $VerbosePreference = "Continue" line in published versions of your Runbooks! 5.16.2 Logging behavior inside an InlineScript To have the above parameter being valid inside an inlinescript, it must be referenced by the using sope modifier. $VerbosePreference = [System.Management.Automation.ActionPreference]$Using:VerbosePreference 5.16.3 Getting Output and Debug Information via PowerShell Job output can of course also be gathered via PowerShell. To do this you have to install the SMA PowerShell module. $websvc = https://sma.demonet.ch $jobs = Get-SmaJob -RunbookId "4501f6a0-4be5-4ccc-b16f-072ddf3e5bd8" - WebServiceEndpoint $websvc foreach ($job in $jobs) {Get-SmaJobOutput -Id "$($job.jobid.guid)" -Stream Any - WebServiceEndpoint $websvc} Here s an example to get the debug stream of a specific job: (Get-SmaJobOutput -Id "22f6d175-f805-4c2e-9de6-e1886b90c89b" -WebServiceEndpoint $websvc -Stream Debug).StreamText

5.16.4 Event Log SMA Runbook activities are logged under Microsoft Service Management Automation Event Log. Or get it more comfortable via PowerShell: Get-WinEvent -ProviderName "Microsoft-ServiceManagementAutomation" where {$_.LevelDisplayName -eq 'Error'}

6. FAQ Q: How does SMA compete with Orchestrator, which solution should I use? A: This hardly depends on your company strategy and on the efforts you may have invested into an existing System Center Orchestrator solution. Microsoft puts a lot of effort in Azure and the onpremises product Windows Azure Pack. While System Center Orchestrator ships with a graphical user interface to create and edit Runbooks, SMA does not yet. If you want to minimize a lock-in to Integration Pack native actions it could be a wise decision, to use PowerShell activities in your Runbooks instead of native ones, so transition to SMA will be smother in the future. If you are not sure why you should have a look at SMA I definitely recommend the following blog post by Christian Booth (MSFT) http://blogs.technet.com/b/systemcenter/archive/2014/01/14/service-management-automationand-sharepoint-mvp.aspx Q: Which permissions should I assign to the Runbook Service Account? A: If serving multi-tenant environments it s a best practice not to assign administrative permissions and use SMA Connection, and Credential assets instead. If deployed in a single-tenant environment you could decide to grant administrative permissions to the SMA Runbook Service Account on managed target systems and applications. However, be aware that while the deployment is growing, you re creating a kind master of disaster account which has at least to be protected with a strong password to be changed frequently. As a best practice, create dedicated connection objects per tenant and product to automate. Q: How can I delegate SMA administrative permissions? A: At the time being, there is role based access model for SMA. Either you are an SMA Admin or you're not. At the end this does not differ a lot from System Center Orchestrator concept. As SMA is v1.0, it will evolve. Q: How can I design and arrange SMA Runbooks, which tool do I have to use? A: At the time being, there is no graphical user interface to design and arrange Runbooks or activities. SMA Runbooks are written in PowerShell as workflow elements. Best authored in PowerShell_ISE and imported to SMA. The built-in editor inside the SMA portal is good for small number of code lines and supports auto-completion and syntax highlighting.

Q: How is SMA licensed? A: As SMA is part of the System Center Orchestrator 2012 R2, it requires either Standard or Datacenter license for System Center 2012 R2. Q: Can SMA be deployed as a standalone product? A: Yes. However, to benefit of a graphical interface (SMA Portal) for editing, invoking gathering log information, Windows Azure Pack Admin Portal components are a prerequisite. Q: Am I limited to use the WAP portal for authoring and execution of SMA Runbooks? A: Absolutely not. Runbooks can be authored outside of the SMA portal as they can be called without touching the portal. Refer to chapter 5 and following on how to create, edit and run SMA Runbooks. Q: Is there any graphical user interface to create and arrange SMA Runbooks? A: No, at the moment this is pure PowerShell work. Refer to chapter 5 and following on how to create, edit and run SMA Runbooks. Q: We spent ages configuring and customizing our Orchestrator Runbooks. If we want to start with SMA, do we have to re-engineer all the existing Runbooks? A: Well the concepts of Orchestrator and SMA Runbooks are completely different. If you ve used a lot of native actions available through integration packs instead of PowerShell scripts, then you might have to invest more time for transition and integration. However, you can start with SMA while keeping the most work intensive Runbooks on Orchestrator. See chapters 5.8 and 5.9 for how to integrate Orchestrator Runbooks with SMA. Q: Why are my Runbooks linked to VM Cloud Automation events not triggered? A: There are two main reasons for that. First, the machine running SPF must trust the SMA web service endpoint certificate. Second, you might have linked the Runbook to an incorrect VM Cloud Automation Event. Refer to the following link for a reference: http://technet.microsoft.com/en-us/library/dn457800.aspx