SCHEDULERS vs. OPERATIONS PROCESS MANAGERS

WHITEPAPER OPERATIONS TAMED. APPLICATIONS ON. SCHEDULERS vs. OPERATIONS PROCESS MANAGERS

In an IT operations or production environment, support staff and administrators are challenged with selecting the right management tools to streamline their job. There is a variety of automation tools associated with job processing and application integration. On the surface, it is possible to confuse these tools. However, once a deeper understanding is developed, it is apparent that they serve different and often complimentary functions. This paper summarizes the various scheduling technologies as they have evolved, and highlights the unique requirements for a robust operations process management platform. Lastly, we review how C2O uniquely satisfies these requirements and delivers staff and time efficiency gains thru complete process lifecycle management. The following is a brief synopsis of the evolution of schedulers larger companies likely have (or will have) some of each generation deployed within their application and operations infrastructure. TRADITIONAL SCHEDULERS (MAINFRAME BATCH JOBS ONLY, PLEASE!) Schedulers have roots in the mainframe world where they are still used today to initiate batch processing jobs. These traditional schedulers perform best at straight-thru processing, where the application processing tasks are relatively straight forward (ex. generate invoices for the day s product shipment activity), are sequential in nature, and typically take place on a single system (mainframe, large multi-cpu server, etc.) where workload can be easily balanced. Typically, these batch jobs are scheduled on a calendar- or time-triggered basis and run to completion without intervention. Because of the high volume of batch processing activity, mainframe administrators depend upon traditional schedulers to start the hundreds or even thousands of jobs they may process in any given day. According to Gartner Research, the three leading job scheduling tools are Control-M (BMC), Tivoli Workload Scheduler or TWS (IBM/Tivoli), and Autosys (CA). While all three are strong in the mainframe environment, they have generally been slow to adapt to the distributed processing / open systems world. Many lack support for Web services as well as J2EE or.net application server platforms, and have only recently introduced relational database repositories. However, most provide integration with major packaged software such as SAP, Oracle, Siebel, etc. for scheduling of batch processing. DISTRIBUTED SCHEDULERS (AUTOMATING TASKS) With the advent of open-system distributed computing, schedulers now needed to coordinate enterprise applications across multiple, heterogeneous platforms dispersed over wide areas. Embedded, basic schedulers (ex. Unix CRONtab) offered platform-specific scheduling however were required on every system in the infrastructure and were therefore difficult to administer. A bevy of new players emerged (Gartner counts more than 50) and a few existing players evolved to address these requirements. Vendor solutions such as UC4, Cronacle (Redwood Software), and newer Java scheduling facilities provide these enhanced distributed abilities. As schedulers matured, the ability to trigger based on more than just time was added; now simple binary events such as a file arrival could trigger a real-time processing to occur. This allowed for rudimentary application integration, albeit still largely constrained to predictable (sequential) task execution, for faster application batch processing. Even with these scheduler advances, applications teams still rely heavily on manual interventions and scripts to perform integration tasks for the applications they support. This is because the complex, distributed nature of the applications, the data-dependencies, and the heretofore illequipped technologies make it difficult to properly model and integrate (and hence automate!) application support processes using time- and event-based scheduling alone there are simply 2

WWW.OPTINUITY.COM too many conditions that those basic approaches can t handle. As a consequence, operations departments are filled with dedicated staff that monitor and manage application processing tasks, develop and execute scripts, and perform remediation steps when needed. Most, if not all of these activities, are candidates for automation or further efficiency improvements and cost savings. OPERATIONS PROCESS MANAGERS (INTEGRATING AND AUTOMATING APPLICATIONS AND PROCESS LIFECYCLE) The process glue activities mentioned above have driven requirements for a newer superscheduler one that replaces manual application integration and support with automated decisioning and action and recent developments have led to solutions that add conditional logic, or the ability to use the data/prior step results themselves, to determine process paths. This intelligence allows for the real time data itself to schedule or initiate further appropriate processes hence the dynamic data and flexible condition logic enables completely automated business applications that respond in real-time without the need for scripts or human intervention. This is done with a set of program objects that are assembled into bespoke procedures that perform all necessary decision logic and automate the response-action flow. An example would be an autorecovery routine that upon seeing an error message, automatically checks contents of a system log, and then based upon the results contained within, takes a certain corrective action (i.e. restart a service) which may vary. Advanced process managers allow scriptless process development and execution with local and global variables and parameter passing- this dramatically simplifies process creation and management. Not having to create and distribute a separate script for every automation instance means faster/safer deployment, far fewer errors, and considerably easier administration. A further distinguishing capability one which the Optinuity C2O platform offers combines the above conditional logic/application integration capability with process development, administration, and execution tools to form a complete process life-cycle management solution. When application support processes are accepted as complex dependent entities with an integral lifecycle, as opposed to a series of disparate functions performed by siloed disciplines, organizational optimization can be achieved resulting in 30-50% staff efficiency increases, lower costs, fewer errors, and improved responsiveness. REQUIREMENTS FOR INTEGRATED OPERATIONS PROCESS MANAGEMENT Streamlining and reducing cost of operations requires a comprehensive end-to-end view of and control over all IT processes that are performed in support of enterprise applications. This is best addressed with an operations process management solution that can perform all stages of the lifecycle including process creation, process administration, and process execution. A robust solution will meet all of the following criteria; Process Creation the ability to quickly design and thoroughly validate processes and then easily incorporate the inevitable changes so that processes stay current and always reflect the real-world production environment. Process Administration the ability to easily (preferably automatically!) document and securely store all processes. Central control for processes and their use is essential to maintain continuity and accountability. Process Execution the ability to support complex conditional logic and data-driven processes for comprehensive application integration and automation. Also, the ability to dynamically alter processes, monitor their execution in real-time, and keep a detailed audit trail of their executions for compliance and organizational learning purposes. 3

The attached Scheduler Analysis table summarizes the distinctive capabilities of the respective technologies and suggests appropriate uses. IMPLICATIONS FOR THE ENTERPRISE Enterprise mission-critical application processing requires resolution of complex business processes and dependencies between different systems, applications and environments. A true solution will not prescribe automation routines, as these can not begin to encompass the uniqueness of each customer s infrastructure and operating procedures. Instead, a viable platform will provide the tools necessary to easily model, validate, deploy, and manage operational processes (hopefully in an integrated fashion!) These are addressed with a robust operations process management platform such as Optinuity C2O. Optinuity C2O, an Operations Process Management platform for IT process automation and lifecycle management, institutes all required aspects of a robust, fully-integrated solution and can be a powerful catalyst for improved IT management and efficiency. For instance, C2O allows operations staff to easily define and validate standardized process workflows and then store them in a central repository where they are available to those authorized to invoke them. In support of COBIT/ITIL best practice guidelines, all processes are self-documenting and C2O facilitates process audit trails so that there is always an accurate current and historical view of operations activities. C2O supports a rich set of application connectors and predefined process building blocks that enable complex procedures to be assembled and validated quickly, speeding distributed application integration and processing. With C2O, operators are presented with a comprehensive business service- or application-wide process view of critical applications and their dependencies, and can easily spot trouble areas, take corrective actions quickly, and resume/recover services fast to enhance service-level predictability. OPTINUITY C2O UNIQUE APPLICATION INTEGRATION FEATURES Optinuity C2O possesses several unique features that make it an ideal platform for enterprise application integration to replace the glue logic traditionally consisting of scripts and repetitive human interventions. These are as follows: 1. Multi-threaded procedures Other products do not support having steps in the workflow that execute in parallel. This is a necessary function for building large, computing intensive, or I/O intensive procedures. Because C2O supports multi-threaded workflows it is able to reduce processing and I/O time dramatically for processes that can take advantage of parallel processing capabilities. 4

WWW.OPTINUITY.COM 2. Parameter passing Because C2O supports parameter passing between process steps and between processes it is possible to define a generic process ONCE and execute it many different ways based on the parameters passed to it. For example let s consider a process that re-cycles a server after performing a number of maintenance and cleanup functions. If the same process is applicable to 100 machines then a single process definition can be used to re-cycle all 100 machines. Other tools require that a separate process instance be defined and distributed for each separate machine and thus become an administrative burden and quality control exposure. 3. Local and global variable controls C2O has an environmental variables dataset capability which encourages process developers to keep machine specific information (e.g. machine names, user IDs, passwords) external from flow definitions and maintain them as secured environment variables. As a result, procedures are much more portable between development, test, and production environments because they don t have environment-specific constants defined in the procedure. Furthermore it is much easier to protect sensitive information since permissions on the environmental variables in the C2O dataset can be masked at the user level. Most products in the scheduling space do not have a robust environment for this kind of variable administration. The C2O environment dataset capability also protects procedures from password changes in the environment. If password administration is managed within the C2O dataset then processes that use those passwords inherit password changes through the variables. The processes themselves do not have to be changed to accommodate a machine name change, a user ID change, or a password change. 4. Process-state preservation and resumption C2O maintains state and process context information for executing workflows. Consequently if a procedure is interrupted for some reason (e.g. a machine crashes) it is possible to resume the process at the current state after the interruption cause has been resolved. Other tools would require that the process be restarted at the beginning which can be devastating if hours of processing have already completed successfully within a maintenance window. 5. Rich object set for script-less process design C2O includes object templates for database commands, file and directory comparisons, and other high level data inspection capabilities that obviate the need to encapsulate these operations within shell scripts. Users can create new complex-function objects as well from existing templates. These, in combination with the Boolean operators and numerous standard connectors (FTP, SNMP, SMTP, SSH, CLI, SOAP), allow end-to-end process construction using the C2O graphical flow design components without requiring new script development (and the administrative challenges associated with script proliferation!). 5

Summary Business today demands 24/7 application availability and real-time responsiveness. To deliver this, applications need to be integrated and the processing handoffs automated to ensure error-free, rapid servicing. IT groups must deliver these services within constrained budgets, and should consider streamlined process management to gain staff and time efficiency and improve operational control. C2O uniquely addresses the requirements for a complete operations process management platform together with process lifecycle management. Whereas in the past, operations processes where performed by siloed specialist using ad-hoc methods and rogue personal scripts, now with C2O, these same processes can be performed with fewer resources in a more methodical manner that leads to lower costs, fewer mistakes, and increased applications availability. For IT management to maximize efficiency and realize significant cost savings, organizations must adopt a comprehensive perspective that includes IT process streamlining and automation. Scheduler Analysis FACTOR TRADITIONAL SCHEDULER DISTRIBUTED SCHEDULER OPERATIONS PROCESS MANAGER Batch / Straight-thru processing jobs (typically on mainframes/single system configs) Complex, distributed applications (typically on heterogeneous platforms), simple stp processes only Calendar/Time and event-based triggers time only Conditional logic with data-driven triggers (required for end-to-end application integration) Script-less process coordination Process lifecycle management (integrated creation, administration, and execution) Example Solutions: TWS (Tivoli), Control-M (BMC), Autosys (CA) UC4, new versions of traditional schedulers Optinuity C20 For more information on Optinuity C2O, please visit our website at www.optinuity.com or contact us directly at 202.292.4920. 6

Scheduler V1.0 May. 2005 2004-2005 Optinuity, Inc. All rights reserved. About Optinuity, Inc. Optinuity C2O provides a complete operations process lifecycle management software platform that enables the coordinated development, validation, management, and execution of the complex, multi-faceted procedures performed repeatedly by IT specialists and operations personnel. C2O streamlines application processing tasks, automates routine application support and administration tasks, speeds problem resolution and disaster recovery, and ensures best-practice and regulatory controls compliance. With C2O, IT organizations can realize 30 50% staff efficiency gains for critical application support processes. WWW.OPTINUITY.COM 7101 WISCONSIN AVE. SUITE 1100 BETHESDA, MD 20814 P 202.292.4920 F 301.652.3169