How To Use The Jobscheduler On A Linux Box (Jid) On A Pcode (Jio) Or Macbook 2.2 On A Microsoft Powerbook 2 (For A Freebie

Transcription

1 JobScheduler - Job Execution and Scheduling System JobScheduler Information Dashboard Work Plan and History March 2015 March 2015 JobScheduler page: 1

2 JobScheduler - Contact Information Contact Information Software- und Organisations-Service GmbH Giesebrechtstr. 15 D Berlin Germany Telephone +49 (0) Telefax +49 (0) Mail [email protected] Web Last Updated: 03/13/ :01 PM This documentation is based on JobScheduler Version Copyright SOS GmbH Berlin. All rights reserved. All trademarks or registered trademarks are the property of their respective holders. All information and materials in this book are provided "as is" and without warranty of any kind. All information in this document is subject to change without further notice. This product includes software developed by the Apache Software Foundation ( We would appreciate any feedback you have, or suggestions for changes and improvements; please forward your comments to [email protected]. March 2015 JobScheduler page: 2

3 JobScheduler - Table of Contents Table of Contents 1 Introduction Installation Language settings Configuration Settings for Oracle Settings for MySql Settings for SQL Server Settings for PostgreSQL Settings for Sybase Settings for DB Standalone Installation Troubleshooting Authentication and Authorization with JID Generation and evaluation of the work plan Show work plan Opening multiple JobScheduler instances simultaneously with JOC Showing jobs that have been planned for the current day but have not yet run Showing jobs from yesterday that ran with errors Hiding jobs Using the search box Showing the current jobs When are orders marked as being "in error"? Color codes for order states Glossary March 2015 JobScheduler page: 3

4 JobScheduler - Introduction 1 Introduction The JID (JobScheduler Information Dashboard) provides an overview of the jobs planned and those that have successfully been completed. JOC (JobScheduler Operations Center) and JOE (JobScheduler Object Editor) can be also integrated into JID if required. JID provide these components March 2015 JobScheduler page: 4

5 JobScheduler - Introduction an overview of the jobs and job chain orders planned an overview of the jobs and job chain orders that have been completed. JOC (JobScheduler Operations Center) JOE (JobScheduler Object Editor) List of actual events and the status of event handlers Jobnet tree, Jobnet Listview and some functions to maintain Jobnets or parts of them. Work Plans JobScheduler calculates the start times of jobs and job chains and starts them at the appropriate time. JobScheduler only considers the start times of the next job or job chain/order and does not consider jobs or orders whose start times lie in the past. However, in everyday job scheduling activities, a calender function or work plan is often useful. JID provides such a plan and provides answers to the following questions: what should have run and what has actually run? what will run in the next few days? A particularly useful feature of work plans is that they provide a list of all the jobs and orders that are scheduled to start. In addition, plans are regularly updated to show the current status of jobs or orders. This means that work plans can be used to provide a quick overview of which jobs and orders have run and which have not, and whether jobs that have been completed ran punctually and have been successful. Job History In addition to providing an overview of the jobs and job chains that have already run and those that are currently running, the Dashboard can provide a centralised summary of a whole system, in which case, the information March 2015 JobScheduler page: 5

6 JobScheduler - Introduction shown can be focused on all or on individual JobScheduler instances in the system. Every job and order that has run can be seen directly in the log view in the lower part of the Dashboard. The work plan and its overview of the jobs that have run provide an important tool for system administrators or help-desk personnel carrying out monitoring and initial error analysis. This is particularly true for the standalone Dashboard version, which does not need a running JobScheduler instance and only requires access to the relevant JobScheduler database. JOC (JobScheduler Operations Center) The JOC View module allows that JOC to be integrated into JID. JOC views can be opened in the JID for any number of JobSchedulers, with each view for a JobScheduler being opened in a separate tab. Note that at the moment, it is not possible to integrate JOC into JID on Linux systems. If a Mozilla instance (e.g. Firefox) is installed, this can be activated using the dashboard.sh script (see below). JOE (JobScheduler Object Editor) The JOE View module allows that JOE to be integrated into JID. The JOE provides the access to the JobScheduler objects in order to manage their configuration. Events March 2015 JobScheduler page: 6

7 JobScheduler - Introduction The Event-View module provides a list of all active events and the a view for the actual status of the event handlers. Jobnet (JobScheduler Networks) The Jobnet-View module provides a gui for monitoring and maintaining the process in a JobScheduler network. March 2015 JobScheduler page: 7

8 JobScheduler - Installation 2 Installation JID is installed together with JobScheduler and can be added to existing installations as an update. In addition, a setup is available for JID, which installs JID as a standalone application. The JID work plan add-on consists of 2 jobs and an illustrative report and is also installed together with the JobScheduler. It is not required for a standalone Dashboard installation. This report shows the runs planned for a specific period for all JobScheduler being monitored. The 2 jobs will be installed on the JobScheduler for which a work plan is to be shown. In addition, the JOC (JobScheduler Operations Center) can be integrated in JID. 2.1 Language settings The language is set according to the settings used in the local operating system. It can be also set using the SOS_LOCALE=en de environment variable. 2.2 Configuration The following files are used to configure the Dashboard and may be modified after installation if required. 1.)./user_bin/dashboard.sh cmd-examples Contains JID start parameters. These set enable_joc=true false: Activates/deactivates the integration of JOC in JID. The default setting for this parameter on Windows system is true. On other systems it is false. set enable_joe=true false: Activates/deactivates the integration of JOE in JID. The default setting for this parameter is false. set enable_job_start=true false: Activates/Deactivates the possibility of starting jobs and orders directly from JID. set enable_jobnet=true false: Activates/deactivates the integration of Jobnet in JID. The default setting for this parameter is false. set enable_events=true false: Activates/deactivates the integration of the event views in JID. The default setting for this parameter is false. 2.)./config/live/sos/daily_schedule/CheckDaysSchedule.job.xml 3.)./config/live/sos/daily_schedule/CreateDaysSchedule.job.xml The configuration files for both these jobs can be edited with JOE (JobScheduler Object Editor). Note that both jobs require a start time before they will run and that this may need to be set in JOE as shown in the screenshot. March 2015 JobScheduler page: 8

9 JobScheduler - Installation The CreateDaysSchedule.job.xml job should run once a day and can be repeated any time as required. The CheckDaysSchedule.job.xml job should be run regularly, for example, every 5 minutes to check the work plan and compare this with the jobs that have already run. If you have more than one JobScheduler then it is sufficient to configure the CheckDaysSchedule job only on one instance. The database settings are specified in the./config/hibernate.cfg.xml file, which is described below. 4.)./config/hibernate.cfg.xml This file contains the database connection settings. It is generated during the JobScheduler set-up and and contains information entered at this time. It will not normally need further modification. However, a text editor can be used to modify this file, if it is necessary to change the database connection settings. This file contains the following DB-specific information, in addition to the username and password. hibernate.connection.driver_class hibernate.dialect hibernate.connection.url Settings for Oracle hibernate.connection.driver_class: oracle.jdbc.driver.oracledriver hibernate.connection.url: jdbc:oracle:thin:@localhost:1521:mydb hibernate.dialect: org.hibernate.dialect.oracle10gdialect Settings for MySql hibernate.connection.driver_class: com.mysql.jdbc.driver hibernate.connection.url: jdbc:mysql://localhost:3306/mydb hibernate.dialect: org.hibernate.dialect.mysqlinnodbdialect March 2015 JobScheduler page: 9

10 JobScheduler - Installation Settings for SQL Server hibernate.connection.driver_class: com.microsoft.sqlserver.jdbc.sqlserverdriver hibernate.connection.url: jdbc:sqlserver://localhost:1433;sendstringparametersasunicode=false;selectmethod=cursor;databasename= mydb hibernate.dialect: org.hibernate.dialect.sqlserverdialect Settings for PostgreSQL hibernate.connection.driver_class: org.postgresql.driver hibernate.connection.url: jdbc:postgresql://localhost/mydatabase hibernate.dialect: org.hibernate.dialect.postgresqldialect Settings for Sybase hibernate.connection.driver_class: com.sybase.jdbc3.jdbc.sybdriver hibernate.connection.url: jdbc:sybase:tds:localhost:5000/mydb hibernate.dialect: org.hibernate.dialect.sybasedialect Settings for DB2 hibernate.connection.driver_class: com.ibm.db2.jcc.db2driver hibernate.connection.url: jdbc:db2://8of9:50000/mydb:drivertype=2;retrievemessagesfromserverongetmessage=true; hibernate.dialect: org.hibernate.dialect.db2dialect 2.3 Standalone Installation The standalone JobScheduler Dashboard installation program has a similar interface to the JobScheduler. After the license agreement is confirmed, the following information has to be entered before installation can take place: installation path; path for configuration and log files, etc; DB system; DB access info and the path to the MySQL JDBC driver Troubleshooting There are two common causes of error if the Dashboard does not start after installation. Both cause the following error message to be generated if the Dashboard is started in debug mode: "org.hibernate.exception.jdbcconnectionexception: Cannot open connection" These are: March 2015 JobScheduler page: 10

11 JobScheduler - Installation That the JobScheduler database does not allow connections to be made from remote hosts. That a firewall on the database host is blocking queries. Tips for solving both of these problems can be found in our "How to connect JID to a remote database" FAQ. March 2015 JobScheduler page: 11

12 JobScheduler - Authentication and Authorization with JID 3 Authentication and Authorization with JID Authentication The authentication will be done by a user name password combination. When starting JID there comes up a form with two fields to identify against the underlying authentication method Authenticatin method There are three available authentication methods authentication against a database authentication against ldap authentication against a shiro.ini file Authorization A user in JID can have severall roles. Each role can be assigned several rights. It is also possible to assign rights explicitly to a user. JID knows the roles: JID: To be able to start JID JOE: To see the JOE view JOC: To see the JOC view EVENTS: To see the Events view JOBNET: To see the Jobnet view You can assign additional rights to the following roles. admin joc_admin jobeditor controller workingplan JID knows the rights jobscheduler:jid:execute To be able to start JID jobscheduler:jid:joetab:show To see the JOE view jobscheduler:jid:joctab:show To see the JOC view jobscheduler:jid:eventtab:show To see the Events view jobscheduler:jid:jobnettab:show To see the Jobnet view jobscheduler:jid:jobstart To be able to start jobs and job chains Rights can be specified with wildcards. The right jobscheduler:jid:* includes all rights starting with jobscheduler:jid: You can enable a wanted function in JID either by assigning a role or a right to a user. A right can be assigned directly to a user or inderictly to a user by assigning the right to a role which is then assigned to the user. March 2015 JobScheduler page: 12

13 JobScheduler - Authentication and Authorization with JID Configure the JobScheduler Security Server The authentication and authorization will be done by the JobScheduler Security Server. This is a REST webservice that can run in a Jetty webserver e.g. within a JobScheduler instance. To enable the JobScheduler Security Server in the Jetty webservice running in a JobScheduler instance please add this the the file web.xml. <servlet> <servlet-name>jersey REST Service</servlet-name> <servlet-class>com.sun.jersey.spi.container.servlet.servletcontainer</servlet-class> <init-param> <param-name>com.sun.jersey.config.property.packages</param-name> <param-value>com.sos.auth.rest</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>jersey REST Service</servlet-name> <url-pattern>/rest/*</url-pattern> </servlet-mapping> Example: Activating JobScheduler security service in web.xml in the dashboard.cmd sh start script you have to specify the adress of the jetty server -security_server= where host is the host name of the jetty server and port is the port for the jetty server. When using the Jetty webserver running in the JobScheduler instance, you find the number of port in the file jetty.xml in the config folder of the JobScheduler instance e.g. -security_server= Configuration with Hibernate To enable the Hibernate Realm, the following shiro.ini must be found in the classpath [main] hibernaterealm = com.sos.dialog.auth.soshibernateauthorizingrealm hibernaterealm.hibernateconfigurationfile=c:\users\nn\documents\sos-berlin.com\jobscheduler\scheduler_current\config\hi bernate.cfg.oracle.xml securitymanager.realms = $hibernaterealm cachemanager = org.apache.shiro.cache.memoryconstrainedcachemanager securitymanager.cachemanager = $cachemanager Example: shiro.ini for Hibernate realm The information containing users, roles and right are stored in the following tables March 2015 JobScheduler page: 13

14 JobScheduler - Authentication and Authorization with JID Table with user and passwords. CREATE TABLE SOS_USER ( "ID" NUMBER(9,0), "SOS_USER_NAME" VARCHAR2(250 BYTE), "SOS_USER_PASSWORD" VARCHAR2(250 BYTE), primaray key("id") ) /*MD5 Hash*/ Directly assigned rights to roles or users. CREATE TABLE SOS_USER_RIGHT ( "ID" NUMBER(9,0), "ROLE_ID" NUMBER(9,0), "USER_ID" NUMBER(9,0), "SOS_USER_RIGHT" VARCHAR2(250), primaray key("id") ) Table with roles CREATE TABLE SOS_USER_ROLE ( "ID" NUMBER(9,0), "SOS_USER_ROLE" VARCHAR2(250 BYTE), primaray key("id") ) Assignment of roles to users CREATE TABLE SOS_USER2ROLE ( "ID" NUMBER(9,0), "USER_ID" NUMBER(9,0), "ROLE_ID" NUMBER(9,0), primaray key("id") ) Example: Database tables for Hibernate realm Configuration with LDAP To enable the Realm LDAP, the following shiro.ini must be found in the classpath Please modify the bold entries according to your needs. In the LDAP Realm it is not possible to assign rights directly to an user. The user in LDAP will have a password which is be used for authentication. The user also will be assigned to an ldap group. The mapping between ldap groups and JID roles is specified in the shiro.ini file. The [roles] section specifies the assignment of rights to roles March 2015 JobScheduler page: 14

15 JobScheduler - Authentication and Authorization with JID [main] ldaprealm = com.sos.dialog.auth.sosldapauthorizingrealm ldaprealm.userdntemplate = cn={0},ou=ehp,cn=manager,dc=my-domain,dc=com ldaprealm.searchbase = ou=ehp,cn=manager,dc=my-domain,dc=com ldaprealm.contextfactory.url = ldap://ehp:389 ldaprealm.groupnameattribute=cn ldaprealm.usernameattribute=sn #Mapping of a ldap group to roles. You can assign more than one role with the seperator sign ldaprealm.grouprolesmap = "SOS01":"admin jid","sos02":"admin" rolepermissionresolver = com.sos.dialog.auth.sospermissionresolveradapter rolepermissionresolver.ini = $inirealm ldaprealm.rolepermissionresolver = $rolepermissionresolver securitymanager.realms = $ldaprealm cachemanager = org.apache.shiro.cache.memoryconstrainedcachemanager securitymanager.cachemanager = $cachemanager [roles] admin = jobscheduler:jid:*, \ jobscheduler:joe:*, \ jobscheduler:joc:* jobscheduler_dashboard_admin = jobscheduler:jid:* jid = jobscheduler:jid:* joc = jobscheduler:joc:execute Example: shiro.ini for LDAP realm Configuration with Ini File It is possible to specify users, roles and passwords in the shiro.ini file without using ldap or a database. The list of users in the section [users] asssings each user a password (first item after assign sign) and a list of roles. In the section [roles] is a list of roles which have a comma seperated list of permissions. [users] root = secret, jobscheduler_dashboard_admin ur = ur, joc, joe joe = joe, joe, jid guest = guest [roles] admin = jobscheduler:jid:joe,jobscheduler:jid:joc,jobscheduler:jid:events jobscheduler_dashboard_admin = jobscheduler:jid:* joc = jobscheduler:jid:joc joc_admin = jobscheduler:jid:joc:* joe = jobscheduler:jid:joe events = jobscheduler:jid:events jobnet = jobscheduler:jid:jobnet Example: shiro.ini for INIFILE realm March 2015 JobScheduler page: 15

16 JobScheduler - Generation and evaluation of the work plan 4 Generation and evaluation of the work plan The work plan is generated by the /sos/dailyschedule/createdaysschedule job which, by default, generates a plan for the current day. The dayoffset parameter is used to specify whether a plan is generated for the future (dayoffset >0) or for the past (dayoffset<0). For example, dayoffset=1 generates the plans for today and tomorrow. Any work plans already existing will be overwritten if new work plans are generated for the same time period. When JID is first installed, the dayoffset parameter has the value of 10, so by default, the work plan will generated for the current day and the following 10 days. All fixed start times are included when generating the work plan. Note that where a job or order is to be run at regular intervals over a specific period of time, JID will only set a single start time for the beginning of the period. This is shown in the dashboard in blue. Evaluation of the work plan (i.e. comparison between jobs and orders planned and those actually completed) is carried out by the /sos/dailyschedule/checkdaysschedule job. This job uses the JobScheduler history tables to evaluate the work plan: If no other parameters are specified, this job will carry out the evaluation for the current day. It is also possible to carry out the evaluation for other time periods by specifying a dayoffset parameter where dayoffset<0. For example, dayoffset=-1 evaluates the work plans for today and yesterday. Note that an existing work plan evaluation created for a given time period will not be deleted when a new evaluation is generated for the same period. 4.1 Show work plan Click on the "Planned" link on the JID to display the work plan. This will cause all jobs and orders planned to be listed. If more than one JobScheduler is being monitored with JID, then the initial work plan view will include jobs and orders from all the JobSchedulers. March 2015 JobScheduler page: 16

17 JobScheduler - Generation and evaluation of the work plan There are, however, different filters which can be used to limit the information shown. These can be activated by clicking the right-hand mouse button. In addition, the JobScheduler-ID can be filtered, as shown in the screen-shot, so that only the jobs and orders planned for a particular JobScheduler instance are shown. The time period filters apply to the lists of planned jobs or jobs that have been completed as well as to the detailed lists for individual jobs or orders. The context menu filter (for example "today") applies to the main lists. Note that the columns are sortable and that the column layout is retained when the session changes Opening multiple JobScheduler instances simultaneously with JOC The JobScheduler instance that is listed at the first position in the SCHEDULER_INSTANCES table is shown in JID when JOC is first opened. The table is filled by the /sos/dailyschedule/createdaysschedule job, which provides details of the JobScheduler instance on which the job runs. Other JobScheduler instances can be opened by clicking the right mouse button on the JOC tab or on the host:port tab as shown in the screenshot. Either of these actions will open a small window in which the host and the port of the instance to be opened can be entered. Any number of instances / tabs can be opened. The right-hand mouse button context menu also contains the function for closing a JOC tab. Note, however, that it is not possible to close the last tab / instance. March 2015 JobScheduler page: 17

18 JobScheduler - Generation and evaluation of the work plan Showing jobs that have been planned for the current day but have not yet run Jobs that have been planned for the current day but have not yet run can be shown by selecting, in this order, the "jobs only ", "delayed", "waiting" and "today" filters in the "planned" tab Showing jobs from yesterday that ran with errors Jobs from yesterday that ran with errors will be shown when yesterday's date is entered as the time period in the "History" view. The "Jobs only" filter is then selected and the listing sorted according to the entries listed in the "Exit" column. Jobs that ended in error will then be shown in red at the start of the list Hiding jobs. Individual jobs can be hidden in the history view. This is done by clicking on a job with the right hand mouse button and selecting the "to ignore list" function in the context menu. The number of jobs that are hidden is shown at the top of the tab in the title. The ignore list can be turned on and off and can be reset. The ignore list will be restored after a restart of the JobScheduler Using the search box A full text search started from the search field can also be used to select from the current list of jobs being shown. The search is carried out in the "Job", "Job Chain" and "Order" columns Showing the current jobs Jobs currently running will be shown in the "History" view when they are sorted according to start time. Jobs currently running will be shown with a grey background, or, to be more exact, entries that do not yet have a completion time will be shown in this way. Note that this could also include jobs that have been deliberately terminated. March 2015 JobScheduler page: 18

19 JobScheduler - Generation and evaluation of the work plan When are orders marked as being "in error"? Orders are marked as being in error (red background in the "History"), if they end with "error", "error" or "fault" status or if they start with an exclamation mark ("!") Color codes for order states Different background colors are used to provide a quick indicator of the state of an order. The colours used depend on whether the "Planned" or "History" tabs are being viewed.. March 2015 JobScheduler page: 19

20 JobScheduler - Glossary Glossary Job Chains Jobs JOC (JobScheduler Operations Center) JOE (JobScheduler Object Editor) Orders A series of jobs that process orders one after the other. The JobScheduler starts the jobs in a job chain automatically, once a order has been started for the chain. Job chains allow a number of orders to be processed in parallel, by starting multiple instances of jobs (tasks). Programs and scripts that are executed by the JobScheduler have to be embedded in jobs. Jobs can contain either start executable files or contain job scripts that use the JobScheduler program interface. More than one instance of a job (task) may run at any one time, should this be required to scale performance. There are two types of jobs: standalone and order jobs. Whereas order jobs are started by orders within a job chain, standalone jobs can be started independently: either manually, scheduled or by directory monitoring. Standalone jobs cannot be run in job chains. JOC (JobScheduler Operations Center) is the JobScheduler interface for monitoring and controlling JobScheduler objects such as jobs, job chains and orders. JOC is opened in a web browser using the address where [scheduler_host] and [scheduler_port] are the host name and the TCP ports number of the JobScheduler (e.g. JOE is the JobScheduler Object Editor. This is used to configure JobScheduler objects (jobs, job chains, orders, schedules, process classes and locks). JOE is started using the script: $SCHEDULER_HOME \bin\jobeditor.cmd (Windows ) $SCHEDULER_HOME /bin/jobeditor.sh (Unix ) Orders activate the processing of job chains. Orders may also contain parameters for the jobs in a job chain. Every job in a job chain has access to the order parameters. Order parameters overwrite job parameters of the same name. Orders can be started according to time. An order processes the jobs in a job chain one after the other. Orders can be configured so that, if a error in processing a job occurs, the order... is removed from the job chain; continues with a further job in the chain; continues with the job that caused the initial error being repeated stands still - that is the order processing is suspended until it is restarted manually. March 2015 JobScheduler page: 20

21 JobScheduler - Glossary Schedules Time-based starting of jobs or orders can either be directly specified for each job or order or can be delegated to a schedule. Individual jobs or orders are then referred to this schedule. This means that if several jobs or orders have the same start parameters, these need only be specified once in the schedule. In addition, one schedule can be replaced by another for a particular period of time, thereby increasing the flexibility of setting job and order start parameters. March 2015 JobScheduler page: 21