JobScheduler. Architecture and Mode of Operation. Software for Open Source

Transcription

1 JobScheduler Architecture and Mode of Operation

2 JobScheduler worldwide Software- und Organisations-Service GmbH

3 Contents Components Supported Platforms & Databases Architecture Job Configuration Deployment Distributed Processing Security Failsafe Operation page: 5

4 Components Run time Components Design time Components page: 8

5 Components Run time components JobScheduler Engine JOC : JobScheduler Operations Center Database File System: Live Folder Configuration Objects page: 9

6 Run time components JobScheduler Engine Running in background (demon / Service) C++ and Java Build-in http-server or Jetty internal and external API Modes of Operation single Instance, single Server remote execution (e.g. with Agents) remote configuration (Supervisor) Load Balancing Failover page: 10

7 Run time components JOC : JobScheduler Operations Center Browser based application Ajax based Connection to JS via TCP/IP Communication via JSE or Jetty http Authentication page: 11

8 Run time components JOC : JobScheduler Operations Center page: 12

9 Run time components Database Access via JDBC Usage Restart capabilities History and logs Fail over load balancing House keeping jobs cleanup history any host and platform Same database for multipe instances page: 13

10 Run time components FileSystem: Live Folder JobScheduler Objects are Flat Files Structured by folders and subfolders XML based syntax of objects JobScheduler file watching active Version control by SVN, Git, page: 14

11 Run time components Configuration objects job job chain order process class schedule lock calender (holidays) parameter (include files) page: 15

12 Design time components JOE : JobScheduler Object Editor JITL : JobScheduler Integrated Template Library JADE : JobScheduler Advanced Data Exchange JID : JobScheduler Information Dashboard page: 16

13 Design time Components JobScheduler Objects: Live Folder (hot folder) JOE : JobScheduler Object Editor JITL : JobScheduler Integrated Template Library JADE : JobScheduler Advanced Data Exchange JID : JobScheduler Information Dashboard page: 17

14 Design time Components JOE : JobScheduler Object Editor page: 18

15 Design time Components JITL : JobScheduler Integrated Template Library Predefined Jobs platform independent File Operations Housekeeping Data Exchange (JADE, SOSFtp) Mail processing Agentless scheduling (SSH) Database: SQL*Plus and PL/SQL Jobs page: 19

16 Design time Components JADE : JobScheduler Advanced Data Exchange Managed File Transfer JobScheduler Adapter (JITL Job Jade) CLI Class Library Ftp, sftp, Ftps, local, WebDav, http, transactional transfer atomic transfer page: 20

17 Design time Components JID : JobScheduler Information Dashboard Single Point of Control History Viewer Daily plan JOE integrated JOC integrated Event processor page: 21

18 Design time Components JID : JobScheduler Information Dashboard page: 22

19 Design time Components JID : JobScheduler Information Dashboard (JOC) page: 23

20 Platforms & Databases Supported Operating Systems Microsoft Windows: 2003, 2008, XP, Vista, 7, 8, 2008 R2, Linux (Suse, RedHat, Cent, ) Solaris (sparc & x86) HP-UX 11 (IA-64) IBM AIX Supported Databases DB2 Oracle MS-SQL Server Sybase MySQL (MariaDB) PostgreSQL H2 Software- und Organisations-Service GmbH

21 Architecture Using a Database with regard to Compliance At a Glance: any platform Browser Access Operations Center (JOC) to control jobs Job Scheduler Editor JOE If you decide to use a database, then you can locate it on any host and platform. Connectivity is established per JDBC. Engine & Operations Center (JOC) Execute jobs Job JDBC Connectivity Job Scheduler Database Store job history and log files any platform dtabase Manage and configure jobs and job chains By the way! The advantage of using a database is that you keep track of the job history and log files for compliance of your IT processes. Job history and log files can be accessed directly by the Operations Center (JOC) per browser. Software- und Organisations-Service GmbH

22 Job Configuration Using Hot Folders At a Glance: Engine & Operations Center (JOC) Job JobScheduler Editor JOE Execute jobs Store jobs and job chains Monitor Hot Folders and apply changes to jobs Hot Folders any platform Jobs and job chains are stored in XML files If you use Hot Folders then JobScheduler will automatically monitor changes to the configuration. JOE enables you to manage configurations in Hot Folders on disk. The JobScheduler monitors the LiveFolder constantly. Any changes are immediately detected. The requested changes are applied right away without the need for a restart of JobScheduler. By the way! You could still use at any time your favorite editor in order to configure jobs in Live Folder. Software- und Organisations-Service GmbH

23 Distributed Processing Remote Execution At a Glance: SSH Server any platform JobScheduler supports the execution of jobs via SSH on remote hosts. No SSH client is required. DB Extension Engine Job Execute jobs on a remote host that runs an SSH server JobScheduler supports the execution of jobs on remote JobScheduler instances. Job chains for execution of individual job steps on remote hosts are.. Job Job Execute jobs on a remote Job Scheduler instance Distribute a job chain to multiple Job Scheduler instances on different hosts Job Engine Job Software- und Organisations-Service GmbH

24 Distributed Processing Remote Execution by Agents Host A Engine Execute job locally Job DB Extension Host B Agent Execute jobs on a remote Job Scheduler instance At a Glance: Job Scheduler supports the execution of jobs via SSH on remote hosts. No SSH client is required. Job Scheduler supports the execution of jobs on remote Job Scheduler instances. Job chains for execution of individual job steps on remote hosts are.. Job Execute job locally Job Software- und Organisations-Service GmbH

25 Distributed Processing Agentless scheduling: Remote Execution by SSH Host A Engine Execute job locally any platform Host B SSH Server At a Glance: JobScheduler supports the execution of jobs via SSH on remote hosts. No SSH client is required. Job chains for execution of individual job steps on remote hosts are.. Job DB Extension Execute job on a remote host that runs an SSH server Job Execute job locally Job Software- und Organisations-Service GmbH

26 Distributed Processing Load Balancing Supervisor Alternatively Job Scheduler Supervisor or Network Storage can be used Network Storage any platform At a Glance: For Load Balancing you can operate an arbitrary number of Job Schedulers in parallel. All Job Schedulers use the same job configurations and database. Engine Job Configuration DB Extension Fetch Enqueued Orders Store Job History Store Order History Job Configuration Fetch Enqueued Orders Store Job History Store Order History Engine Job configurations are loaded from a network storage, alternatively from a clustered database (Managed Jobs). Job Schedulers concurrently execute jobs and job chains that are synchronized by a central database. By the way! Database Cluster database any platform This solution fits well if you want to scale the same jobs for parallel processing on multiple hosts. Since you can process jobs parallely you can achieve high efficiency in your processing time. Software- und Organisations-Service GmbH

27 Failsafe Operation Automatic Failover from Primary to Backup Job Schedulers Supervisor & Operations Center (JOC) Alternatively Job Scheduler Supervisor or Network Storage can be used Network Storage At a Glance: For automatic failover you have to set up one or more Backup Job Scheduler Instances. The Backup Instances use the same configuration and database as the Primary Job Scheduler Instance. Job Configuration Job Configuration Job configurations are loaded from a network storage, alternatively from a clustered database (Managed Jobs). Primary Engine Send primary heartbeats Store Job History Store Order History Send backup heartbeats Check primary heartbeats Backup Engine The backup Job Schedulers constantly check if the Primary Job Scheduler is up and running and will assume control in case of failure of the primary instance. Database Cluster The Backup Job Schedulers will not execute any jobs, unless the heartbeats of the Primary Job Scheduler instance fail to be detected. Software- und Organisations-Service GmbH

28 Security Basic Authentication At a Glance: PHP If you want to apply an individual authentication, you can choose between Basic HTTP authentication and IP address authentication. Engine & Operations Center (JOC) If you use Basic HTTP authentication the user specific passwords are stored MD5 encrypted in the Job Scheduler configuration. In case you use IP address authentication a fixed address per client or network range is specified in the configuration. IP address authentication per client Browser Basic HTTP authentication per client However, this does not apply for the use of terminal servers that assign the same address and for use of HTTP proxies. By the way! any platform If you have a network with dynamic IP assignment and lots of users then these authentication solutions would increase the required configuration efforts. Software- und Organisations-Service GmbH

29 Security External Authentication Repository Access repository for user authentication PHP Engine & Operations Center (JOC) Engine & Operations Center (JOC) Forward access to the Operations Center (JOC) At a Glance: You can choose to use a web server as a proxy between the clients and the Job Scheduler instances. An arbitrary number of Job Schedulers is addressed by one web server. LDAP, PAM and subsequent mechanisms by Apache can be used for individual authentication. By the way! You could authenticate users against existing Unix accounts using passwd. any platform Web Server Access Operations Center (JOC) Browser any platform Software- und Organisations-Service GmbH

30 Open Source Job Scheduler Let the rabbit do the job Software- und Organisations-Service GmbH

31 If you have any questions, please feel free to contact us Klaus-Dieter Büttner Managing Partner CTO Tel.: Dagmar Reimold Sales Manager Tel.: SOS GmbH, Berlin - - page: 40