How To Run A Hybrid Batch Processing Job On An Z/Os V1.13 V1 (V1) V1-R13 (V2) V2.1 (R13) V3.1.1

Transcription

1 Co:Z Load Balancer on the zenterprise June 22, Copyright 2012 Dovetailed Technologies, LLC Slide 1

2 Agenda zenterprise Environment Hybrid Batch Processing Overview Co:Z Load Balancer Features Demonstration Testing Observations Copyright 2012 Dovetailed Technologies, LLC Slide 2

3 zenterprise Environment Overview z/os V1 R13 - IBM Java SDK 6.0+, JZOS - IBM Ported Tools for Z/OS - Co:Z Co-Processing Toolkit - Co:Z Load Balancer zbx configuration - (2) Linux - (2) AIX - (1) Windows Server zbx installed software - IBM GPMP - IBM Java SDK Co:Z Target Systems Toolkit - Cygwin (windows only) Copyright 2012 Dovetailed Technologies, LLC Slide 3

4 z/os Hybrid Batch Processing Ability to execute a program or script on a distributed (target) system from a z/os batch job Target program may already exist and should require little or no modification Target program's input and output are redirected from/to z/os spool files or data sets Target program may access other z/os resources: DD's, data sets, UNIX files and programs Target program's exit code is adopted as the z/os job step condition code Copyright 2012 Dovetailed Technologies, LLC Slide 4

5 Co:Z Load Balancer Features Implements Server/Application State Protocol (SASP) and Interfaces with IBM zenterprise Unified Resource Manager (zurm) Distributes hybrid batch work to zbx virtual servers Server groups configurable using XML and validated using configurable group confidence quorum When deployed on z/os, - Integrates with z/os Console, write to operator (WTO) logging - Supports console commands for restart and shutdown Web Service for status, advice, restart, shutdown Copyright 2012 Dovetailed Technologies, LLC Slide 5

6 Co:Z Load Balancer Requirements IBM Ported Tools for z/os Co:Z Launcher For each virtual server: - IBM supported operating system, including AIX(p), Linux(x), and Windows(x) - IBM Guest Platform Management Provider (GPMP) - Co:Z Target Systems Toolkit For the system running the Co:Z Load Balancer: - (z/os) IBM Java SDK (non-z/os) IBM Java SDK 6.0+ or Oracle Java SE 6.0+ Copyright 2012 Dovetailed Technologies, LLC Slide 6

7 Co:Z Load Balancer Availability Released May 2012 Product documentation and download Copyright 2012 Dovetailed Technologies, LLC Slide 7

8 Co:Z Load Balancer on zenterprise/zbx z196/z114 z/os zbx x Linux/Win... target advice p AIX SASP HMC / zurm Copyright 2012 Dovetailed Technologies, LLC Slide 8

9 Demonstration Overview Review Load Balancer configuration Review status reported by zurm - Web interface for status and advice testing Run load balanced job - Hybrid Batch PDF Generator - Review z/os console messages Shutdown Co:Z Load Balancer - using z/os console command Copyright 2012 Dovetailed Technologies, LLC Slide 9

10 Co:Z Load Balancer config.xml <?xml version="1.0" encoding="ibm-1047"?> <config lbuid="coz-lb01" gwmserver=" " groupconfidencequorumpct="50"> <group name="zbxgrp"> <member label="aixb20201" port="22" host=" " weight="60" /> <member label="aixb20203" port="22" host=" " weight="60" /> <member label="xlinuxb20702" port="22" host=" " weight="60" /> <member label="xlinuxb20703" port="22" host=" " weight="60" /> </group> <group name="wingrp"> <member label="xwindowsb020803" port="22" host=" " weight="60" /> </group> </config> Copyright 2012 Dovetailed Technologies, LLC Slide 10

11 Co:Z Load Balancer JCL //COZLB JOB (),MSGCLASS=H,NOTIFY=&SYSUID //PROCLIB JCLLIB ORDER=IBMUSER.COZ.SAMPJCL //******************************************************************* //* Run Co:Z LoadBalancer via JZOS //******************************************************************* //JAVA EXEC PROC=JVMPRC60, // JAVACLS='com.dovetail.coz.sasp.LoadBalancer', // ARGS='-f config.xml' //* //CEEDUMP DD SYSOUT=* //STDENV DD *. /etc/profile COZLB_HOME=/u/vendor/coz-lb cd $COZLB_HOME export JAVA_HOME=/usr/lpp/java/J6.0/J6.0 export PATH=/bin:${JAVA_HOME}/bin export LIBPATH=/lib:/usr/lib:${JAVA_HOME}/lib/s390/j9vm export IBM_JAVA_OPTIONS="-Xms16m -Xmx32m -Dfile.encoding=ISO8859-1" CLASSPATH=$COZLB_HOME:${JAVA_HOME}/lib:${JAVA_HOME}/lib/ext for i in "${COZLB_HOME}"/*.jar; do CLASSPATH="$CLASSPATH":"$i" done export CLASSPATH="$CLASSPATH": // Copyright 2012 Dovetailed Technologies, LLC Slide 11

12 Co:Z Load Balancer Status and Advice Load balancer "coz-lb01" is started with: Connection(/ :3860) groupconfidencequorumpct(50%) coz-lb01.wingrp - quorum: 100% xwindowsb020803: :22,weight=56,REGISTRATION,CONTACTED,CONFIDENT coz-lb01.zbxgrp - quorum: 100% AIXB20201: :22,weight=56,REGISTRATION,CONTACTED,CONFIDENT AIXB20203: :22,weight=56,REGISTRATION,CONTACTED,CONFIDENT xlinuxb20702: :22,weight=56,REGISTRATION,CONTACTED,CONFIDENT xlinuxb20703: :22,weight=56,REGISTRATION,CONTACTED,CONFIDENT # Co:Z SASP LoadBalancer advice for group=zbxgrp target-host= Copyright 2012 Dovetailed Technologies, LLC Slide 12

13 Co:Z Load Balancer advice.sh #!/bin/sh # Uses the IBM Ported Tools "curl" command to get load balancing # advice from the Co:Z LoadBalancing server # Usage: # advice.sh groupname # Customize this to point to your IBM Ported Tools curl path CURL=${CURL:-"/usr/lpp/ported/bin/curl"} # Customize these to match your Co:Z SASP LoadBalancer server LBHOST=${LBHOST:-" "} LBPORT=${LBPORT:-"8860"} echo "Executing: $CURL -ss >&2 exec $CURL -ss Copyright 2012 Dovetailed Technologies, LLC Slide 13

14 Co:Z Load Balancer Hello World JCL //LBHELLO JOB (),'DOVETAIL',MSGCLASS=H,NOTIFY=&SYSUID //PROCLIB JCLLIB ORDER='IBMUSER.COZ.SAMPJCL' //* //********************************************************************* //* Batch job to run the Co:Z Launcher under the Co:Z Load Balancer //********************************************************************* //* //LBHELLO EXEC PROC=COZPROC //INPUT DD DISP=SHR,DSN=SYS1.MACLIB(ACB) //COZCFG DD * target user=cozuser properties exit=/u/vendor/coz lb/advice.sh zbxgrp ssh tunnel=false server host= //STDIN DD * echo Hello $(hostname)! fromdsn b DD:INPUT gzip c tofile b /tmp/out.gz // Copyright 2012 Dovetailed Technologies, LLC Slide 14

15 Generated PDF Example Copyright 2012 Dovetailed Technologies, LLC Slide 15

16 Generated PDF JCL //LBITEXT JOB (),'DOVETAIL',MSGCLASS=H,NOTIFY=&SYSUID,CLASS=A //PROCLIB JCLLIB ORDER='IBMUSER.COZ.SAMPJCL' //************************************************************** //* Using Co:Z Launcher/Co:Z Load Balancer, run PDF Generator //************************************************************** // SET PRFX=IBMUSER.COZ.ITEXT //DELOLD EXEC PGM=IEFBR14 //PDFOUT DD DSN=&PRFX..PDFOUT, // DISP=(MOD,DELETE,DELETE),SPACE=(CYL,1) //LBITEXT EXEC PROC=COZPROC, // REGSIZE='32M',LIBRARY='IBMUSER.COZ.LOADLIB' //FLDDATA DD DISP=SHR,DSN=&PRFX..FLDDATA //COMDATA DD DISP=SHR,DSN=&PRFX..CONFIG(COMDATA) //COMMAP DD DISP=SHR,DSN=&PRFX..CONFIG(COMMAP) //FLDMAP DD DISP=SHR,DSN=&PRFX..CONFIG(FLDMAP) //TEMPLATE DD DISP=SHR,DSN=&PRFX..TEMPLATE //PDFOUT DD DSN=&PRFX..PDFOUT, // DISP=(NEW,CATLG),SPACE=(CYL,(10,10),RLSE), // DCB=(RECFM=U,BLKSIZE=27998) //COZCFG DD * target-user=cozuser properties-exit=/u/vendor/coz-lb/advice.sh zbxgrp ssh-tunnel=false server-host= //STDIN DD * cd /home/cozuser/coz-itext export CLASSPATH="." for i in./*.jar; do CLASSPATH="$CLASSPATH":"$i" done exec java com.dovetail.coz.itext.itextpdfgenerator \ --common-data <(fromdsn DD:COMDATA) \ --common-map <(fromdsn DD:COMMAP) \ --field-data <(fromdsn DD:FLDDATA) \ --field-map <(fromdsn DD:FLDMAP) \ --pdf-template <(fromdsn -b DD:TEMPLATE) \ --pdf-output >(todsn -b DD:PDFOUT) // Copyright 2012 Dovetailed Technologies, LLC Slide 16

17 Co:Z Load Balancer Console Commands F COZLB,APPL=RESTART IBMUSER F COZLB,APPL=RESTART JOB Co:Z LoadBalancer RESTART received JOB CZLB030I SASP connection stop was requested JOB CZLB032I SASP connection stopped JOB CZLB012I LoadBalancer SASP client stopped JOB CZLB034I Starting SASP connection to: :3860 JOB CZLB004I Socket connected; localport=1261 server: / :3860 JOB CZLB035I SASP connection started successfully JOB CZLB011I LoadBalancer SASP client started P COZLB IBMUSER P COZLB JOB Co:Z LoadBalancer command STOP received JOB CZLB030I SASP connection stop was requested JOB CZLB032I SASP connection stopped JOB CZLB012I LoadBalancer SASP client stopped Copyright 2012 Dovetailed Technologies, LLC Slide 17

18 zbx Virtual Server Workload (Stacked) 100% 30 Minute Stress Load (Stacked) ~75% CPU on Linux1 90% 80% 70% Weight 60% 50% 40% Linux-2 Linux-1 AIX-2 AIX-1 30% 20% 10% 0% 80 Minutes Copyright 2012 Dovetailed Technologies, LLC Slide 18

19 zbx Virtual Server Workload Minute Stress Load ~75% CPU on Linux Weight AIX-1 AIX-2 Linux-1 Linux Minutes Copyright 2012 Dovetailed Technologies, LLC Slide 19

20 Summary Co:Z Launcher + zenterprise => z/os Hybrid Batch Processing Co:Z Load Balancer can be used to distribute hybrid batch work to zbx virtual servers Co:Z Toolkit is free to use under a community license Commercial support contracts are available Visit for more information Copyright 2012 Dovetailed Technologies, LLC Slide 20