Co:Z Load Balancer on the zenterprise June 22, 2012 http://dovetail.com info@dovetail.com Copyright 2012 Dovetailed Technologies, LLC Slide 1
Agenda zenterprise Environment Hybrid Batch Processing Overview Co:Z Load Balancer Features Demonstration Testing Observations Copyright 2012 Dovetailed Technologies, LLC Slide 2
zenterprise Environment Overview z/os V1 R13 - IBM Java SDK 6.0+, JZOS - IBM Ported Tools for Z/OS - Co:Z Co-Processing Toolkit - Co:Z Load Balancer zbx configuration - (2) Linux - (2) AIX - (1) Windows Server zbx installed software - IBM GPMP - IBM Java SDK 6.0+ - Co:Z Target Systems Toolkit - Cygwin (windows only) Copyright 2012 Dovetailed Technologies, LLC Slide 3
z/os Hybrid Batch Processing Ability to execute a program or script on a distributed (target) system from a z/os batch job Target program may already exist and should require little or no modification Target program's input and output are redirected from/to z/os spool files or data sets Target program may access other z/os resources: DD's, data sets, UNIX files and programs Target program's exit code is adopted as the z/os job step condition code Copyright 2012 Dovetailed Technologies, LLC Slide 4
Co:Z Load Balancer Features Implements Server/Application State Protocol (SASP) and Interfaces with IBM zenterprise Unified Resource Manager (zurm) Distributes hybrid batch work to zbx virtual servers Server groups configurable using XML and validated using configurable group confidence quorum When deployed on z/os, - Integrates with z/os Console, write to operator (WTO) logging - Supports console commands for restart and shutdown Web Service for status, advice, restart, shutdown Copyright 2012 Dovetailed Technologies, LLC Slide 5
Co:Z Load Balancer Requirements IBM Ported Tools for z/os Co:Z Launcher For each virtual server: - IBM supported operating system, including AIX(p), Linux(x), and Windows(x) - IBM Guest Platform Management Provider (GPMP) - Co:Z Target Systems Toolkit For the system running the Co:Z Load Balancer: - (z/os) IBM Java SDK 6.0+ - (non-z/os) IBM Java SDK 6.0+ or Oracle Java SE 6.0+ Copyright 2012 Dovetailed Technologies, LLC Slide 6
Co:Z Load Balancer Availability Released May 2012 Product documentation and download http://dovetail.com/products/loadbalancer.html Copyright 2012 Dovetailed Technologies, LLC Slide 7
Co:Z Load Balancer on zenterprise/zbx z196/z114 z/os zbx x Linux/Win... target advice p AIX SASP HMC / zurm Copyright 2012 Dovetailed Technologies, LLC Slide 8
Demonstration Overview Review Load Balancer configuration Review status reported by zurm - Web interface for status and advice testing Run load balanced job - Hybrid Batch PDF Generator - http://dovetail.com/samples/coz/howto-itext-doc.pdf Review z/os console messages Shutdown Co:Z Load Balancer - using z/os console command Copyright 2012 Dovetailed Technologies, LLC Slide 9
Co:Z Load Balancer config.xml <?xml version="1.0" encoding="ibm-1047"?> <config lbuid="coz-lb01" gwmserver="172.29.120.20" groupconfidencequorumpct="50"> <group name="zbxgrp"> <member label="aixb20201" port="22" host="172.29.255.117" weight="60" /> <member label="aixb20203" port="22" host="172.29.255.118" weight="60" /> <member label="xlinuxb20702" port="22" host="172.29.255.119" weight="60" /> <member label="xlinuxb20703" port="22" host="172.29.255.120" weight="60" /> </group> <group name="wingrp"> <member label="xwindowsb020803" port="22" host="172.29.255.121" weight="60" /> </group> </config> Copyright 2012 Dovetailed Technologies, LLC Slide 10
Co:Z Load Balancer JCL //COZLB JOB (),MSGCLASS=H,NOTIFY=&SYSUID //PROCLIB JCLLIB ORDER=IBMUSER.COZ.SAMPJCL //******************************************************************* //* Run Co:Z LoadBalancer via JZOS //******************************************************************* //JAVA EXEC PROC=JVMPRC60, // JAVACLS='com.dovetail.coz.sasp.LoadBalancer', // ARGS='-f config.xml' //* //CEEDUMP DD SYSOUT=* //STDENV DD *. /etc/profile COZLB_HOME=/u/vendor/coz-lb cd $COZLB_HOME export JAVA_HOME=/usr/lpp/java/J6.0/J6.0 export PATH=/bin:${JAVA_HOME}/bin export LIBPATH=/lib:/usr/lib:${JAVA_HOME}/lib/s390/j9vm export IBM_JAVA_OPTIONS="-Xms16m -Xmx32m -Dfile.encoding=ISO8859-1" CLASSPATH=$COZLB_HOME:${JAVA_HOME}/lib:${JAVA_HOME}/lib/ext for i in "${COZLB_HOME}"/*.jar; do CLASSPATH="$CLASSPATH":"$i" done export CLASSPATH="$CLASSPATH": // Copyright 2012 Dovetailed Technologies, LLC Slide 11
Co:Z Load Balancer Status and Advice http://zentzos:8860/status Load balancer "coz-lb01" is started with: Connection(/172.29.120.20:3860) groupconfidencequorumpct(50%) coz-lb01.wingrp - quorum: 100% xwindowsb020803: 172.29.255.121:22,weight=56,REGISTRATION,CONTACTED,CONFIDENT coz-lb01.zbxgrp - quorum: 100% AIXB20201: 172.29.255.117:22,weight=56,REGISTRATION,CONTACTED,CONFIDENT AIXB20203: 172.29.255.118:22,weight=56,REGISTRATION,CONTACTED,CONFIDENT xlinuxb20702: 172.29.255.119:22,weight=56,REGISTRATION,CONTACTED,CONFIDENT xlinuxb20703: 172.29.255.120:22,weight=56,REGISTRATION,CONTACTED,CONFIDENT http://zentzos:8860/advice/zbxgrp # Co:Z SASP LoadBalancer advice for group=zbxgrp target-host=172.29.255.118 Copyright 2012 Dovetailed Technologies, LLC Slide 12
Co:Z Load Balancer advice.sh #!/bin/sh # Uses the IBM Ported Tools "curl" command to get load balancing # advice from the Co:Z LoadBalancing server # Usage: # advice.sh groupname # Customize this to point to your IBM Ported Tools curl path CURL=${CURL:-"/usr/lpp/ported/bin/curl"} # Customize these to match your Co:Z SASP LoadBalancer server LBHOST=${LBHOST:-"127.0.0.1"} LBPORT=${LBPORT:-"8860"} echo "Executing: $CURL -ss http://$lbhost:$lbport/advice/$1" >&2 exec $CURL -ss http://$lbhost:$lbport/advice/$1 Copyright 2012 Dovetailed Technologies, LLC Slide 13
Co:Z Load Balancer Hello World JCL //LBHELLO JOB (),'DOVETAIL',MSGCLASS=H,NOTIFY=&SYSUID //PROCLIB JCLLIB ORDER='IBMUSER.COZ.SAMPJCL' //* //********************************************************************* //* Batch job to run the Co:Z Launcher under the Co:Z Load Balancer //********************************************************************* //* //LBHELLO EXEC PROC=COZPROC //INPUT DD DISP=SHR,DSN=SYS1.MACLIB(ACB) //COZCFG DD * target user=cozuser properties exit=/u/vendor/coz lb/advice.sh zbxgrp ssh tunnel=false server host=172.29.255.115 //STDIN DD * echo Hello $(hostname)! fromdsn b DD:INPUT gzip c tofile b /tmp/out.gz // Copyright 2012 Dovetailed Technologies, LLC Slide 14
Generated PDF Example Copyright 2012 Dovetailed Technologies, LLC Slide 15
Generated PDF JCL //LBITEXT JOB (),'DOVETAIL',MSGCLASS=H,NOTIFY=&SYSUID,CLASS=A //PROCLIB JCLLIB ORDER='IBMUSER.COZ.SAMPJCL' //************************************************************** //* Using Co:Z Launcher/Co:Z Load Balancer, run PDF Generator //************************************************************** // SET PRFX=IBMUSER.COZ.ITEXT //DELOLD EXEC PGM=IEFBR14 //PDFOUT DD DSN=&PRFX..PDFOUT, // DISP=(MOD,DELETE,DELETE),SPACE=(CYL,1) //LBITEXT EXEC PROC=COZPROC, // REGSIZE='32M',LIBRARY='IBMUSER.COZ.LOADLIB' //FLDDATA DD DISP=SHR,DSN=&PRFX..FLDDATA //COMDATA DD DISP=SHR,DSN=&PRFX..CONFIG(COMDATA) //COMMAP DD DISP=SHR,DSN=&PRFX..CONFIG(COMMAP) //FLDMAP DD DISP=SHR,DSN=&PRFX..CONFIG(FLDMAP) //TEMPLATE DD DISP=SHR,DSN=&PRFX..TEMPLATE //PDFOUT DD DSN=&PRFX..PDFOUT, // DISP=(NEW,CATLG),SPACE=(CYL,(10,10),RLSE), // DCB=(RECFM=U,BLKSIZE=27998) //COZCFG DD * target-user=cozuser properties-exit=/u/vendor/coz-lb/advice.sh zbxgrp ssh-tunnel=false server-host=172.29.255.115 //STDIN DD * cd /home/cozuser/coz-itext export CLASSPATH="." for i in./*.jar; do CLASSPATH="$CLASSPATH":"$i" done exec java com.dovetail.coz.itext.itextpdfgenerator \ --common-data <(fromdsn DD:COMDATA) \ --common-map <(fromdsn DD:COMMAP) \ --field-data <(fromdsn DD:FLDDATA) \ --field-map <(fromdsn DD:FLDMAP) \ --pdf-template <(fromdsn -b DD:TEMPLATE) \ --pdf-output >(todsn -b DD:PDFOUT) // Copyright 2012 Dovetailed Technologies, LLC Slide 16
Co:Z Load Balancer Console Commands F COZLB,APPL=RESTART IBMUSER 00000290 F COZLB,APPL=RESTART JOB07646 00000090 Co:Z LoadBalancer RESTART received JOB07646 00000090 CZLB030I SASP connection stop was requested JOB07646 00000090 CZLB032I SASP connection stopped JOB07646 00000090 CZLB012I LoadBalancer SASP client stopped JOB07646 00000090 CZLB034I Starting SASP connection to: 172.29.120.20:3860 JOB07646 00000090 CZLB004I Socket connected; localport=1261 server: /172.29.120.20:3860 JOB07646 00000090 CZLB035I SASP connection started successfully JOB07646 00000090 CZLB011I LoadBalancer SASP client started P COZLB IBMUSER 00000290 P COZLB JOB07646 00000090 Co:Z LoadBalancer command STOP received JOB07646 00000090 CZLB030I SASP connection stop was requested JOB07646 00000090 CZLB032I SASP connection stopped JOB07646 00000090 CZLB012I LoadBalancer SASP client stopped Copyright 2012 Dovetailed Technologies, LLC Slide 17
zbx Virtual Server Workload (Stacked) 100% 30 Minute Stress Load (Stacked) ~75% CPU on Linux1 90% 80% 70% Weight 60% 50% 40% Linux-2 Linux-1 AIX-2 AIX-1 30% 20% 10% 0% 80 Minutes Copyright 2012 Dovetailed Technologies, LLC Slide 18
zbx Virtual Server Workload 80 30 Minute Stress Load ~75% CPU on Linux1 70 60 Weight 50 40 30 AIX-1 AIX-2 Linux-1 Linux-2 20 10 0 80 Minutes Copyright 2012 Dovetailed Technologies, LLC Slide 19
Summary Co:Z Launcher + zenterprise => z/os Hybrid Batch Processing Co:Z Load Balancer can be used to distribute hybrid batch work to zbx virtual servers Co:Z Toolkit is free to use under a community license Commercial support contracts are available Visit http://dovetail.com for more information Copyright 2012 Dovetailed Technologies, LLC Slide 20