Gothenburg Mainframe and Continuous Integration. Jan Marek com. CA Technologies. Session S610

Transcription

1 Jan Marek com CA Technologies Session S610 Mainframe and Continuous Integration

2 Agenda Introduce continuous integration concept What problem we were facing Overview of the solution Examples of main components Questions & Answers

3 Agile software development background Scrum Master Product Owner Team

4 What does continuous integration mean? It is a practice in software engineering merging all developer working copies with a shared integration mainline several times a day Intended to be used in combination with automated unit tests written for testdriven development It is a continuous application of quality control as opposed to traditional practice of doing quality control after completing all development So it perfectly fits the idea of iterative software delivery

5 Continuous integration workflow

6 Benefits and costs Current build available for demo or actual delivery to end user Avoids last-minute chaos at release dates and ensures compatibility of several pieces done by different teams Integration bugs are detected early and are easy to track down Enforces discipline of frequent automated testing Constructing an automated test suite requires a considerable amount of work There is also some work involved to set up and maintain a build system

7 Problems we were facing Legacy code with low test coverage ISPF as well as pure batch applications Legacy test suite Complex batch job streams parameterized by PROCs It would take moths to understand the logic and extend the coverage Iterative software delivery which required frequent robust testing

8 What we wanted GUI for test libraries and results Single interface to run, track and check test results Ability to reuse the legacy batch jobs Ability to build new tests more quickly Simple setup of the testing environment Various stages, releases, subsystems Simple concept Not re-inventing the wheel

9 What we have built Framework written in Python (some classes in Java) that simplifies writing of scripts and automates testing of mainframe applications Test ISPF applications, batch programs and prepare data for tests using SQL and DB2 commands and utilities Main features Simple access to DB2 via SQL without hardcoded values SSID, userid and so Simple submitting of jobs from the framework and access to their outputs Simple access to fields in ISPF panels by their ID (based on label or ISPF panel source code)

10 Languages Python Designed for code readability Express concepts in fewer lines of code than in languages such as C++ or Java A bit slower than Java programs Best suited as a glue languages Java Intended for portability - write once and run anywhere Better characterized as a low-level implementation language Jython Python designed to run on the Java platform

11 High level architecture JES x3270 FTP Local computer Jenkins server JDBC driver Subversion repository

12 Subversion best practicies Frequent commits Commit early and often to find issues as soon as possible Policy for trunk No syntax error on trunk, framework unit tests must pass Commit related code changes together Do not commit unrelated code in the same commit Do not commit build artefacts Use ignore list Use comments when you commit Comments help others understand changes Post commit event that sends an to your development mailing list Use authentication

13 Connection to DB2 via JDBC driver Used to execute SQL queries and commands Ensure you have the following jars in your classpath db2jcc.jar db2jcc_licence_cu.jar Part of any DB2 LUW 10.5 JayDeBeApi allows you to connect from Python to database using JDBC import jaydebeapi connection = jaydebeapi.connect('org.hsqldb.jdbcdriver', ['jdbc:hsqldb:mem:.', 'SA', ''], '/path/to/hsqldb.jar',) curs = connection.cursor() curs.execute('select * FROM SYSIBM.SYSDUMMY1') curs.fetchall()

14 x3270 IBM 3270 terminal emulator Runs on Linux, Mac OS X, Solaris, Cygwin, Windows Runs over a TELNET connection emulating either 3279 or 3278 Available in several different forms x3270 is for use on X graphics displays wc3270 is the Windows console version s3270 is for screen-scraping scripts pr3287 is for printer emulation

15 s3270 Implements RFCs 2355 (TN3270E), 1576 (TN3270) and 1646 (LU name selection) Supports IND$FILE file transfer Opens a telnet connection to an IBM host and allows a script to control the host login session Supports the telnet-passthru service to allow outbound connection through a firewall machine

16 py3270 A Python interface to x3270 IBM 3270 terminal emulator It provides an API to x3270 or s3270 subprocess Contains the Emulator class which we have extended Main benefit automatic recognition and naming of input fields General functions such as login, fields recognition TSO and ISPF commands, reading of panelid and messages DB2 and DB2 Tools specific commands and options

17 py3270 example from py3270 import Emulator # use x3270 so you can see what is going on em = Emulator(visible=True) # or not (uses s3270) em = Emulator() em.connect('3270host.example.com') em.fill_field(17, 23, 'mylogin', 8) em.fill_field(18, 23, 'mypass', 8) em.send_enter() # maybe look for a status message if not em.string_found(1, 2, 'login succesful'): abort() # do something... # disconnect from host and kill subprocess em.terminate()

18 FTP to JES Alternative to RJE Submit jobs and retrieve the results using TCP/IP FTP JCL must be created on the submitting platform SITE JESLRECL=xxx (up to 254) The job name must match the user s userid plus one character (QUOTE) SITE FILETYPE=JES to connect to JESPLEX PUT to submit job, GET to retrieve the output from the spool

19 FTP to JES commands example ftp> quote site filetype=jes 200 Site command was accepted ftp> put test.jcl 200 Port request OK. 125 Sending Job to JES Internal Reader FIXrecfm It is known to JES as JOB04661 ftp> dir 200 Port request OK. 125 List started OK SYSLBDT JOB04661 OUTPUT 4 Spool Files GET JOBnnnnn.s where nnnnn is the job number from the ls or dir ftp subcommand and s is the SYSOUT file sequence number. GET JOBnnnnn.X where nnnnn is the job number and X is used to indicate to retrieve ALL of the JOB s SYSOUT into one file. The X may be specified in either upper or lower case.

20 FTP to JES FTP class FTP class which uses the FTP to JES interface to Submit job Wait for job to be completed Find spool files Get spool files Parses JES system messages Create/Update/Delete datasets

21 FTP to JES Get jobs Python example self.ftp.sendcmd("site FILETYPE=JES") reply = self.ftp.sendcmd("site JESJOBNAME=%s JESSTATUS=%s JESOWNER=%s" % (USER1*, ALL, USER1)) assert reply.startswith("200 ") Output from FTP ftp> dir EZA1701I >>> PORT 127,0,0,1,4, Port request OK. EZA1701I >>> LIST 125 List started OK for JESJOBNAME=USER1*, JESSTATUS=ALL and JESOWNER=USER1 EZA2284I JOBNAME JOBID OWNER STATUS CLASS EZA2284I USER1 TSU00025 USER1 OUTPUT TSU ABEND=222 3 spool files EZA2284I USER1A JOB00209 USER1 OUTPUT A ABEND=806 3 spool files EZA2284I USER1 JOB00201 USER1 OUTPUT A RC= spool files EZA2284I USER1J JOB00208 USER1 OUTPUT A (JCL error) 3 spool files EZA2284I USER1 JOB00193 USER1 OUTPUT A RC= spool files EZA2284I USER1A JOB00200 USER1 OUTPUT A ABEND=806 3 spool files 250 List completed successfully.

22 FTP to JES Create dataset Python example self.ftp.sendcmd("site FILETYPE=SEQ") self.ftp.sendcmd("site MGMTCLASS=mgmtclass") self.ftp.sendcmd("site RECFM=recfm") self.ftp.sendcmd("site PRIMARY=primary") fp = StringIO("") reply = self.ftp.storlines("stor dsn", fp) assert reply.startswith("250") ''' Use SITE subcommand to set DCB parameters and STOR command to create dataset Use STOR with fp to write into sequential dataset Use SITE DIRECTORY=dirblocks with self.ftp.sendcmd("mkd dsn") to create PDS/PDSE '''

23 Continuous integration server Free, open source continuous integration tool written in Java Server-based system running in a servlet container It supports CVS, Subversion, Git, RTC and so Builds can generate test reports in various formats Jenkins can display the reports and generate trends and render them in the GUI

24 Jenkins projects and test suits

25 Jenikns test results

26 Questions & Answers