Tobias Flohre Ein Standard für die Batch- Entwicklung JSR-352
Tobias Flohre Düsseldorf @TobiasFlohre www.github.com/tobiasflohre blog.codecentric.de/en/author/tobias.flohre tobias.flohre@codecentric.dewww.codecentric.de
AGENDA Grundlagen JSR-352 Properties, Job-Parameter, Restart, Skip, Retry und Listener Parallelisierung Implementierungen und Betriebsoptionen
AGENDA Grundlagen JSR-352 Properties, Job-Parameter, Restart, Skip, Retry und Listener Parallelisierung Implementierungen und Betriebsoptionen
WAS IST EIN BATCH? Traditionelles Batch-Pattern Read Process Write
BATCH HISTORIE 80er 90er 2000 2008 2009 2013 Mainframe Batch: COBOL, PL/1 Java Batch WebSphere XD Compute Grid (BDS Framework) Spring Batch v1 Spring Batch v2 Spring Batch v3 JSR-352/JEE7 70/80er: Cobol, PL1 Java Batch: Gott Objekte, Spaghetticode, Java main() IBM WebSphere XD Compute Grid, Batch Datastream Framework (BDS) (2007/2008) Spring Batch 1.0 (April 2008) Spring Batch 2.0 (April 2009) JSR-352 Final Release Specification (Mai 2013)
JSR-352 HISTORIE IBM Chris Vignola 2011 Formierung Expert Group JSR-352 VMWare / Pivotal Michael Minella (Spring Batch Lead) Wayne Lund 24.05.2013 Final Release JSR-352 28.05.2013 Final Release JEE 7 Spring Batch Apache BatchEE 11.05.14 7
BATCH FRAMEWORK Restart Skip Persistente Job-Metadaten Retry Automatisches Transaktionsmanagement Skalierungsfeatures
DOMAIN / KONFIGURATION / ABLAUF Job Step Item ItemReader ItemProcessor ItemWriter Chunk
DOMAIN / KONFIGURATION / ABLAUF Job wird in XML erstellt (Job-XML) Domain Specific Language manifestiert sich in XML- Namespace Zentrale Elemente job step chunk reader processor writer item-count <job id="myjob" > <step id="mystep" > <chunk item-count="2" > <reader ref="myreader" /> <processor ref="myprocessor" /> <writer ref="mywriter" /> </chunk> </step> </job> Ablage unter META-INF/batch-jobs 11.05.14 10
DOMAIN / KONFIGURATION / ABLAUF Reader, Processor und Writer implementieren bestimmte Interfaces ItemReader Object readitem() ItemProcessor Object processitem(object item) ItemWriter void writeitems(list<object> items) Implementierungen nicht Bestandteil der Spec 11.05.14 11
DOMAIN / KONFIGURATION / ABLAUF Begin Step item == null list of items item-count reached Open transaction ItemReader Item readitem() ItemProcessor 1 processitem 2 (Item) ItemWriter writeitems (List<Item>) Commit transaction item == null at (1) false 3 true Finish Step 11.05.14 12
DOMAIN / KONFIGURATION / ABLAUF Inkasso Kraftfahrt Job * Step Inkasso Kraftfahrt am 22.03.13 * JobInstance Inkasso Kraftfahrt am 22.03.13 erster Versuch * JobExecution * StepExecution
DOMAIN / KONFIGURATION / ABLAUF Infrastrukturkomponenten BatchRuntime JobOperator public static JobOperator getjoboperator() public long start(string jobxmlname, Properties jobparameters) public void stop(long executionid) public List<Long> getrunningexecutions(string jobname) public JobExecution getjobexecution(long executionid) public Set<String> getjobnames() public List<JobInstance> getjobinstances(string jobname, int start, int count) 11.05.14 14
AGENDA Grundlagen JSR-352 Properties, Job-Parameter, Restart, Skip, Retry und Listener Parallelisierung Implementierungen und Betriebsoptionen
PROPERTIES UND JOB-PARAMETER <job id="job1"> <properties> <property name="base" value=".txt"/> </properties> <step id="step1"> <chunk> <properties> <property name="file" value="myfile#{jobproperties['base']}"/> </properties> </chunk> </step> @Inject @BatchProperty(name="file") </job> private String filename;
PROPERTIES UND JOB-PARAMETER <property name="filename" value="#{jobparameters['filename']}"/> <property name="filename" value="#{systemproperties['filename']}"/> <property name="filename" value="#{partitionplan['filename']}"/> public long start(string jobxmlname, Properties jobparameters)
RESTART public interface ItemReader { public void open(serializable checkpoint) throws Exception; public void close() throws Exception; public Object readitem() throws Exception; public Serializable checkpointinfo() throws Exception; } public interface ItemWriter { public void open(serializable checkpoint) throws Exception; public void close() throws Exception; public void writeitems(list<object> items) throws Exception; public Serializable checkpointinfo() throws Exception; }
RESTART Begin Step Aufruf open() list of items Aufruf checkpointinfo() Open transaction ItemReader Item readitem() ItemProcessor 1 processitem 2 (Item) ItemWriter writeitems (List<Item>) Commit transaction 3 Aufruf close() Finish Step 11.05.14 19
SKIP UND RETRY <chunk item-count="2" skip-limit="25"> <skippable-exception-classes> <include class="java.lang.exception" /> <exclude class="java.io.ioexception" /> </skippable-exception-classes> </chunk> <chunk item-count="2" retry-limit="25"> <retryable-exception-classes> <include class="java.lang.exception" /> <exclude class="java.io.ioexception" /> </retryable-exception-classes> </chunk>
LISTENER <job id="simplejob"> <listeners> <listener ref="myjoblistener"/> </listeners> <step id="chunkstep"> <listeners> <listener ref="mysteplistener"/> </listeners> <chunk item-count="2"> <reader ref="dummyitemreader"/> <processor ref="logitemprocessor"/> <writer ref="logitemwriter"/> </chunk> </step> </job> JobListener StepListener ChunkListener ItemReadListener ItemProcessListener ItemWriteListener SkipReadListener SkipProcessListener SkipWriteListener
AGENDA Grundlagen JSR-352 Properties, Job-Parameter, Restart, Skip, Retry und Listener Parallelisierung Implementierungen und Betriebsoptionen
PARALLELISIERUNG PartitionMapper Name Kategori e Tobias A Dennis A Frank A Daniel B Ben B Max B Christia n C Dirk C Thomas C
PARALLELISIERUNG PartitionMapper Name Kategori e Tobias A Dennis A Frank A Daniel B Ben B Max B Christia n C Dirk C Thomas C PartitionPlan kategorie=a kategorie=b kategorie=c
PARALLELISIERUNG Thread 1 Open transaction ItemReader Item readitem() ItemProcessor 1 processitem 2 (Item) ItemWriter writeitems (List<Item>) Commit transaction kategorie=a 3 Thread 2 Open transaction ItemReader Item readitem() ItemProcessor 1 processitem 2 (Item) ItemWriter writeitems (List<Item>) Commit transaction kategorie=b 3 Thread 3 Open transaction ItemReader Item readitem() ItemProcessor 1 processitem 2 (Item) ItemWriter writeitems (List<Item>) Commit transaction kategorie=c 3 PartitionCollector PartitionAnalyzer PartitionReducer
AGENDA Grundlagen JSR-352 Properties, Job-Parameter, Restart, Skip, Retry und Listener Parallelisierung Implementierungen und Betriebsoptionen
BETRIEB EINER BATCH-ANWENDUNG Batch application (JSR-352) Batch framework Dependency injection framework Transaction management Resources Servlet / EJB Container
BETRIEB EINER BATCH-ANWENDUNG Batch application (JSR-352) WAR / EAR App-Server s JSR-352 implementation Transaction manage- Resourment (JTA) ces CDI Servlet / EJB Container Application Server Infra- Structure
BETRIEB EINER BATCH-ANWENDUNG Batch application (JSR-352) BatchEE Spring Batch WAR / EAR App-Server s JSR-352 implementation Transaction manage- Resourment (JTA) ces CDI Servlet / EJB Container Application Server Infra- Structure
BETRIEB EINER BATCH-ANWENDUNG Batch application (JSR-352) WAR / EAR Spring Batch (JSR-352) Spring dependency injection Transaction management (JTA) Resources Servlet / EJB Container Application Server Infrastructure Testbarkeit Community Open Source Features: Parallelisierung, JavaConfig, Spring Data, Hadoop...
BETRIEB EINER BATCH-ANWENDUNG Batch application (JSR-352) WAR / EAR BatchEE Spring Batch (JSR-352) Spring dependency injection Transaction management (JTA) Resources Servlet / EJB Container Application Server Infrastructure
BETRIEB EINER BATCH-ANWENDUNG Batch application (JSR-352) Spring Batch (JSR-352) Spring dependency injection Spring Boot Fat JAR (Spring) Transaction management Resources Servlet Container https://github.com/codecentric/spring-boot-starter-batch-web
FRAGEN? Tobias Flohre Merscheider Straße 1 42699 Solingen tobias.flohre@codecentric.de www.codecentric.de blog.codecentric.de www.meettheexperts.de 11.05.14 33