BornAgain: a versa.le framework for modelling and fi6ng GISAS

BornAgain: a versa.le framework for modelling and fi6ng GISAS Céline Durniak, Walter Van Herck, Gennady Pospelov, Joachim WuIke Scien&fic Compu&ng Group at MLZ Jülich Center for Neutron Science JCNS Koordinierungstreffen, October 28-29, 2013, Jülich

Scientific Computing group at MLZ Organiza.on o A joint group of HGF and TUM Group members o Joachim WuIke, nominated July 2011 o Walter Van Herck, since Januar 2012 o Gennady Pospelov, since Januar 2012 o Céline Durniak, since Januar 2013 o An6 Soininen, since June 2013 o Marina Ganeva, since September 2013 Our mission is the development, integra&on, and maintenance of data- analysis sodware for specific scafering techniques Scope of responsibility o Support instruments for ques.ons at the interface of physics and IT o Enhance understanding of experimental results by providing simula.ons BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 02

Scientific Computing group at MLZ Current projects o BornAgain: GISAS simula.on and analysis Céline Durniak, Walter Van Herck, Gennady Pospelov, Joachim WuFke o Mul.ple scaiering simula.on AnQ Soininen, Joachim WuFke o Raw data reduc.on for SPHERES and TOFTOF Céline Durniak, Marina Ganeva, Joachim WuFke o Data catalogue Joachim WuFke BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 03

BornAgain project o Mo.va.on o Architecture o Func.onality o Graphical User Interface prototype o Infrastructure for code development o Summary BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 04

o Mo.va.on o Architecture o Func.onality o Graphical User Interface prototype o Infrastructure for code development o Summary BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 04

Motivation Heinz Maier- Leibnitz Zentrum (MLZ) FRM- II neutron source in Garching bei Munich HDRI High Data Rate Processing and Analysis Ini.a.ve From local (MLZ) and more broader needs (HDRI) o Start community project for data analysis and simula.on o Grazing incidence small angle scaiering as a first task different user experiments (neutron and x- rays) more generally structured than exis.ng sofware IsGISAXS as reference sofware o Apply acquired experience to other experiments BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 05

Grazing Incidence Small Angle Scattering o beam is directed on a surface with a very small incident angle o devia.on from perfectly smooth layered system leads to beam scaiered in an out- of- specular direc.on o 2D detector records the intensity of scaiered wave giving access to lateral and ver.cal sample structure informa.on BornAgain framework JCNS Koordinierungstreﬀen, October 28-29, 2013, Jülich 06

Distorted Wave Born Approximation (DWBA) o accounts for reflec.on- refrac.on effects close to the cri.cal angle o interference between scaiering from direct and reflected beam Scalar case f(, ) h out V ini!2 2 = 4 terms Scalar case f(, ) h out V ini!4 4 = 16 terms (2 2) See talk of Walter Van Herck given at ILL hip://apps.jcns.fz- juelich.de/doku/sc/posters_and_talks polariza.on BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 07

Starting IsGISAXS as a star.ng point: o Successful sofware which is a de facto standard in the user community o Simula.on in DWBA o FORTRAN 90, 13k lines of code o Windows (with GUI), Linux (without GUI) o No longer ac.vely supported BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 08

Starting IsGISAXS limita.ons o no support for mul. layer systems o no mul.ple types of par.cles in single layer o no polarized neutrons and magne.c materials o no paralleliza.on o usability IsGISAXS parameter file BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 09

Starting New sofware plaoorm or extend exis.ng? Extending IsGISAXS o code reuse From scratch C++ o object oriented approach in sample descrip.on o performance o developers background Mul.Layer Layer 1 Interface 1 Layer 2 Interface 2 Layer 3 Nanopar.cles 1 Nanopar.cles 2 Roughness parameter Formfactor Interference func.on Formfactor Interference func.on BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 10

Strategy Reproduce func.onality of IsGISAXS o IsGISAXS examples as milestones along the way Extend it with most demanded features o mul.layers, par.cle assemblies, polarized neutrons and magne.c domains o usability Other requirements o open source code o no dependency on proprietary sofware o decouple physical modelling, data, GUI o mul. plaoorm (Linux, Mac, Windows) Build user community and follow their demands BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 11

BornAgain Development started in April 2012 The name of the sofware, BornAgain, indicates the central role of the Distorted Wave Born Approxima.on o generic frame for modeling and fi6ng mul. layer samples at grazing incidence geometry o Current Release 0.9.1 status beta transi&on to produc&on release with user help BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 12

Basic software architecture Main features: o Programming language C++ o Core library consist of 30k lines of code o Python bindings with boost- python o Linux, Mac, Windows BornAgain Python bindings Python bindings libbornagaincore Samples and algorithms libbornagainfit Interface to minimizers BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 14

Basic software architecture External dependencies: o Well established libraries: Boost, sw3, Eigen, GSL o ROOT from High Energy Physics community (op.onal) BornAgain Python bindings Python bindings libbornagaincore Samples and algorithms libbornagainfit Interface to minimizers GSL Boost Eigen ROOT BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 15

Basic software architecture Working with BornAgain. o Running user C++ program User C++ program External graphics BornAgain Python bindings Python bindings libbornagaincore Samples and algorithms libbornagainfit Interface to minimizers GSL Boost Eigen ROOT BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 16

Basic software architecture Working with BornAgain. o Running user Python script User Python script External graphics BornAgain Python bindings libbornagaincore Samples and algorithms Python bindings libbornagainfit Interface to minimizers Matplotlib GSL Boost Eigen (Py)ROOT BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 17

Functionality Func.onality (scalar) o Arbitrary number of layers o Arbitrary number of different included par.cles (form factors, material) o Arbitrary number of interference func.ons between the par.cles o Inter- layer roughness (also with correla.on) o Nanopar.cle assemblies BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 19

Functionality Func.onality (polariza.on) o Arbitrary number of layers o Arbitrary number of different included par.cles (form factors, material) o Arbitrary number of interference func.ons between the par.cles o Inter- layer roughness (also with correla.on) o Nanopar.cle assemblies BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 20

Usage examples Example: simula.on of cylindrical nano par.cles with interference func.on on top of substrate using python script BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 21

Usage example: simulation with python script BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script ambience material substrate material Par.cle material BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script air layer substrate layer BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script Nano par.cle is defined by the material and form factor Addi.onal transforma.on can be defined for orienta.on and posi.on of the par.cle BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script InterferenceFunc.on1DParaCrystal defines interference func.on of regular la6ce with long range order gradually destroyed BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script InterferenceFunc.on1DParaCrystal defines interference func.on of regular la6ce with long range order gradually destroyed Par.cleDecora.on Holds informa.on about par.cle types, posi.ons and interference BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script InterferenceFunc.on1DParaCrystal defines interference func.on of regular la6ce with long range order gradually destroyed Par.cleDecora.on Holds informa.on about par.cle types, posi.ons and interference Layer Decorates air layer with par.cle decora.on BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script Mul.Layer owns decorated air layer and substrate layer BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script Mul.Layer owns decorated air layer and substrate layer Simula.on Holds beam and detector parameters, accept mul. layer as an input and perform calcula.ons BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage example: simulation with python script OutputData Simulated intensity as a func.on of outgoing α f, φ f angles can be retrieved as NumPy array BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 22

Usage examples Example: fi6ng BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 26

Usage examples: fitting the data this is your minimizer BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 23

Usage examples: fitting the data Fi6ng in BornAgain o Variety of mul.dimensional minimiza.on algorithms and strategies o The choice over possible fi6ng parameters, their proper.es and correla.ons o Full user control on chi2 calcula.ons o Simultaneous fit of arbitrary number of datasets BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 24

Usage example: fitting o Every number (or group of numbers) used in sample construc.on can be used as fit parameter Mul. Layer Layer #0 Layer #1 thickness Par.cleDecora.on thickness FormFactorCylinder radius height Library Minuit2 GSL TMVA Minimiza&on algorithms provided by ROOT Algorithm Migrad, Simplex, Fumili, Scan Fletcher- Reeves Conjugate Gradient Algorithm BFGS Conjugate Gradient Algorithm Levenberg- Marquardt Algorithm Simulated Annealing Algorithm Gene.c Algorithm Name to find corresponding parameters in sample parameter database BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 25

Usage example: demo BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 26

Planning GUI Objec.ves o Simula.on and fit from graphical user interface o Short learning curve for users o Developing through prototyping and frequent usability tes.ng BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 28

Bad graphical user interface BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 29

BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 30

Planning GUI Current GUI prototype: o 20k lines of code o 3 man- months First public beta: Simula.on from GUI o 30k lines of code +3 man- months Fi6ng from GUI o 40k lines of code Around 60-100k lines of code for the GUI at the end +3 man- months In the future BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 31

Planning GUI Current GUI prototype: o 20k lines of code o 3 man- months First public beta: Simula.on from GUI o 30k lines of code +3 man- months +3 man- months Agile Fi6ng methods from GUI breaks o 40k lines of code tasks into small increments with minimal planning In the future and do not directly involve long Around term 60-100k planing lines of code for the GUI at the end BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 31

Development process General prac.ces o More agile development o Quality control o Good documenta.on Sofware tools o Bug/issue tracking: Redmine o Version control: Git o Nightly build server: TeamCity o Code documenta.on: Doxygen o Code browser: OpenGrok o Unit tests: Googletest BornAgain git repository number of lines of code vs &me BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 33

Development process: agile development o workflow consist of sprint cycles every 3-4 weeks during which the team create finished por.ons of product One day I want to fly away Users lasts 3-4 weeks Backlog: Sprint: Release Developers Item 124 Item 125 Learn to fly Fly away Item 124 Item 125 Item 127 Item 98 Item 101 Item 102 BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 34

Development process: agile development Workflow is managed via Redmine management tool o user request handling, bug tracking tool, issue tracking BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 35

Development process: quality assurance Unit test control o small test cases are executed afer each compila.on to ensure code stability Nightly build system o based on TeamCity o automa.c nightly compila.on and tes.ng of the whole project BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 36

Development process: quality assurance Func.onal tests o higher level tests produces scaiering plots which are automa.cally analyzed o 14 scaiering geometries tested through Python and C++ API s o special tests compare BornAgain results with IsGISAXS results BornAgain IsGISAXS rela.ve difference rela.ve difference spectra BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 37

What has been done April, 2012 April, 2013 Released first public version of the framework (home workshop at Garching, April,4) o Mac/Linux support o IsGISAXS func.onality is mostly in place o Fi6ng, Python bindings o Project infrastructure is seiled o One real life example: mesocrystal of self assembled nanopar.cles BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 39

What has been done May October, 2013 o Implemented GUI prototype 3 man- months o Replaced build system with user friendly CMake 1 man- month o Wrote alfa- version of user manual 30 pages 1 man- month o DWBA formalism for polarized neutrons studies 1 man- month ;) o Refactored the framework for polarized DWBA 1 man- month o Implemented Windows build and installer 2 man- months o Different IT studies CMake, PyQt4, SIP, Qt5 1 man- month o Made 6 talks at different occasions including Abingdon, UK, Emerging GISAXS 2013 workshop 1.5 man- months o Made one poster for Hamburg, DESY GISAXS2013 workshop BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 40

What has to be done Started to receive requests o Jean Francois Moulin (3 layered system with hexagonal la6ce on top) o Oleg Petracic, Alice Klapper (BPM media) o Stefan MaIauch (lateral structured test sample) o Sabrina Disch (magne.c mesocrystal) o Elisabth Josten (mesocrystal) o A number of requests from ILL Graphical user interface 30% Plans.ll June, 2014 o Public beta of graphical user interface (simula.on) o Test of polarized DWBA on magne.c sample User requests Refactoring, missed func.onality User manual Build and release support Possible.me budget alloca.on for the next 6 months 30% 20% 10% 10% BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 41

Where to get information Google Scien.fic Compu.ng at MLZ You will arrive hip://apps.jcns.fz- juelich.de Posters And Talks BornAgain hip://apps.jcns.fz- juelich.de/bornagain Have copy of given talk, see other talks Source code Quick start User manual Redmine issue tracker BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 42

BornAgain making DWBA manageable Scien.ﬁc Compu.ng at MLZ Thank you BornAgain framework JCNS Koordinierungstreﬀen, October 28-29, 2013, Jülich 43

Backup BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 44

Usage example: mesocrystals Grazing incidence small- angle x- ray scaiering from a mesocrystalline system o mesocrystals have cylindrical shape and size of ~1000 nm o mesocrystal consist of an FCC la6ce composed of 5 nm spherical par.cles BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 45

Usage example: mesocrystals Experimental data compared to a BornAgain simula.on o 11 parameters fit o 12 hours on six core (12 threads), 2000 itera.ons, 20 sec/itera.on beam_intensity la6ce_length_a la6ce_length_c nanopar.cle_radius sigma_nanopar.cle_radius meso_height meso_radius sigma_meso_height sigma_meso_radius sigma_la6ce_length_a surface_filling_ra.o roughness 5.01e+12 6.21 6.57 4.70 0.37 112.22 945.63 1.00 1.61 1.16 0.17 28.72 experimental data courtesy Elisabeth Josten et al Born BornAgain simula&on in collabora&on with Artur Glavic and Elisabeth Josten BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 46

C++ and Python API s C++ Python BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 47

Usage example: development of graphical user interface Status prototyping o rely on Qt5 (Qt4) and ROOT libraries o BornAgainCore library used as a plugin Objec.ves o drag and drop sample editor o interac.ve plots o access to large collec.on of minimizers o Python and C++ scrip.ng BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 48

BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 49

Development process: quality assurance Nightly build system o based on TeamCity, one build configura.on Code profiling o regular tracking of memory and CPU consump.on BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 50

Development process: good documentation Code documenta.on o Doxygen generated, star.ng to write user manual Theory and conven.ons o described and stored in the repository parallel to the code development BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 51

Quick start BornAgain framework JCNS Koordinierungstreffen, October 28-29, 2013, Jülich 52