Collaborative development of open source software for mobile and web data capture and management

Transcription

1 Collaborative development of open source software for mobile and web data capture and management Achievements, tools and processes, lessons learned and dynamics of changing mobile platforms Jørn Klungsøyr

2 DESIGN COLLECT MANAGE DESIGN Video tutorials available at: doc.openxdata.org

3 Design Design complex forms in graphical interface Supported question types include e.g.: Text, Numbers, Date, Time, Single / Multi-select, Multimedia, Geotagging Define skip logic & answer validation criteria

4 Collect - Mobile Structured SMS manually coded forms for any phone

5 Collect - Web Web-based forms can be accessed through desktop computer, laptop, netbook, pads/tabs and smartphones with access to the internet and JavaScript support

6 Manage Own your data - host on your own servers Define roles for different types of users (e.g. administrator, study manager, mobile data collector) Export data to CSV or direct to a relational database Manage data and forms using a standard web-browser

7

8

9

10

11 Dynamics of changing mobile platforms Palm (C/C++) PocketPC / Windows Mobile (VB.net / C#) J2ME (Java) Nokia / Sony Ericsson / Blackberry Linux (C/C++) iphone/ipad (Objective C) Android (Java) WebOS HTML5 / JavaScript Requires flexible architecture to support multiple platforms, sizes and versions.

12 The race for the rigth platform EpiHandy was developed using.net / Microsoft Technologies (MSSQL, IIS, VB, C#) Running on Windows and PocketPC EpiHandyMobile using Java (J2ME and J2SE) connected to existing.net based applicaiton OpenXdata was developed in Java. Replaced EpiHandy and built on EpiHandyMobile OpenXdata uses GWT/JavaScript for web front end. Next steps? JavaROSA based android, native android, ecollect HTML5/JavaScript client.

13 Collaborative development

14 The Garage hacker approach GarageHacking / Anarchy - just make it work High level of trust/autonomy - no way to handle review/merits Distributed teams no fixed/regular interaction cross sites. Ant based - no dependency management Dependencies as binaries in subversion Fragile release management Mobile build required WTK Long release cycles (1/2-1 year) Numerous forks and paralell efforts

15

16 Formalizing collaborative development Formal consortium founded in June 2010 Organizations as members Management Committee Steering Committee Core Developers External Contributors

17 Community processes Agile / XP and Apache community influenced Tickets Management Committee Prioritized backlogs Core committers - how to become Milestones Patch submission Daily standups - skype 2 week iterations with releases of each discrete component Public releases handled through maintenance releases.

18 Apache / MySQL hybrid model Projects / Sources Backlog Iterations / Releases

19 Continous Integration Maven Code build and dependency management tool Subversion - Code repository Trac Ticket and development management Jenkins Continuous Integration server (build server) Nexus Artifact repository (dependencies) Cobertura Test coverage FindBugs Code analyse Checkstyle Code style checking Concordion - Acceptance tests

20 Integration stories and experiences Clinical Study in Peru with ~24K participants (OpenMRS) Routine HIV reporting in South Africa ~10K users (DHIS) DOTS for MDR-TB in Pakistan++ (OpenMRS Xforms) Community HW - MoTeCH in Ghana(MoTeCH+OpenMRS) Mobile Clinical Trial: OMEVAC in Uganda (OpenClinica) Immunization Registry: mvac Albania - Workflow (YAWL) Web mapping: OpenXmapper (GeoServer/OpenLayers) Web analytics: OpenXanalyzeR (R for analysis) Offline web (ecollect) Reporting (JasperReports) Security: OpenXsecure (BouncyCastle)

21 Preliminary findings: MoTeCH case (I) Initial assumption to use structured SMS by health workers to create and update medical records using their own phones using "free" SMS. o The required messages were long and complex o The costs of SMS's would sum up to be a significant cost Calculated costs of SMS compared to forms based application (J2ME). o It was much cheaper to give new phones and pay for mobile data than cover costs of "free" SMS. User comparison / validation workshop o SMS was surprisingly cumbersome, challenging and unfamiliar o openxdata was intuitive and required little training to start using Conclusion: o "on all fronts it ended up being better. It was cheaper, better ease of use, better quality of data that could be presented and we could collage much more data - enabling more functionality in the system."

22 Preliminary findings: MoTeCH case (II) OpenXdata introduced risks and challenges o Knew they had to change/modify to make more robust o No proper release process Interoperability was challenging o Used pieces of openxdata not the full package o Used whitebox integration with OpenMRS Unstable components o Xforms designer in openxdata was extremely fragile

23 Preliminary findings: Problems Designer has been source of a lot of griefs and tears. Performance issue - some things can be quite slow in the admin interface. The form designer takes quite some time to load. Large projects managing users - usernames, passwords - especially if users only have access to mobile. Lack some features like automated reports/statistics Problems with Data validation and Form version creation. Lack of support for advanced logic Development branches re-integration Lack of multitenant hosted/cloud solution

24 Preliminary findings: Useful functions Ability for a someone who is not a programmer to design a questionnaire a very powerful questionnaire with complex skip logic Flexibility to create forms based applications for any type of domain. Change management in custom applications a change can have huge implications. How we can manage that in openxdata is incredible. Remote used openxdata at the furthest locations on low-end phones in disconnected mode. Enabled individual level data at point of service Xforms and XML based Web-based dashboard and management Supports very large and complex forms on both web and mobile Automated availability of data in regular SQL database.

25 Preliminary findings: Reasons for using It is open source and free and cost of maintenance is very low. Do not need latest phone models. Dashboard works in regular browsers Mobile works on low-end / low-cost phones Flexibility to adapt it to the requirements of clients Ability to deliver solutions fast due to not starting from skratch Immediate availability of data once uploaded from phone Data is validated during collection Reduced cost of data capture Digitized individual level data automated reporting Mobile is easy field workers have not have had challenges the cell phones is ubiquos

26 DESIGN COLLECT MANAGE DESIGN Video tutorials available at: doc.openxdata.org

27 EXTRA

28 Technologies used Java Spring Hibernate GWT/GXT MySQL Apache Tomcat Xforms and XML Liquibase Database change management Testing frameworks - JUnit / Spring Testing Framework Build Tool Maven Dependeny management - Nexus Mobile: J2ME

29 Key Innovations Binary Protocol plugin handle multiple versions of clients. Non-binary protocol support e.g. REST Serverside processing of XForms and XML Versioned Hybrid XML and Relational DB

30 Hardware infrastructure 2 locations 400km apart in Norway o University of Bergen o University College of Ålesund Local backup and cross site replication LDAP Linux servers and services. Pgina is investigated to offer authentication for windows clients. Account registration 30+ vm s on XenServer (4 physical 2 simple) CentOS 5.7 Ubuntu (switching to12.04) Windows 7 / 2003

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45 DESIGN COLLECT MANAGE DESIGN Video tutorials available at: doc.openxdata.org