OpenROAD XML and Source Code Management

Transcription

1 New Features and Possibilities OpenROAD XML and Source Code Management Bodo Bergmann Senior Software Engineer, Ingres Corp.

2 For information contact Product Management at This presentation contains forward-looking statements that are based on management s expectations, estimates, projections and assumptions. Forward-looking statements are made pursuant to the safe harbor provisions of the Private Securities Litigation Reform Act of 1995, as amended. These statements are not guarantees of future performance or functionality and involve certain risks and uncertainties, which are difficult to predict. Therefore, actual future functionality, features, results and trends may differ materially from what is forecast in forward-looking statements due to a variety of factors. Slide 2

3 Contents OpenROAD Software Configuration Management (SCM) Tool Integration From Requirement to Realization New Features XML Support Classes XML Export File Format Exploiting the new features Subversive ideas for OpenROAD development

4 How it started Request from Ingres/OpenROAD partner Lufthansa Systems Aeronautics Requirement to manage OpenROAD sources (components and applications) in Software Configuration Management (SCM) Tool Used already for other sources Used to manage several versions in several branches Need for automatic "diff" and "merge" of component versions

5 Problem with using OpenROAD Export Files Undocumented Format Encoded format Not human readable "Diffs" hard (or even impossible) to read Export files contain meta information about sources Timestamps & user name of creation / last change meta information get changed without touching the source "diff" will show differences Export Import Compileapp Export files can't be edited/merged Will corrupt export file Problems when importing with different II_CHARSET

6 Discussion about possible solutions 1. Use Documentapp Output 2. Develop applications to write and read a diffable & mergeable format by either Fetching components from DB or Converting the export file read file and re-instantiate sources 3. New XML-based export file format

7 Use Documentapp Output + Pros Can use existing tool to generate human-readable output Usable for "diff" - Cons Document App output is Not structured consistently hard to parse Too verbose Contains computed attribute values Not the "truth" Does not contain all attributes required Need application to parse file and re-instantiate sources Solution not integrated in Workbench => Could only be used for some component types Even then: If you don't use certain attributes Changes for other sources would have to be applied manually

8 Applications to write/read a new format + Pros Can provide a dedicated and therefore efficient diffable & mergable format - Cons Very complex & expensive Requires internal knowledge of source objects Need applications to write sources into newly formatted file parse file and re-instantiate sources Solution not integrated in Workbench New attributes will require changes to the writing/parsing applications => Could be implemented Solution for this purpose only

9 New XML-based export file format + Pros Human readable Solution integrated in Workbench Will use existing "w4gldev backupapp" infrastructure Export files can be validated against an OpenROAD-specific XML Schema XML format offers opening to other tools - Cons Very complex & expensive Requires internal knowledge of source objects => Could be implemented Solution can also be used for other purposes

10 Development Partnership Project New feature "XML-based Export File Format" Ingres: Functional Spec, Design & Implementation Lufthansa Systems (LSY): Review of Specs, Testing Targeted for OR2006 SP2 OR4.1 still used for development at LSY Parallel OR2006 installation for Import/Export SCM: Subversion Most commonly used Open Source SCM Might later be replaced by another SCM SCM / DB synchronization TBD

11 XML-based Export File Format Broken into two sub-features: 1. XML Support for OpenROAD 2. XML Export File Format

12 OpenROAD Community Involvement New features were announced to community Functional Specs and Detailed Design Specs made available in the community Wiki Public review meetings held Features discussed with partners & customers at public events Development Summit UK IUA Conference

13 XML Support for OpenROAD Generic XML support classes for generating and parsing arbitrary XML files Easy to use for 4GL developer No need to use additional UserClasses and/or external classes XML documents (file/url) are parsed into object structure representing the tree-based structure of XML documents Support of UTF-8 and UTF-16 output

14 XML Support for OpenROAD (cont.) Set of new System Classes XMLElement Represents an XML element node XMLAttribute Represents an attribute of an element XMLDocument Provides methods to create and parse XML documents Additional low-level methods to write documents without populating tree-like object structure Allows creation of big XML files <?xml version="1.0" encoding="utf-8" standalone="yes"?> <A attr1='50'> <B>Value of B</B> </A> XMLElement XMLAttribute XMLDocument

15 XML Classes in the SystemClass Hierarchy Object XMLAttribute XMLNode XMLDocument XMLElement XMLText XMLCDATA XMLComment XMLPI

16 XMLAttribute Represents an attribute of an XML element part of starting tag name-value pair Attributes Name Value ClientInteger ClientText

17 XMLNode Abstract class Represents a content part of an XMLElement Attributes ParentElement ClientInteger ClientText

18 XMLElement Represents an XML element Delimited by the starting and ending tags in an XML document Attributes Name Attributes Array of XMLAttribute Children Array of XMLNode

19 XMLElement Methods AddAttribute SetAttributeValue GetAttributeValue RemoveAttribute AddChild AddChildElement GetCharacterValue GetChildElementCharacterValue RemoveChild

20 XMLText Represents an untagged text part of the XML element Attributes Value

21 XMLCDATA Represents a CDATA section within an XML element Delimited by <![CDATA[ and ]]> tags within an XML document Attributes Value

22 XMLComment Rrepresents a comment within an XML element Delimited by <!-- and --> tags within an XML document Attributes Value

23 XMLPI Represents a processing instruction within an XML element Delimited by <? and?> tags within an XML document und has a target name different from "xml" Attributes Target Value

24 XMLDocument Used to read and create XML documents Attributes RootElement Errortext ParsedXMLEncoding ParsedXMLStandalone ParsedXMLVersion

25 XMLDocument Methods ParseURL ParseString WriteToFile IsValidName

26 XMLDocument Low-level methods WriteStartDocument WriteEndDocument WriteStartElement WriteEndElement WriteElement WriteTextData WriteCDATA WriteComment WriteProcessingInstruction AbortWrite

27 XML Support for OpenROAD - Example DECLARE xd = XMLDocument; root = XMLElement; ENDDECLARE { root.name = 'A'; root.addattribute(name='attr1', value='50'); root.addchildelement(name='b', valuetext='value of B'); xd.rootelement = root; xd.writetofile(filename='c:\temp\my.xml', indent=true, standalone=true, encoding=xe_utf8); } <?xml version="1.0" encoding="utf-8" standalone="yes"?> <A attr1='50'> <B>Value of B</B> </A> XMLElement XMLAttribute XMLDocument

28 XML Support for OpenROAD - Example DECLARE xd = XMLDocument; ENDDECLARE { IF xd.parseurl(urllocation='c:\temp\my.xml')<>er_ok THEN MESSAGE xd.errortext; ELSE MESSAGE 'RootElement: '+xd.rootelement.name; } ENDIF;

29 XML Export File Format Human-readable Allows "diff" and "merge" UTF-8 encoding solves character set problem Independent of internal storage in DB repository Will be documented Will use documented class and attribute names XML Schema will be provided openroad.xsd Export file can be validated

30 XML Export File Format (cont.) In addition to existing export file format New (optional) parameters for Export/Import methods New (optional) flags for "w4gldev backupapp" New Workbench Export/Import options & dialogs Additional environment variables II_W4GL_EXPORT_FORMAT II_W4GL_IMPORT_FORMAT II_W4GL_EXPORT_INDENTED

31 XML Export File Format (cont.) Configurable meta information usage To decide if meta information should be written during export AlterBy, AlterDate, Creator, CreateDate Optionally export application without components Application export file containing Application Properties Included Applications Required to recreate an application from file before importing its components

32 XML Export File Format - Examples

33 XML Export File Format - Examples

34 XML Export File Format - Examples

35 XML Export File Format - Examples

36 XML Export File Format - Examples Export component w4gldev backupapp out mydb myapp mycomp.xml cmycomp xml Export application (without components) w4gldev backupapp out mydb myapp myapp.xml appsource xml Import component w4gldev backupapp in mydb myapp mycomp.xml xml w4gldev backupapp in mydb myapp mycomp.xml Will try current export format (exp) first, then XML

37 XML-based Export File Format Current Status Design Phase Functional Spec XML Support XML Export File Format Implementation 90% 90% Test In test at partner (implicitly) In test at partner Release Target: OR2006 SP2 Target: OR2006 SP2

38 XML-based Export File Format Demo Workbench Command Line

39 Exploiting the new features Subversive ideas for OpenROAD development

40 OpenROAD Development Source Repository 4GL sources (applications & components) stored in proprietary repository based on DB tables Client/Server Often shared by development team Sources can be exported into proprietary export file format (text) OpenROAD Development clients OpenROAD Repository (Database)

41 OpenROAD DB Repository Limitations Only for OpenROAD sources No workspaces / work areas Sharing of repository causes problems No change lists/sets No atomic submission of several components Limited revert/undo of changes Limited versioning No requirement to provide change description

42 OpenROAD DB Repository Limitations (cont.) No branching No integration with issue tracking systems No automation support (trigger) Limited reporting No change history Limited access control (all or nothing)

43 Requirements in large development teams Different sources OpenROAD, C, C++, OS/SQL scripts, config files, etc. Flexibility Branches for different release levels and/or customers Diffing & Merging of sources Governance Security Change tracking

44 The idea Software Configuration Management (SCM) systems Designed for software change tracking and controlling Provide tools for diff, merge and reporting Have interfaces to issue/defect tracking systems Provide support for workspaces and change lists Conclusion: SCM systems Fulfill requirements for development in large teams Address limitations of the OpenROAD DB Repository

45 The Idea (cont.) Export of OpenROAD sources into XML format Human-readable Manage/Version XML export files using SCM system e.g. Subversion Synchronize OpenROAD DB repositories with SCM workspaces via XML exports/imports svn client OR IDE OpenROAD Repository Development client XML exports w4gldev backupapp

46 Synchronization SCM OpenROAD OpenROAD Repository and SCM have to be synchronized (Un)Locking of components & files on Check-Out/In Synchronize OpenROAD DB Repositories with changes submitted by other users/workspaces into SCM repository Separate DB repositories required for different workspaces Undo/Revert of changes Undo of "Delete" or "Change" requires re-import of original version Undo of "Add" requires deletion

47 Synchronization SCM OpenROAD (cont.) Manually Requires developers to strictly follow a defined workflow Automated - requires additional tool ("OR2SCM") Combination of SCM commands (e.g. svn ) OpenROAD System Class method invocations OpenROAD command line tools (w4gldev backuppapp/destroyapp) SQL commands (for locking of sources in DB) "OpenSCM" Tool prototype developed by Luminary

48 OpenROAD Subversion synchronization Task New application New component Change application / component Delete application / component Actions (OpenROAD, SQL, OS, Subversion) 1. Create new directory for application 2. exportapp (appsourceonly) 3. svn add 1. exportapp c 2. svn add 1. svn lock 2. Unlock in OR DB 3. Edit app/comp 4. exportapp c (after change) 1. svn lock 2. svn delete 3. Unlock in OR DB 4. destroyapp c Comment Directory for each application containing 1 file for application properties 1 file for each component Take lock to prevent later conflict, i.e. merge

49 OpenROAD Subversion synchronization Task Undo changes Submit changes Actions (OpenROAD, SQL, OS, Subversion) 1. svn revert 2. For added: destroyapp (-c) For changed/deleted: importapp (-c) 3. Lock in OR DB 1. Re export all added/changed apps/comps 2. svn commit 3. Lock in OR DB Comment Status should be same as before change Changes to sources in OR DB could occur after initial export Synchronize work area 1. svn update 2. For changed/deleted files: Unlock in OR DB 3. For added/changed files: importapp (-c) For deleted files: destroyapp (-c) 4. Lock in OR DB

50 Automated synchronization OpenROAD Developer uses OR2SCM tool for repository related tasks Development client OR2SCM tool svn client XML exports OpenROAD IDE for editing OR IDE SQL w4gldev backupapp OR2SCM could also be plugged into OpenROAD IDE Possible Open Source project OpenROAD Repository

51 SCM Integration Vision Allow direct use of an SCM-based repository by OpenROAD Workbench Optionally replaces DB-based repository OpenROAD Workbench will fetch components directly from the SCM workspace file system Will abandon the need for additional process/tool for synchronization between SCM and DB repositories

52 SCM Integration Vision (cont.) OpenROAD Developer only uses OpenROAD IDE Development client svn client XML exports OR IDE

53 Exploiting the XML Export Format New format allows new features and enhancements: Ability to provide OpenROAD Open Source (Empire) through an SCM repository Makes collaboration and contributions easier New documentation options Using XML based tools and technologies, e.g. XSLT Documentapp alternative

54 Exploiting the XML Export Format (cont.) Allow creation/editing of OpenROAD Sources outside the OpenROAD IDE, e.g. using Editor (Plain Text or XML based) Eclipse Plug-In OO-CASE/UML tools to define UserClass hierarchy XML technologies MDA tools Text processors Extend scripting capabilities of OpenROAD Even complex frames defined in XML could be run from an OpenROAD scripting engine e.g. orrun.exe

55 Example A total Eclipse of my heart

56 Questions & Answers

57 Thanks very much for your attention! For more information contact