Difference Between Model-Driven and Traditional Iterative Software Development

Transcription

1 Process Implications of Model-Driven Software Development Author: Jorn Bettin Version 1.0 September 2004 Copyright 2003, 2004 SoftMetaWare Ltd. SoftMetaWare is a trademark of SoftMetaWare Ltd. All other trademarks are the property of their respective owners. SoftMetaWare

2 Revision History Version Author Description 1.0 Jorn Bettin Translation of the corresponding German article in OBJEKTspektrum, Sept REVISION HISTORY FRAMEWORKS DEVELOPMENT OF DOMAIN-SPECIFIC FRAMEWORKS WHY GENERATION, IF FRAMEWORKS DO MOST OF THE WORK? DOMAIN ENGINEERING LANGUAGE DESIGN PLATFORM AND INFRASTRUCTURE DEVELOPMENT MODEL-DRIVEN APPLICATIONS APPLICATION ENGINEERING CODING OF BUSINESS LOGIC COMPILING AND BUILD CONCLUSIONS REFERENCES...9 On the surface, Model-Driven Software Development (MDSD) appears to be a fundamentally new paradigm compared to traditional software development. Upon closer examination however, MDSD turns out to mainly shift the focus of iterative development to a higher level of abstraction. Just as with the transition from assemblers to compilers, the main barrier to acceptance is not technological but mainly cultural. Hence, it should not come as a surprise that model-driven techniques do not gain broad acceptance over night. Changes to software development processes need to be introduced carefully, since all participants in the software development process are affected, especially if the changes are not simply about learning new APIs or the syntax of a new programming language. This article explains the main differences between model-driven development and traditional iterative software development. Page 2

3 1 Frameworks Because frameworks play a critical role in MDSD, it is a good idea to investigate framework usage and framework development. The lesson of small is beautiful also applies to frameworks in the model-driven context. The MDSD paradigm starts with manual development of a reference implementation that covers all architectural layers and that can be held very narrow in terms of coverage of application functionality. The reference implementation should already include the binding to appropriate commercial or Open Source frameworks. The next step consists of abstracting framework integration code from the reference implementation, and capturing the results in the form of code templates. This step only increases the costs of developing a proof-of-concept prototype by about 15%-25%. Considering the amount of repetitive framework integration code contained in typical J2EE applications, it is easy to see that using a model-driven approach can save a lot of work. 2 Development of domain-specific frameworks The term domain is typically used in conjunction with expert knowledge in specific industries, such as banking, manufacturing, etc. In MDSD the term domains is also used to refer to sub-domains of the software engineering domain. Especially if a software development team starts without deep industry-specific domain knowledge, the team should at least use all its technical knowledge to automate repetitive tasks in the software development process. MDSD follows the agile mantra of avoiding speculation about the requirements of future applications, and every investment in framework development needs to be underpinned by the requirements of a concrete application. This approach prevents a scenario where a framework team proceeds to develop elaborate frameworks in the absence of concrete problems that need to be solved. Nevertheless, in a larger project, it is recommended to create a dedicated framework team to support one or more application development teams - not to shield framework developers from reality, but to simplify project management and release management. The objectives and priorities for the framework team need to be set by an architecture group, which consists of the team leaders/application architects of the application development teams. The framework team consists of technology specialists who develop reference implementations, write the integration code for external frameworks, derive code templates, and develop small frameworks as required. The results are validated by the architecture group, which acts as the customer of the framework team, and which therefore controls budget and priorities. 3 Why generation, if frameworks do most of the work? Even in small frameworks the source code and additionally available framework documentation is often not sufficient for framework users to quickly gain the necessary insight into "correct" framework usage, i.e. usage as intended by the framework developer. The old time is money applies especially to software development projects. Ensuring correct framework usage is one of the real strengths of MDSD and modeldriven generation of framework completion code. After creating a small example Page 3

4 application, framework authors can extract code templates to automate the generation of framework completion code. At the same time framework users are shielded from framework implementation details. Of course there are those who argue that this not beneficial but rather dangerous, amounting to a dumbing down of application developers. However, this line of reasoning is suspiciously similar to the line of defense taken in favor of assembler programming against the use of higher level programming languages. MDSD tools enable framework developers to do a "proper job". It is time to raise the bar of framework quality expectations. Framework users are not interested in the internal complexity of frameworks, instead they are looking for robust building blocks that add value by simplifying their work. MDSD allows deep framework knowledge to reside with a small group of people, and enables this knowledge to be made available in an optimal format to a large number of application development teams. 4 Domain engineering In software product line engineering [SEI] terminology, the development of an application platform and corresponding tools and process building blocks is known as domain engineering. language design [adapted DSL editor] platform & infrastructure development [reference implementation passes all unit tests] extraction of code templates from the reference implementation Validation of application architecture of generated reference implementation [generated code = reference implementation code] Figure 1 Domain Engineering workflow Page 4

5 As experienced practitioners in the software industry have known all along - there is no universally applicable application development process. Broad software development process frameworks can be used as a starting point for instantiating an application development process, however they suffer from a number of weaknesses: The level of experience of individual team members is not taken into account. Alistair Cockburn provides a very nice discussion of this topic in [Coc01]. Often teams do not have the necessary expert knowledge to efficiently identify relevant process parts and make them available in a digestible form as a project or corporate standard. Only methods that are explicitly geared towards software product line engineering explicitly distinguish between application development and building an application production facility (domain engineering) Any process simplifications that can be achieved with domain specific frameworks and automation are not taken into account. Decision frameworks for using off-the-shelf or Open Source components as an alternative to in-house development are only mentioned briefly if at all in most process frameworks. MDSD provides a number of best practices ([Bet04a], [Bet04b]) to compensate for these deficiencies, with the objective to substantially reduce the up-front costs and the risks normally associated with software product line engineering approaches. 5 Language design The success of using MDSD can be measured by the compactness of the resulting application models: If it takes an inordinate amount of time to maintain the models, then hardly any application development team will be tempted to use the modeldriven approach. Model-driven application development can start as soon as a Domain Specific Language (DSL) has been defined in the form of a formal meta model. Either a MDSD generator tool is used to create a dedicated DSL editor, or normal text editors or UML tools are used to capture application models. The required usability of a DSL editor depends entirely on the specific project context. When larger teams have to work with the editor day-in and day-out, such as is the case in software product development, then it makes sense to invest in building a slick application-modeling tool. Today's MDSD tool kits still leave much to be desired in terms of generating high-quality modeling tools. However, MDSD provides the ideal means for self-help: By using appropriate code templates, it is fairly easy to generate a simple DSL editor. The generated editor can subsequently be refined manually as required, and can be further refined by integrating graphical modeling components at the appropriate points. Thus, incrementally, a highly usable application modeling tool is being constructed. 6 Platform and infrastructure development As already mentioned, the development of a DSL should be seen as the final step in building a domain specific framework. A framework can be seen like the engine in a car it is essential, but it only adds real value when built into a usable product. In this Page 5

6 analogy, a DSL provides the highly automated production facility that assembles engines and other parts into fully functional cars. After appropriate frameworks have been derived from a reference implementation, the next steps consist of refactoring the reference implementation to use the frameworks, and of regression testing the generated reference implementation. The final step before using the newly built application production facility to build real applications, is the definition of a non-trivial test application that covers interesting boundary conditions, which for economical reasons were not covered in the reference implementation. The test application model is used to provide a broad and realistic test of the entire application architecture and code templates. This test will very effectively uncover most errors that may have slipped in during the generalization (derivation of code templates) of the reference implementation. The technique of using a test model to validate the application architecture on a broad basis is a very significant advantage over traditional software development, and it is very much in the spirit of agile and iterative approaches, where early risk minimization is a high priority. 7 Model-Driven Applications To what extent do DSLs motivated by industry domain knowledge play a role in MDSD? Deep industry domain knowledge can be used to provide end users with model-driven applications, i.e. applications that contain a DSL that is used at run time by end users. Some examples that may be familiar: spreadsheets, workflow definition tools, electricity grid modeling components, etc. In all these examples, end users can define arbitrarily complex models in a familiar or easy-to-learn notation. The value of such applications lies in automated processes that read the models and perform model-based calculations. Experience shows that DSLs motivated by industry domain knowledge have a very high stability and life expectancy. DSLs motivated by deep knowledge of software implementation technologies such as frameworks in the J2EE space are primarily used as part of the application development process in combination with generative techniques. DSLs motivated by industry domain knowledge are mostly embedded directly into the business process supported by the application, and are used at run time in conjunction with interpretative approaches or with generation on-the-fly. Here the boundary between application development process and business process (usage of the application) starts to blur - a new perspective for most software developers and users. The answer to the question of which MDSD techniques end up being used in a specific context is one of the results of a carefully executed domain analysis and design process [Cle01]. 8 Application engineering Up to this point, we have concentrated on the domain engineering process. Examining the application engineering process is just as interesting, as it is this process that delivers real applications. In order to be successful with MDSD, the domain engineering process and the application development process need to interact with as little friction as possible. The best results are achieved with an agile approach that uses timeboxed iterations. In fact, the architectural consistency that can be enforced with MDSD turns out to be the key to scaling up agile methods. Page 6

7 application modeling [formal comopnent specifications] (re)generation of integration code [updated integration code] coding of business logic and presentation logic [compilation or unit test errors in application] [updated source code] compiling and build [unit test errors in application] [application passes all unit tests] [comilation or unit test errors in integration code] Of course automated unit tests are preferred! notify domain engineering team Figure 2 Application Engineering workflow The necessary application modeling steps depend on the characteristics of the defined application modeling language (DSLs), and are thus entirely in the hands of the domain engineers. Weaknesses in the modeling language detected by application developers can be addressed iteratively and suggestions for improvement can be implemented and rolled out incrementally, as an "upgrade to the application production facility". It is this degree of freedom to tune and tweak the application development process to the specific context, which allows efficiency improvements over more "rigid" application development processes that try to provide a common denominator for all possible software development projects. Thus, MDSD addresses a class of process issues that is orthogonal to the class of process issues addressed by the various agile methods. Page 7

8 domain engineering (Iteration n+1) feedback integration and feedback application development (Iteration n) Figure 3 Application development is always one iteration behind the iterattions of the domain engineering. Here it becomes obvious why timeboxing is essential. The illustration clearly shows the iterative nature of MDSD 9 Coding of Business Logic MDSD does not mandate that complete applications need to be defined as models on a level of abstraction that is clearly above traditional source code. Instead, MDSD encourages a pragmatic approach, where some aspects of an application can be implemented using traditional general-purpose languages. Often general-purpose languages are well suited for the implementation of business logic and logical expressions. MDSD provides a number of best practices on the specific topic of integrating hand crafted code and generated code [Bet04a]. 10 Compiling and build The compile and build process in MDSD follows the same rules as in traditional iterative development: The build process should be automated to the highest possible degree. When using MDSD this means that besides executing automated unit tests as part of the build, calls of code generation components need to be included in the appropriate places. 11 Conclusions In this article, I have outlined a number of important elements of model-driven software development processes. It becomes clear that MDSD provides a counter point to universally applicable application development processes. Thus it is applicable in all those cases, where deep domain knowledge is available, and it enables this domain knowledge to be exploited for process optimization. However, it should be noted that MDSD requires some investment primarily an investment in the education of staff. The success of MDSD depends on following a well-planned adoption strategy. Technically oriented software architects and application developers may suspect that the biggest difficulty is the availability of appropriate tooling, however the challenges lie elsewhere: By now there is a whole range of commercial and Open Source MDSD tool kits that are suitable for industrial use. By definition, MDSD implies the development of a tailored application development process and of tailored tools. The quality of the generator tool kit does play a role, but it certainly is not the only factor to consider. Careful and Page 8

9 meticulous integration of all the tool components used in the development process is another major factor that can easily be overlooked. Only a small part of software development consists of programming, and there is no tool that can replace hard analysis work. The real potential of MDSD lies in methodological support for integrating domain analysis into the end to end development process, which has a direct influence on the quality of frameworks and DSLs. The best generator is of little help, if corners are cut during domain analysis, and the DSL turns out to be a poor notation for describing the problem domain. Lastly, the bigger obstacles usually are cultural. After all, the introduction of MDSD represents a significant change to software development processes, and the necessary learning curve cannot be compressed arbitrarily. The MDSD adoption process, which is described in [Sta04] and [Bet04b], and the first iterations of domain analysis, are best performed with the assistance of an expert with experience in introducing MDSD. 12 References [Bet04a] [Bet04b] [Bet04c] [Cle01] [Coc01] [SEI] [Sta04] Jorn Bettin, Model-Driven Software Development: An emerging paradigm for industrialized software asset development, mdsd-and-isad.pdf Jorn Bettin, Model-Driven Software Development Teams, Building a Software Supply Chain for Distributed Global Teams, distributed-software-product-development.pdf Jorn Bettin, Model-Driven Software Development Activities, The Process View of an MDSD Project, mdsd-process.pdf Craig Cleaveland, Program Generators with XML and Java, Prentice Hall, 2001 Alistair Cockburn, Agile Software Development, Addison- Wesley, 2001 Carnegie Mellon Software Engineering Institute, Product Line Practice, T. Stahl, M. Völter, J. Bettin, Modelgetriebene Softwareentwicklung ("Model Driven Software Development"), dpunkt, 2004, to be published Page 9