A Proposed Hybrid Web Engineering Process Model for Large-Scale Web-Based Applications Development in Large Web Development Enterprises

A Proposed Hybrid Web Engineering Process Model for Large-Scale Web-Based Applications Development in Large Web Development Enterprises Submitted By Omaima Nazar Ahmad AL-Allaf Supervised By Prof. Dr. Asim El-Sheikh Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy In Computer Information Systems Faculty of Information Systems and Technology The Arab Academy for Banking and Financial Sciences Amman-Jordan August 2008

II

AUTHORIZATION I, the undersigned Omaima Nazar Ahmad Al-Allaf authorize the Arab Academy for Banking and Financial Sciences to provide copies of this Dissertation to Libraries, Institutions, Agencies and any Parties upon their request. Name: Omaima Nazar Ahmad Al-Allaf Signature: Date: 12/8/2008 III

DEDICATION All respect and very special appreciation to my Dear mother Doctor.Nida a Hazem for her encouraging and supporting to me And also to my lovely daughter Dalia for her encouraging and patience to me With Respect Omaima Al-Allaf 12 August 2008 IV

ACKNOWLEDGEMENTS I would like to thank God, first and foremost, for having guided me through the critical stages of my life to successful completion. This thesis is the result of almost four years of full-time study, and during this time I have been supported by many people. I am delighted to now have the opportunity to express my sincere gratitude and appreciation to all of them. I am deeply indebted to my supervisor, Professor Asim Elsheikh for his stimulating suggestions, guiding hand and expertise in the field of Software Engineering. Special thanks also due to the staff members of the Department of information Systems- The Arab Academy for Banking and Financial Sciences-Amman-Jordan for their constant cooperation and help during this period of research. I would like to extend my deepest gratitude and sincere thanks to my mother for her affectionate support that has acted as a necessary stimulus to my academic quest. Finally, my thanks go to my daughter DALIA has helped me to achieve my goal through her patience, understanding and willingness to accommodate my dream. Omaima Al-Allaf 12 August 2008 V

Published Papers Omaima N. Al-Allaf, Guidelines for Using Multimedia in Webpage and User Interface Design, Proceedings of the 4 th International Multiconference on Computer Science and Information Technology (CSIT2006), ASU (private), Vol.3, pp.568-577, Amman, Jordan, 5-7 April 2006. Omaima Nazar Ahmad AL-Allaf and Asim El-Sheikh, Guidelines for Constructing Web Engineering Process Models for Developing Large-Scale Web-Based Applications, The 1 st international conference on digital communications and computer applications (DCCA 2007), Jordan University of Science & Technology, pp.651-666, Irbid, Jordan, 19-22 March 2007. VI

ABSTRACT Currently, the level of adoption of current practice and software process improvement (SPI) in large enterprises is unknown. A literature and a practical survey of web development methodologies has been conducted in this thesis to identify web engineering practice and to understand the extent of web practices currently in use in large enterprises to help improving the software processes. This survey clarified that, there were several web methodologies that has concentrated on delivering functional web applications but most of these applications run over time and budget and without consideration on SPI and quality requirements. The survey confirmed also that the: tools and technology, and standards and procedures are partially adopted, whereas, the project management planning, customer involvement practices, web metrics, and the control of the development process are barely used by these enterprises. A Hybrid web engineering process model for developing the large web applications in large enterprises has been proposed in this thesis. This model consists of many activities to follow: possible division of large web application into many small sub applications; possible division of large number of developers into many sub teams; identify a management team to control and monitor the development process; Stakeholders and customers inclusion and feedback during web development process; requirement analysis, management and development; risk analysis of all large web application s components; adopting the Spiral model by management team to manage and monitor the development progress; adopting the Throwaway Prototype model by each sub team for requirements gathering; adopting the XP agile method by each sub team for customer communication during the development process; adopting WebEng process model by each sub team to match web application s properties and challenges; adopting a suitable user interface design approach; conducting CMMI levels key process areas; conducting web engineering practices; integrating SQA activities; and training the mmanagers and developers on CMMI and web engineering practices. This model is focuses on overall development process phases such as: requirement gathering and analysis; cost, time and effort estimation; planning; design; coding; testing, verification and validation; integration; and documentation. According to Hybrid model evaluation with CMMI levels key process areas, we investigate that: This model steps covered and largely satisfied both of CMMI Level2 and CMMI Level3 process areas; whereas, satisfied CMMI Level4 process areas; but partially satisfied CMMI Level5 process areas. According to Hybrid model comparison with XP, Spiral and WebEng in VII

their compatibilities with CMMI levels process areas; we investigate that this model can overcome the limitations of these process models to use them in combinations. Finally, the evaluation of the Hybrid model according to CMMI process areas has been conducted in this research. This evaluation has been carried out by the aiding of many professional developers and managers currently working in many large web-development Jordanian enterprises. The evaluation results shows that the Hybrid model satisfied about 85.57% of CMMI level2, 86.86% of CMMI level3, 73.35% of CMMI level4, and finally about only 31.65% of CMMI level5. The study indicates that large enterprises can benefit from Hybrid model if they apply it in good manner with sufficient training because it was being noted from the analysis of Hybrid steps that: this model can overcome as possible as the problems of large web applications development mentioned in the literature researches and also web development problems in these large web enterprises. VIII

Table of Contents Contents Page Chapter 1: Overview of the Thesis 1 1.1 Introduction 1 1.2 The Development Process of Web-Based Applications 1 1.3 The Large-Scale Web-Based Applications 2 1.4 The Importance of this Research... 3 1.5 The Problems in Developing Large-Scale Web-Based Applications. 5 1.6 The Research Objectives 5 1.7 Literature Review. 6 1.7.1 Literatures Related to Development Methodologies of Web Applications 6 1.7.2 Literatures related to development methodologies of large web applications 9 1.8 The Literature Analysis (Problems Obtained from Literature). 12 1.9 The Research Methodology... 13 1.10 Outlines of the Thesis 14 Chapter Two: Review of Web-Based Applications Development and Development Process Models 18 2.1 Introduction 18 2.2 The Software.. 18 2.3 Web-Based Applications 19 2.3.1 Multidisciplinary Nature of Web Development... 19 2.3.2 Web-Based Applications versus Traditional Software. 20 2.3.3 Characteristics of Simple and Advanced Web Applications 20 2.3.4 The Components of Web-Based Applications.. 21 2.3.5 The Basic Requirements for Web Engineering. 21 2.3.6 Challenges of Web-Based Applications Development. 21 2.4 The Software Development Process 22 2.4.1 The Perceived Problems in Software Development 22 2.4.2 The Basics of Good Software Development 23 2.4.3 Ad-hoc Development... 23 2.4.4 The Importance of Software Process in Large Organizations. 24 2.4.5 Traditional Software Process Models (Heavyweight) 24 2.4.5.1 Waterfall Software Process Model 24 2.4.5.2 Incremental Process Model 25 2.4.5.3 The Rapid Application Development (RAD) Model 25 2.4.5.4 Prototyping Model. 25 2.4.5.5 Spiral Software Process Model. 26 2.4.5.6 Rational Unified Process Model (RUP)... 27 2.4.6 Agile Process Methodologies (Lightweight). 28 2.4.6.1 Extreme Programming (XP). 29 2.4.6.2 Strengths and Limitations of Agile Methodologies 30 2.4.6.3 Introducing Agile Methodologies in Large Organizations... 30 2.4.6.4 Agile Methodologies for Web-Based Applications Development 31 2.4.7 Agile Methods vs. Traditional Methods 33 2.4.8 Mixing Agile Methods with Traditional Methods 34 2.4.8.1 Challenges to Implementing Agile Processes in Traditional Organizations 36 2.5 Web-Based Applications Development Process 36 2.5.1 The Web Engineering Process Model (WebEng) 37 2.5.2 Factors Responsible for the Failure of Web-Based Applications. 37 2.5.3 Steps for Successful Web-Based Applications Development... 38 IX

Contents Page 2.5.4 Large-Scale Web-Based Applications Development 38 2.5.5 Requirements Engineering (RE) in Web Development 39 2.5.6 The Modeling of Web-Based Applications.. 41 2.5.6.1 Model-Driven Development (MDD) of Web-Based applications 41 2.5.7 Design Models for Web-Based Applications 42 2.5.8 Cost, Effort and Time Estimation of Web-Based Applications 44 2.5.9 The Stakeholders of Web-Based Applications. 45 2.5.10 Project Planning.. 45 2.5.11 Management of Web Development 46 2.5.12 Tools for Web Engineering 47 2.5.13 Testing Web-Based Applications 48 2.5.14 Documenting Web-Based Applications.. 49 Chapter 3: Software Process Improvement and Quality Assurance in Large Enterprises 3.1 Introduction. 50 3.2 The Software Process Improvement with CMM 50 3.3 The Capability Maturity Model Integration (CMMI) 51 3.3.1 The CMMI Capability Maturity Levels 52 3.3.2 The Categories of CMMI Key Process Areas.. 53 3.4 Organizations with Software CMM Experience. 54 3.4.1 Immature Versus Mature Software Organizations... 54 3.4.2 Training. 54 3.5 The Software Process Standards (ISO Standards). 55 3.6 The Importance of Software Process Improvement (SPI). 56 3.7 CMMI for Large Enterprises.. 56 3.7.1 Lack of Research about SPI for Large Enterprises.. 57 3.7.2 The Need of SPI to Solve Problems in Large Enterprises 57 3.7.3 Development of SPI Activities in Large Enterprises 58 3.7.4 The SPI Success Factors in Large Organizations. 58 3.8 Factors Affecting Organizational Change in SPI Efforts... 58 3.9 The SPI and Agile Methodologies.. 59 3.9.1 The Elements of SPI in Agile Software Development 60 3.9.2 Comparison of SPI Elements in Traditional and Agile Development.. 60 3.9.3 Agile Adoption and Improvement Model (AAIM). 61 3.9.4 Compatibility of Agile Methods with CMMI Requirements 62 3.10 The Software Quality Assurance (SQA). 63 3.10.1 The Components of Software Quality Assurance... 63 3.10.2 The Quality Attributes of Web-Based Applications... 64 3.10.3 The Quality Assurance in Web Development Organizations. 65 3.10.4 Web Applications Quality and Support in XP 66 3.10.5 Risk Management and SQA... 66 3.10.6 The Quality Assurance of Large-Web-Applications and SPI. 67 3.10.7 Integrating Quality Activities in Project Life Cycle... 69 50 Chapter 4: An Analytical Survey of Large-Scale Web-Based Applications Development in Large-Scale Jordanian Enterprises 70 4.1 Introduction 70 4.2 Survey Methodology.. 70 4.2.1 Statistical Methods Used in Data Analysis... 71 4.3 The Analysis of the First Questionnaire 72 4.3.1 Factors Used to Select the Large Enterprises (Research Population).. 72 X

Contents Page 4.3.2 General Characteristics of Large-Scale Jordanian Enterprises. 72 4.3.3 Interviews in Large Web Development Enterprises. 73 4.3.4 Development Teams in Large-Jordanian Enterprises... 74 4.4 The Statistical Analysis of the Second Questionnaire 74 4.4.1 Respondent Background... 75 4.4.1.1 Current Position of Respondent. 75 4.4.1.2 Activities Currently Work.. 75 4.4.1.3 CMMI-Related Training. 76 4.4.1.4 Participated in Previous Forms of Software Process Assessment 76 4.4.1.5 Experience of Developers in Present Organization... 77 4.4.1.6 Overall Experience of Developers.. 77 4.4.1.7 Level of Experience with Web Based Applications Development 78 4.4.2 Development and Test Methods... 79 4.4.2.1 The Size of Jordanian Enterprises.. 79 4.4.2.2 Web Application Domain... 79 4.4.2.3 Web-Based Applications Development Involves... 80 4.4.2.4 The Developers Familiarity with Development Methodologies 80 4.4.2.5 Development Methodologies Used by Enterprises. 81 4.4.2.6 Test Types Required by Organization 82 4.4.2.7 Assurance Activities are Performed 83 4.4.2.8 Who Performs the Assurance Activities?.. 84 4.4.3 Description of Possible Symptoms at Organizations... 84 4.4.4 Web Engineering Practices... 87 4.4.4.1 Organizational issues.. 87 4.4.4.2 Standards and Procedures... 89 4.4.4.3 Web Metrics 91 4.4.4.4 Control of the Development Process.. 93 4.4.4.5 Tools and Technology 94 4.5 Summary of Statistical Analysis of the Second Questionnaire.. 96 4.5.1 Summary of Analysis Related to Respondents Background.. 96 4.5.2 Summary of Analysis Related to Development and Test Methods.. 96 4.5.3 Summary of Possible Symptoms at Organizations.. 97 4.5.4 Summary of Web Engineering Practices. 98 Chapter 5: Hybrid Web Engineering Process Model for Large-Scale Web-Based Applications Development 100 5.1 Introduction 100 5.2 The Need for a Standard Web Engineering Process Model.. 100 5.3 Problems (Related to Large Web Development) to be Solved 101 5.4 Guidelines for Constructing Web Engineering Process Models 103 5.5 The Proposed Hybrid Web Engineering Process Model 104 5.5.1 The Features of the Hybrid Web Engineering Process Model 105 5.5.2 The Components of the Hybrid Model 105 5.5.2.1 The Role of the Management Component 106 5.5.2.2 The Role of Development Component.. 108 5.5.3 Stakeholders Coordination in the Hybrid Model 109 5.5.4 Requirement Engineering and Management in Hybrid Model... 110 5.5.4.1 Requirements Development... 111 5.5.5 Planning Phase in Hybrid Model 111 5.5.6 Risk Analysis and Management in the Hybrid Model 112 5.5.7 Cost, Time and Effort Estimation in Hybrid Model... 113 XI

Contents Page 5.5.8 Management of Large Web Application Development in Hybrid Model.. 113 5.5.8.1 Management of Large Number of Developers in Hybrid Model.. 113 5.5.8.2 The Properties of Teams in the Hybrid Model.. 114 5.5.8.3 The Properties of Sub Teams Members in the Hybrid Model.. 115 5.5.8.4 The Relationships of Development Teams in Hybrid Model 115 5.5.8.5 Management within XP Principles Adoption 116 5.5.8.6 Quantitative Project Management. 117 5.5.9 User Interface Design in the Hybrid Model 117 5.5.9.1 User Interface Design Approach for Large Web Applications 118 5.5.10 Configuration Management in Hybrid Model.. 119 5.5.11 Testing, Verification and Validation in Hybrid Model. 120 5.5.12 Project Monitoring and Control.... 121 5.5.12.1 Organizational Process Performance... 121 5.5.13 Technical Solutions in the Hybrid Model. 121 5.5.14 Integration in Hybrid Model 121 5.5.14.1 Product Integration.. 122 5.5.14.2 Integrated Project Management... 122 5.5.14.3 Integrated Teaming.. 122 5.5.14.4 Integrated Supplier Management. 122 5.5.14.5 Organizational Environment for Integration 122 5.5.15 Developers Training in the Large Web Development Enterprises.. 123 5.5.15.1 Organizational Training... 123 5.5.16 Integrating QA Activities with Hybrid Model.. 124 5.5.16.1 The Activities to the different SQA processes. 125 5.5.16.2 Internal Assessment and Light Formal Reviews.. 125 5.5.16.3 External Assessment... 126 5.6 Software Development Process Models Used in Hybrid Model... 126 5.6.1 Spiral Model Adopted by Centralized Team.. 126 5.6.1.1 Advantages of the Spiral Model 126 5.6.2 Throwaway Prototype. 127 5.6.2.1 The Limitations and Strengths of Prototype Model... 127 5.6.3 The Web Engineering Process Model. 128 5.6.4 The XP Adopted by Sub Teams in the Hybrid Model 128 5.6.4.1 The Benefits of Using XP in Hybrid Model.. 128 5.7 Integrating Web Engineering Practices in Hybrid Model. 129 5.8 Software Process Improvement in the Hybrid Model.. 130 5.9 The Problems Solved By the Hybrid Model. 131 5.10 The Advantages and the Limitations of the Hybrid Model. 133 Chapter 6: CMMI-Evaluation of the Proposed Hybrid Web Engineering Process Model 6.1 Introduction 134 6.2 Hybrid Model Evaluation According to CMMI Levels 134 6.2.1 The Compatibility of Hybrid Activities with CMMI Level2.. 134 6.2.2 The Compatibility of Hybrid Activities with CMMI Level3.. 136 6.2.3The Compatibility of Hybrid Activities with CMMI Level4 140 6.2.4 The compatibility of Hybrid Activities with CMMI Level5 140 6.3 The Coverage of XP, Spiral, WebEng and Hybrid with CMMI 142 6.3.1 The Coverage of XP, Spiral, WebEng and Hybrid with CMMI-Level2 142 6.3.2 The Coverage of XP, Spiral, WebEng and Hybrid with CMMI-Level3 143 6.3.3 The Coverage of XP, Spiral, WebEng and Hybrid with CMMI-Level4 145 6.3.4 The Coverage of XP, Spiral, WebEng and Hybrid with CMMI-Level5 146 6.3.5 The Coverage of XP, Spiral, WebEng and Hybrid with CMMI-Generic Practices... 146 XII 134

Contents Page 6.3.6 Summary of the Hybrid Analysis versus CMMI process Areas.. 147 6.4 Hybrid Model Evaluation by Large Jordanian Enterprises 148 4.6.1 CMMI level2 Process Areas and Goals. 149 4.6.2 CMMI level3 Process Areas and Goals. 153 4.6.3 CMMI level4 Process Areas and Goals. 160 4.6.4 CMMI level5 Process Areas and Goals 161 3.6.5 The Summary of Hybrid model Evaluation 162 Chapter7: Discussion and Conclusions 163 7.1 Introduction 163 7.2 Discussion of the Survey Findings 163 7.3 Conclusions 165 7.4 Contributions. 169 7.5 Recommendations to Avoid Failure.. 170 7.6 Future Works in Research. 172 7.7 Limitations of the Research.. 172 References 173 Appendix A.. 191 Appendix B. 195 Appendix C. 202 XIII

List of Tables Table Name Page Table (2.1): Traditional versus Agile Software Development 33 Table (2.2): The Summary of Risk-Based Method 35 Table (2.3): The Core RE Activities 39 Table (2.4): Estimation Dimensions and Corresponding Project Factors. 44 Table (3.1): The Factors Affecting Organizational Change in SPI Efforts 59 Table (3.2): The Fundamentals of Software Development in Plan-Driven and Agile 61 Table (3.3): The Quality Attributes of Web-Based Applications. 65 Table (4.1): Factors Used by Enterprises to Measure the Size of Web Applications 72 Table (4.2): Descriptive Statistics of the Current Position of Employee 75 Table (4.3): The Descriptive Statistics of Respondents Current Work Activities 76 Table (4.4): Descriptive Statistics of Developers Receiving any CMMI Training... 76 Table (4.5): Descriptive Statistics of Respondents Participated in Forms of SPA 77 Table (4.6): Descriptive Statistics of Developers Experience in Present Organization 77 Table (4.7): Descriptive Statistics of Overall Software Experience of Developers 78 Table (4.8): Descriptive Statistics of Level of Experience with Web Development... 78 Table (4.9): Descriptive Statistics of Different Sizes of Large Jordanian Enterprises 79 Table (4.10): Descriptive Statistics of Application Domain in Jordanian Enterprises... 79 Table (4.11): Describe If Development Involve In-House, Outsource or Reusability 80 Table (4.12): Statistics of Developers Familiarity with Development Methodologies 81 Table (4.13): Descriptive Statistics of Development Methods Used by Enterprises 82 Table (4.14): Descriptive Statistics of Test Types Required by Organization 82 Table (4.15): Descriptive Statistics of Performed Assurance Activities 83 Table (4.16): Frequent Distribution of Who Performed Assurance Activities 84 Table (4.17): Developers Perceptions Toward Symptoms at Organization 85 Table (4.18): The Statistics of Organizational Issues in Web Engineering Practices.. 88 Table (4.19): Statistics of Standards and Procedures in Web Eng Practices 90 Table (4.20): The Frequency Distribution of Web Metrics in Web Engineering Practices.. 92 Table (4.21): The Frequency Distribution of Control of the Development Process.. 94 Table (4.22): The Frequent Distribution of Tools and Technology in Web Eng Practices 95 Table (5.1): Problems in the Literature 102 Table (5.2): Problems of Large Web Development in Large Jordanian Enterprises 102 Table (5.3): Problems of Web Engineering Practices in Large Jordanian Enterprises 102 Table (5.4): The Features of Hybrid by Principles.. 106 XIV

Table Name Page Table (5.5): Problems in the Literature solved by the Hybrid model.. 131 Table (6.1): Hybrid Activities Against the Activities of CMMI Level2. 135 Table (6.2): Hybrid Activities Against the Activities of CMMI Level3.. 137 Table (6.3): Hybrid Activities Against the Activities of CMMI Level4 141 Table (6.4): Hybrid Activities Against the Activities of CMMI Level5 141 Table (6.5): Coverage of CMMI-Level2 Areas by XP, Spiral, WebEng and Hybrid... 143 Table (6.6): Coverage of CMMI-Level3 Areas by XP, Spiral, WebEng and Hybrid. 144 Table (6.7): Coverage of CMMI-Level4 Process Areas by XP, Spiral, WebEng and Hybrid 146 Table (6.8): Coverage of CMMI-Level5 Process Areas by XP, Spiral, WebEng. and Hybrid.. 146 Table (6.9): Coverage of CMMI Generic Practices by XP, Spiral, WebEng and Hybrid 147 Table (6.10): The Results of CMMI level2 Requirements Management 149 Table (6.11): The Results of CMMI level2 Project Planning.. 150 Table (6.12): The Results of CMMI level2 Project Monitoring and Control... 150 Table (6.13): The Results of CMMI level2 Supplier Agreement Management 151 Table (6.14): The Results of CMMI level2 Measurement and analysis.. 152 Table (6.15): The Results of CMMI level2 Process and Product Quality Assurance. 152 Table (6.16): The Results of CMMI level2 Configuration Management 153 Table (6.17): The Results of CMMI level3 Requirements Development 154 Table (6.18): The Results of CMMI level3 Technical solution.. 154 Table (6.19): The Results of CMMI level3 Product Integration... 155 Table (6.20): The Results of CMMI level3 Verification. 156 Table (6.21): The Results of CMMI level3 Validation 156 Table (6.22): The Results of CMMI level3 Organizational Process Focus. 157 Table (6.23): The Results of CMMI level3 Organizational Process Definition.. 157 Table (6.24): The Results of CMMI level3 Organizational Training... 158 Table (6.25): The Results of CMMI level3 Integrated Project Management... 159 Table (6.26): The Results of CMMI level3 Risk Management.. 159 Table (6.27): The Results of CMMI level3 Decision Analysis and resolution 160 Table (6.28): The Results of CMMI level4 (2 Process Areas) 161 Table (6.29): The Results of CMMI level5 (2 Process Areas) 161 Table (6.30): The Summary of Hybrid Model Evaluation by CMMI. 162 Table (7.1): The Hybrid Model Activities Compatibility with CMMI levels KPAs... 168 XV

Figure Name List of Figures Figure (1.1): The Framework of Web Engineering Process 4 Figure (1.2): The Research Methodology. 15 Figure (1.3): The Outlines of the Thesis. 17 Figure (2.1): The Web Engineering A multidisciplinary field.. 20 Figure (2.2): Complementary and Conflicting Goals in a Software project. 23 Figure (2.3): The Throwaway Prototype.. 26 Figure (2.4): The Spiral Software Process Model 27 Figure (2.5): The Rational Unified Process.. 28 Figure (2.6): The Life Cycle of the XP Process.. 30 Figure (2.7): The General Process for User Requirements Analysis.. 40 Figure (2.8): The Model-Driven Approach for Web Systems 42 Figure (2.9): A More Detailed Model for a Web Application, Showing User Roles and Interactions 47 Figure (3.1): The Staged View of CMMI 53 Figure (3.2): Agile Adoption and Improvement Model.. 62 Figure (4.1): Factors Used by Enterprises to Measure the Size of Web Applications 72 Figure (4.2): Relationships between Teams in Enterprises. 74 Figure (4.3): Describe the Frequencies of Current Position of Employee 75 Figure (4.4): The Frequencies of Respondents Current Work Activities 76 Figure (4.5): The Frequencies of Developers Receiving Any CMMI Training. 76 Figure (4.6): Participation in Software Process Improvement. 77 Figure (4.7): Frequencies of Developers Experience in Present Organization.. 77 Figure (4.8): Frequencies of Overall Experience of Developers 78 Figure (4.9): Frequencies of Level of Experience with Web Development 78 Figure (4.10): Frequencies of Different Sizes of Large Jordanian Enterprises... 79 Figure (4.11): Frequency Distribution of Application Domain.. 80 Figure (4.12): Describe If Development Involve In-House, Outsource or Reuse 80 Figure (4.13): Frequencies of Development Methods Which developers Familiar With 81 Figure (4.14): Frequencies of Development Methods Used by Enterprises... Figure (4.15): Frequency distribution of Test Types Required by Organization 83 Figure (4.16): Frequent Distribution of Performed Assurance Activities 83 Figure (4.17): Frequent Distribution of Who Performed Assurance Activities.. 84 Figure (4.18): The Mean of Severity Score Description of All Symptoms 86 Figure (4.19): The Means of Occurrence Frequency Score Description of Symptoms 87 XVI Page 82

Figure (4.20): Organizational Issues in Web Engineering Practices... 89 Figure Name Page Figure (4.21): Standards and Procedures in Web Engineering Practices 91 Figure (4.22): Web Metrics in Web Engineering Practices 93 Figure (4.23): Control of the Development Process in Web Engineering Practices... 94 Figure (4.24): Tools and Technology in Web Engineering Practices. 96 Figure (4.25): Overall Best Practices Adoption in Large Jordanian Enterprises. 99 Figure (5.1): The Proposed Guidelines for Constructing Web Eng Process Models.. 104 Figure (5.2): Web-Based Application Division.. 104 Figure (5.3): The Complete Hybrid Web Engineering Process Model... 107 Figure (5.4): The Interaction Process between the Management and Sub Team... 108 Figure (5.5): The Integration of Web Engineering Process with XP Practices... 109 Figure (5.6): Relationships between teams in enterprises 116 Figure (5.7): Steps of UI design of Sub Web Application 119 Figure (6.1): The Results of CMMI level2 Requirements Management. 149 Figure (6.2): The Results of CMMI level2 Project Planning.. 150 Figure (6.3): The Results of CMMI level2 Project Monitoring and Control.. 151 Figure (6.4): The Results of CMMI level2 Supplier Agreement Management 151 Figure (6.5): The Results of CMMI level2 Measurement and Analysis. 152 Figure (6.6): The Results of CMMI level2 Process and Product Quality Assurance.. 152 Figure (6.7): The Results of CMMI level2 Configuration Management. 153 Figure (6.8): The Results of CMMI level3 Requirements Development 153 Figure (6.9): The Results of CMMI level3 Technical Solution.. 155 Figure (6.10): The Results of CMMI level3 Product Integration 155 Figure (6.11): The Results of CMMI level3 Verification 156 Figure (6.12): The Results of CMMI level3 Validation... 156 Figure (6.13): The Results of CMMI level3 Organizational Process Focus 157 Figure (6.14): The Results of CMMI level3 Organizational Process Definition. 158 Figure (6.15): The Results of CMMI level3 Organizational Training. 158 Figure (6.16): The Results of CMMI level3 Integrated Project Management. 159 Figure (6.17): The Results of CMMI level3 Risk Management... 160 Figure (6.18): The Results of CMMI level3 Decision Analysis and Resolution. 160 Figure (6.19): The Results of CMMI level4 (2 Process Areas).. 161 Figure (6.20): The Results of CMMI level5 (2 Process Areas).. 161 Figure (6.21): The Hybrid Evaluation by Large Enterprises According to CMMI. 162 XVII

List of Abbreviation Abbreviation of the study s constructs Hybrid Hybrid Web Engineering Process Model WebEng. Web Engineering Process Model XP... Extreme Programming QA.. Quality Assurance SQA Software Quality Assurance SPI.. Software Process Improvement CMM. Capability Maturity Model CMMI. Capability Maturity Model Integration SEI.. Software Engineering Institute KPA s. Key Process Areas SBPQ. Software Best Practice Questionnaire DB.. Data Base ASP Agile Software Development Process OO. Object Oriented OOP Object Oriented Programming RUP... Rational Unified Process RE.. Requirements Engineering ISO. International Organization for Standardization UI... User Interface UID. User Interface Design SCM... Software Configuration Management SEPG.. Software Engineering Process Group XVIII

Chapter 1 Overview of the Thesis 1.1 Introduction During the last decades, the web-based applications are evolving rapidly using many new technologies, programming languages and models that are being used to increase the interactivity and usability of these applications. A web-based application is a software system based on technologies and standards of the World Wide Web Consortium (W3C) that provides web resources such as content and services through a user interface and web browser [1]. The web-based applications have the properties of network intensiveness, concurrency, unpredictable load, performance, availability, data driven, content sensitive, continuous evolution, immediacy and security [2]. Web engineering is concerned with the use of scientific, hypertext engineering, software engineering best practices, advances, management principles, disciplined, systematic methodologies, iterative process, better development tools and a set of good guidelines to the successful development, deployment, performance evaluation and maintenance of high quality web sites and web-based applications [3]. The emerging field of web engineering aims to bring the web application s development under control, minimizes risks, enhances web site maintainability, quality and successfully manages the diversity and complexity of web application s development to avoid major failures of web applications development [4]. The complexity of designing, developing, maintaining and managing web applications have increased significantly when web applications have evolved [4]. The objective of this chapter is to give a brief description of web-based application, web engineering, development process of web-based applications and large web applications. This chapter includes the importance of this research, problems in developing large-scale web applications and research objectives. This chapter also includes literature review, literatures related to development methodologies of web applications, literatures related to development methodologies of large web applications and analysis of these literatures. Finally, this chapter includes research methodology and outline of this thesis. 1.2 The Development Process of Web-Based Applications The development of modern web-based applications requires a methodologically sound engineering approach because these web applications are complex software systems. The web engineering comprises the use of systematic and quantifiable approaches based on software 1

engineering in order to accomplish the specification, implementation, operation and maintenance of high-quality web applications [1]. The web site s structure, functionality and information contained within it may evolve and change continuously over time to keep information current and to meet user requirements. So it is not possible to specify fully what a web site should contain at the start of the development process. Therefore, the ability to maintain information, scale the structure and functions of web site are major factors that make web application development different in comparison to traditional software development. These factors should be taken into consideration when developing web applications [5]. Many of new methodologies related to web development focused on the user interface design but failed to address the wider aspects, complexity and overall development process of web-based applications [6]. At the same time, the traditional software process models of traditional information systems have challenges to accommodate web specific aspects into their techniques and practices. Therefore, the development of web-based applications requires a mix of web site development techniques together with properties of traditional software process models [7]. 1.3 The Large-Scale Web-Based Applications The simplest form of web applications consists of a set of linked hypertext files that present information using text and limited graphics. When the importance of e-business applications grow, the web applications are evolving into sophisticated computing environments that provide standalone features, computing functions, content to the end user and are integrated with corporate databases and business applications [2]. The properties of large web applications are defined in few literature researches. Garland and Anthony [8] defined a large software project as a system that: including large quantities of source code typically millions of lines; including large number of developers (more than 50, often geographically distributed); including large number of interactive functions; using of multiple programming languages; using multiple mechanisms (files, relational and object databases); and distributing the components over several hardware platforms. Whereas, Bell [9] defined the large-scale software as a system that: comprising a large number of interactive activities that carried out by large number of developers (at least 50) working over lengthy time spans. While, Curtis, et. al. [10] defined the large-scale software project as a system that: involving at least 10 people; involving real-time, distributed or embedded applications; and including hundreds of thousands of lines of delivered system source code. 2

On the other hand Burstin and Ben-Bassat [11] defined the large software as a system that having large and diverse community of users, organizational and interactive activities; including hundreds of thousands of lines of code; including hundreds to thousands pages of documentation; requiring long development time; consuming very large resources during the development and maintenance phases. Whereas Tamai and Itou [12] defined the largesoftware project as a system that: is large in program size, investment cost and development period; consisting of large collections of data; and integrating huge number of interacted functions. And Barnett [13] defined large-scale software project as a system that: involving developers from different companies and cultures; comprising teams with 50 or more members; integrating with legacy applications and third-party software packages; and including a number of sub-project teams that integrate their sub-systems to build the final application. Finally, Delic and Dayal [14] defined large-scale software as a system that: is heterogeneous and very dynamic; involving complex interactive activities among many humans, applications, services and devices; including hundreds of thousands of strategic decisions to operational decisions are made on each business day within the large enterprises. 1.4 The Importance of this Research The large-scale web application consists of large number of interactive web pages and functions that supports multiple users who access these functions through internet and connected networks. The development of web-based applications especially large-scale webbased applications has become common because of the growth of the internet and networks. Therefore, successfully developing large-scale web applications that will execute correctly and consistently in a distributed environment where hundreds or millions of requests need to be serviced is a very difficult task. The development methodologies of web-based applications are still in their early stages and new technologies continue to discover. Sometimes these technologies are alternatives and other times they are used in combinations. Balakrishnan and Somasundaram [15] addressed the key requirements for the successful development of large projects such as: user involvement; management support; clear statement of requirements and the ability to verify them; and proper planning [15]. On the other hand, McDonald and Welland [16] suggested that there is a need to focus on many factors to achieve the success of web applications development such as: more requirements analysis; clear analysis of business needs; better 3

testing and evaluation of web deliverables; and more focus on the issues associated with the evolution of web applications [16]. There is no uniform approach exists to web applications development. Therefore, the web developers need new techniques and guidelines that capture requirements and integrate them within a systems development methodology [17]. The WebEng is a web engineering process model that developed by Pressman [2]. The WebEng consists of many steps as shown in figure (1.1). Pressman focused in this model on the development of web-based applications in general and did not consider the size of these web applications. After that, many researches worked to create high quality web-based applications that deliver a set of complex content and functionality to a broad population of end users. Bahli and Tullio [18] mapped and discussed in their research the progress made so far on web engineering development. They summarized and classified the literature empirical studies on web engineering based on the six phases of the Pressman web engineering process model. They found that much of the web engineering researches focused on the engineering activity. Almost 70% of the published researches concerned with the engineering part of web applications development process such as architectural design, navigation design and interface design tasks of engineering activity and customer evaluation seem to draw more research. Their results suggested a significant need for theory-based research in web engineering. Figure (1.1): The Framework of Web Engineering Process [2] According to as listed above, the importance of this research is to propose a web engineering process model that we hope to be more suitable for successful development of large web applications and deliver these applications within the time, budget and effort. 4

1.5 The Problems in Developing Large-Scale Web-Based Applications Many organizations failed to develop large and high-performance web applications. There are many literature researches which addressed the factors responsible for the failure of web applications and highlights serious problems that affecting the large web applications development. The most important problems that caused the failure of development of large web applications are as follows: 1. Problems in requirement analysis phase [16]. 2. Poor project management of development process [16]. 3. Poor project estimation [16]. 4. The development of web applications that exceeds the budget [19]. 5. Flawed design and development process [20]. 6. Poor understanding of methodology to develop large web-based applications [20]. 1.6 The Research Objectives The objectives of this research are as follows: 1. Proposing a web engineering process model for the development of large-scale webbased applications. We will build this model according to the modification and combination of the basic properties of well known traditional software, agile and web engineering process models. 2. Focusing on the overall development process of large web applications because most of the other earlier researches and methodologies focused only on one development phase such as design, requirements, etc. 3. Reaching a suitable methodology that achieve as possible as an effective development of large web applications and reducing the likelihood of failure of these applications. We will focus in the proposed web engineering model to overcome as possible as the most problems of large web applications development and deliver large web applications on time, effort and within budget. The research questions are: Q.1: Is the proposed web engineering process model compatible with the Software Process Improvement (SPI) Capability Maturity Model Integration (CMMI) key process areas and goals (KPA s) to improve the web development process of the large enterprises and their software quality assurance (SQA)? Q.2: Can the adopting of this proposed web engineering process model overcome as possible as the limitations and challenges of traditional software process models and agile methodologies? 5

Q.3: Can the adopting of this proposed web engineering process model overcome as possible as the development problems of large web applications mentioned in literature researches and studies? Q.4: Can the adopting of this proposed web engineering process model in large enterprises for the development of large web applications within budget, effort and time and overcome as possible as the web development problems? 4. Analyzing the compatibility of the proposed model according to CMMI KPA s to determine the advantages properties of this model against other development models. 5. Finally, the objective also is to evaluate this model (according to CMMI levels) by many professional developers and managers currently working in many large Jordanian enterprises which undertaken large web development. The objective of this evaluation is to determine which CMMI level can be reached by these enterprises when adopting this proposed model. 1.7 Literature Review Many software process models and approaches for web applications development had been studied and proposed in the literature. Until now there are few studies about proposing web process models for developing large web applications. However, there is much room for improving existing models and approaches or coming up with new effective web models for the development of large web applications. We divided the literature review into two sections: development methodologies for web applications and the development methodologies for large web applications. 1.7.1 Literatures Related to Development Methodologies of Web Applications This section includes the literatures related to web engineering process methodologies and techniques for developing web applications without mention to the size of these web applications. A summary and analysis of these literatures are given below: Kushwaha, et. al. [20] addressed the factors that responsible for failure of web applications. They focused on user-centric approach of requirement engineering to help increasing the success rate of web applications development. They proposed the major activities that should be part of a web application s development in order to improve the reliability of web applications. They proposed cognitive process of socio-psychological requirement engineering to overcome the blurring between the requirement engineering and analysis, and also to adopt a more comprehensive approach before cognitive requirement 6

engineering is carried out. They highlighted fifteen complexity issues that have a major effect on the successful development of web applications. Hsieh [21] presented a development methodology for web applications that serves as a road map to guide web development. This methodology consists of three phases: structural design, detailed design and implementation. Structural design views a system at the web-page level and detailed design deals with issues of intra-page. The implementation is logically segmented into four steps: implementation of the visual interface, client-side scripting, severside scripting for the visible interface, and server-side scripting for pages and tasks without a visual interface. Souza and Falbo [22] presented an agile approach for development of web applications which used the concept of agile modeling. They started web engineering process with the identification of business needs, followed by project planning. Next, requirements are detailed and modeled, analysis and design. Then the application is built using tools specialized for the web applications. Finally, the system is tested and delivered to end users. Their approach included four activities: requirement specification, analysis, design and implementation. These four activities form should be applied in an iterative fashion, allowing for user feedback and requirements and system evolution. Whitson [23] described a web development process (WebHelix) for the development of web applications. This model consists of project management plan that creates a task list from a systems architecture diagram and uses chart to synchronize the team workload. The WebHelix methodology used a modified spiral approach to systems development. Some parts of the design process, like the business analysis and planning, are done once at the beginning of the process. Some parts of the process, like deployment and maintenance are done once at the end of the process. The major creation of the web application is done by using set of steps repeatedly, producing a set of more complete prototypes, with the final prototype being the completed web application. A WebHelix process consists of the business success evaluation, planning, analysis, design, coding and integration, testing, evaluation, deployment and training, maintenance and future updates. Crane and Clyde [24] described a lightweight two-phase process that integrated advantages from incremental development, throwaway prototyping and waterfall process models to address the challenges of development of web applications such as: frequent changes in requirements; time and cost constraints; and producing quality user interfaces within the 7

restrictions of the web environment. Their process consists of cascading phases of interdependent waterfall models. Phase1 follows a modified incremental development model with an emphasis on rapid prototyping and involves the iteration of development activities (requirements definition, systems analysis, design, coding and testing). Phase1 helps the development team to understand the requirements and adapt to frequent changes. Whereas phase2 follows waterfall model with an emphasis on quality and consists of well-defined steps that correspond closely to development activities. Unlike phase1, progress is measured in terms of completion of development activities, instead of the completion of features. Phase2 builds on the knowledge gained from the phase1 and produces a product that has a better user interface, more extensible and maintainable. Zhao and Chen [25] proposed a component-oriented model of web application in which a web application is regarded as a collection of components. Each component have its own functionality and cooperating with others through certain interfaces. They aimed to establish a foundation for methodology to effective development and maintenance of web application that can benefit from the notion of component-oriented approach. Under this model a web application is consisted of client components and server components. All components expose their functionality through interfaces. The client component of a web application will carry out all the tasks at the client side including interacting with end users and any other related activities. The server component takes care of all the processing works to happen at the server side and they do not have user interfaces. This model designed to be used at the application definition phase only during the development process to capture the features of web applications. This model lacked systematic and engineering methodology that can cover the whole lifecycle of web applications. Bochicchio and Paiano [26] discussed a design/prototyping environment and presented the definition and features of JWeb (an environment to support design and prototyping of complex WWW sites). They discussed reasons of why an environment supporting design/prototyping is useful such as: support in the early stage; multi delivery; technological flexibility; design advanced options; and design incompleteness. The JWeb system is a design environment with fast prototyping capabilities based on the hypermedia design methodology (HDM) with the following features: it supports design of the hyper-base and access structures; it supports a clean separation between design in the large and design in the small; and it allows building an environment based on a relational DB to hold the multimedia contents. In many cases they used the prototype as initial version of the application to be worked on in 8

order to build up the final version. Sometimes, the prototype could be used as an implementation platform. Finally, we can say that, they didn t discuss design models. Bochicchio and Fiore [27] described an environment (tool) named WARP (Web Application Rapid Prototyping) for the analysis and the rapid prototyping of web applications. This environment offers a set of online software tools, which assist the designer and the user browsing of a web application in all its different aspects. The environment is based on models and techniques used in the hypermedia, information systems and software engineering fields, adapted and mixed. The WARP project proposed a methodology and a development environment for fast prototyping of web applications which support the development cycle from requirement analysis and design of the information and system features till the detailed definition of these schemas and the production of applications (coding, database creation, database filling and creation of interfaces). The process of developing a web application with WARP consists of many steps: collection of requirements; generation of the supporting database; management of the contents; and definition of presentation aspects. After the prototype has created, it must be evolved to create the final web application [27]. Griffiths, et. al. [28] proposed simple web method (SWM) for web engineering. They had built and introduced the prototype WIPSE (Web Integrated Project Support Environment) called PAWS (Project Administration Web Site) that implements some of these whole life cycle activities and to support web engineering. They introduced the following stages and techniques in SWM: project planning; analysis (statement of purpose, audience definition, content analysis and modeling, constraint analysis, and marketing analysis and planning); design (structure/web architecture, page design, visual style, navigation, and metaphor definition); building (preparation, prepare content, build web site, test and evaluate and launch); and maintenance. All of these above efforts offered facilities to speed up design and development of web applications. However, none of them provided an efficient support for the end-to-end process of generation and maintenance of whole web applications. 1.7.2 Literatures related to development methodologies of large web applications A summary and analysis of literatures related to development process of large-scale web applications is given below: Aoyama [29] proposed an agile software development process (ASP) in order to meet the conflicting of delivering software faster, facilitating distributed development and maintaining 9