Educating Software Engineers to Become Systems Engineers

Transcription

1 Educating Software Engineers to Become Systems Engineers Supannika Koolmanojwong and Barry Boehm Center for Systems and Software Engineering (CSSE) University of Southern California (USC) Los Angeles, CA, , USA {koolmano, Abstract Our two-semester USC core software engineering project course CS577ab devotes its first semester to having students learn and do systems engineering on a real-client project. This requires a good deal of just-in-time lectures, tutorials, and homework to prepare the students, and feedback in terms of mentoring, artifact grading, and live milestone reviews to help them succeed. This paper provides some initial motivation and context; discusses our approach to introduce systems engineering into software engineering relative to that in the GSwE 2009 curriculum guidelines, SEBOK draft 2010, and SWEBOK 2004; describes the course practices during the systems engineering and software engineering semesters; and summarizes the project results and conclusions. 1. Introduction The increasing diversity and interdisciplinarity of software development raises an awareness that software engineers need to deal with more than just software, they need to participate in engineering the system, too. In the past, a software engineer could just sit on the side in an architecture-specification meeting and wait for a hardware engineer or a systems engineer to give them precise specifications to code up. This frequently leads to awkward software constraints, and major risks or disasters in software system development [2]. Recent process guidelines and standards such as the Integrated Capability Maturity Model (CMMI) [5], ISO/IEC for software engineering [11] and ISO/IEC for systems engineering [12] emphasize the need to integrate systems and software engineering processes. Various curriculum guidelines, such as SWEBoK 2004 [18], GSwE 2009 [9], and SEBoK draft 2010 [16], show increasing integration between systems and software engineering. In addition, software engineering and systems engineering have many commonalities such as requirements development, design, integration, verification and validation, testing, and maintenance. Hence, software engineers need to play a stronger role in systems engineering. Thus, it is important to inject the concept of systems engineering into software engineering classes. Our two-semester USC core software engineering project course supports this concept by providing lectures, tutorials, assignments, project activities, tools, and feedback from course staff to equip the software engineering students with systems engineering knowledge and experience. Additionally, we use the Incremental Commitment Spiral Model, which is applicable to both systems and software engineering, as the process model. This paper elaborates on how the systems engineering concepts are embedded and embraced in the software engineering class. Section 2 describes the nature of the software engineering class and the system and software process model. Section 3 reports the similarities between systems engineering concepts and the class materials. Section 4 describes how the course staff helps the students to come up the systems engineering learning curve. Section 5 shows feedback from the students and the clients. Section 6 presents the resulting conclusions.

2 2. Background and Related Work 2.1. The MSCS-SE Core Software Engineering Project Course CSCI577ab [6] is the core software engineering project course within the University of Southern California (USC) s Master of Computer Science with Specialization in Software Engineering (MSCS-SE). The main objective of the course is to prepare students for software leadership careers through the 2040 s. Software Engineering I or CSCI577a in the Fall semester focuses on software-intensive systems engineering, including system operational concept formulation, requirements negotiation and definition, prototyping, COTS and services evaluation and selection, system and software architecture definition, life cycle plans and processes, risk analysis, feasibility analysis, and verification and validation. Software Engineering II or CSCI577b in the Spring semester focuses on software product creation, integration, test, documentation, transition, and maintenance with an emphasis on quality software production. Figure 1. Overview of Software Engineering Project Class Milestones. As shown in Figure 1, before the fall semester starts, the course staff reviews the project proposals submitted by potential clients. If the project will not fit into a 2- semester software engineering class, the proposal will be dropped. This is done by the first milestone, the Exploration Commitment Review (ECR). The second milestone is the Valuation Commitment Review (VCR), which will be conducted after the student teams analyze the clients current business processes and explore alternatives. The third milestone is the Foundations Commitment Review (FCR), which evaluates whether the project is feasible and ready to define various project foundations such as the system and software architecture and the project plans. The Development Commitment Review (DCR) is the fourth milestone which evaluates whether the team has built the right foundations, including the detailed architecture and plans for the Development phase. To accommodate the possible client and team changes over the semester break, the fifth milestone, Rebaselined Development Commitment Review (RDCR) is held to confirm that all the success critical stakeholders are committed to the foundations and that the team is ready to implement the product based on the agreed foundations. A Core Capability Drivethrough (CCD) is held after the first development iteration in order to check the usability and completeness of the core functionalities. The Transition Readiness Review (TRR) evaluates whether the team is ready to perform system and software transition. The Operations Commitment Review (OCR) evaluates whether the clients will be able to successfully operate and maintain the system and software.

3 2.2. The Incremental Commitment Spiral Model The Incremental Commitment Spiral Model (ICSM) [4][14] is a refined version of the original spiral process model [1]. ICSM covers the full system development life cycle consisting of five incrementally-defined life cycle phases (Exploration, Valuation, Foundations, Development, and Operations phases). The ICSM, as shown in Figure 2, is not a single one-size-fits-all model but a risk-driven framework for tailoring a process that best fits a project s situation by using the risk-based decision options at the end of each spiral. The ICSM integrally includes software development, and it is currently used as the process model for system development in our software engineering class. Figure 2. Overview of the Incremental Commitment Spiral Model The four underlying principles of the ICSM are: 1) Stakeholder value-based system definition and evolution The project should be developed based on satisfying the value propositions of all successcritical stakeholders. Otherwise, the stakeholders will frequently not commit to their project roles, which will lead to project rejection or major rework. 2) Incremental commitment and accountability Stakeholders do not commit to a single pre-defined set of requirements and resource contributions, but commit incrementally as the nature of the system is better understood. Otherwise, the project often becomes locked into out-of-date concepts of what the system should provide its stakeholders, leading to project rejection or major rework. 3) Concurrent system and software definition and development Contrary to sequential development, the concurrent development of requirements, solutions, hardware, software, and human factors allows the project to move faster, avoid premature commitments, and be more flexible to yield the best results. 4) Evidence and risk-based decision making Making the evidence of project feasibility a first-class deliverable provides a way to synchronize and stabilize the concurrently-defined system elements. Shortfalls in evidence are uncertainties that identify the level of risk of proceeding without stronger evidence of project feasibility.

4 Table 1. Required skills of a systems engineer Key skills of a systems engineer [9] Identifying the success-critical stakeholders Determining key stakeholders value propositions Helping stakeholders collaborate in negotiating a mutually satisfactory set of specifications Adapting the plans and specification in mutually satisfactory ways to respond to changes Managed Problem Solving Engineering Problem Solving Strategic Problem Solving Characteristics of a successful systems engineer [19] Understanding the system from multiple perspectives Understanding the system from multiple perspectives Understanding a new system/concept during its presentation Understanding the system from multiple perspectives Understanding the implications of proposed changes Understanding systems synergy Understanding limits to growth Understanding the systems without getting stuck on the details Understanding analogies and parallelism between systems Understanding the whole system and seeing the big picture Understanding interconnections; closed-loop thinking Thinking creatively Acquiring systems engineering skills through ICSM Practices Students identify success critical stakeholders such as end users, maintainer, and administrator by using CRACK model (Collaborative, Representative, Authorized, Committed, and Knowledgeable ) WinWin negotiation is the key activity for brainstorming stakeholders win conditions and constraints. Added information is drawn from other client interaction activities such as team meetings, prototyping, business process identification, business case analyses, and acceptance test planning Win conditions are used as a starting point in defining a mutually satisfactory set. Moreover, risk analysis, architecture review board, prototypes, life cycle plan, feasibility evidence development are all key ingredients for software engineering teams to exercise this skill. Weekly project plan updates, change and configuration management, verification and validation, and peer review are examples of activities that allow student teams to plan and re-plan according to the project status This skill can be acquired through various project activities such as risk identification and management, quality management, configuration management, team management, project planning and tracking Student teams are required to apply their engineering knowledge to develop a product that will satisfy the clients requirements. Various activities allow student teams to exercise this skill such as operational concept engineering, requirements engineering, system and software architecting, process engineering, and test engineering To tackle the problems, student teams need to set up a strategy to plan, implement, monitor, and solve problems during their project development. This skill can be achieved through win-win negotiation, analysis of alternatives, business case analysis, life cycle planning, and solution development

5 3. Systems Engineering Coverage in the Software Engineering Class 3.1. Systems Engineering Skills and Problem Solving Approach To prepare students for the systems engineering tasks, students need to learn and acquire various skills through practice. Table 1 provides information about ICSM activities that relate to the required systems engineering skills based on GSwE 2009 [9] and characteristics of a successful systems engineer [19] Systems Engineering Roles By taking roles, students obtain first hand experience in learning and internalizing their systems engineering responsibilities and obligations. Table 2 compares twelve systems engineering roles that are suggested by Sheard [17] to the roles in our software engineering class. Please note that, the twelfth role, a classified ads systems engineer, refers to job description of a systems engineer posted in the classified ads, which could represent any tasks or unrelated tasks that are not in the first eleven roles. Table 2 shows that systems engineering roles and software engineering are fairly compatible. However, there are other roles that are not mentioned in [17] system engineering but appear in software engineering projects, such as COTS or Non-Developmental Item (NDI) evaluator and acquirer. On the other hand, with the limitation of the class schedule, students may play only a partial role in coordinating the transition to client logistics/operations, maintenance, and disposal engineering. Table 2. Systems engineering roles Twelve Systems Engineering Roles [16] Requirements Owner System Designer System Analyst Validation/Verification Engineer Logistics/Operational Engineer Glue Among Subsystems Customer Interface Technical Manager Information Manager Process Engineer Coordinator Classified Ads Systems Engineer ICSM Roles Requirements Engineer / WinWin Negotiation Shaper System Architect / Prototyper / UML modeler Operational Concept Engineer/ Feasibility Analyst Verification and Validation Personnel / Tester Life Cycle Planner/ Transition Coordinator Builder Operational Concept Engineer / Project Manager Feasibility Analyst/ Life Cycle Planner / Project Manager Quality Focal Point Life Cycle Planner/ Feasibility Analyst Project Manager/ Life Cycle Planner COTS-NDI Analyst/Business Case Analyst/Trainer

6 3.3. Systems Engineering Activities, Artifacts and supporting tools As part of the system development, with various supporting tools, students have to produce various artifacts, which will allow them to have hands-on experience in developing core-essential industrial type artifacts. Table 3 shows a complete mapping between system engineering activities [8][9] and the project artifacts produced by the team and the supporting tools provided by the class. Table 3. Project artifacts and tools supporting systems engineering activities Systems Engineering Activities [8] [9] Requirements Analysis System Architecture System Implementation Integration & Verification Transition & Validation Use, Support, Disposal Project management Overall Coordination ICSM Project Artifacts Operational Concept Development System and Software Requirements Description WinWin Negotiation Report Prototype Report System and Software Architecture Description UML diagrams Prototype Functioning system Iteration Plan Iteration Assessment Report NDI-Interoperability Report Agile Review Forms Verification and Validation Report Agile Review Forms Transition Plan Training Materials Release Description User Manual Support Plan Regression Test Package Weekly Project Plan Weekly Progress Report Weekly Risk Identification and Analysis Weekly Effort Report Life Cycle Plan Feasibility Evidence Description Support Information Document Support Information Document Executive Summary ICSM Supporting Tools WikiWinwin Tool Benefit Chain Diagram IBM Rational System Architect irise Subversion Development team s choice Bugzilla istudio Bugzilla Development team s choice Microsoft Project DART Effort Reporting System COCOMO II COCOTS ICSM Electronic Process Guide

7 Class lectures, class activities and project activities also help students to prepare themselves for systems and software engineering roles. Table 4 shows that class and project activities provide full coverage on systems engineering activities. The first column represents system engineering life cycle activities. The second column represents the lectures, readings and assignments that are covered in the software engineering class. The last column represents activities that are conducted in the class. Table 4. Systems engineering topics coverage in software engineering class Systems Engineering Activities [9] Requirements Analysis System Architecture System Implementation Integration & Verification Transition & Validation Use, Support, Disposal Project Management ICSM Topics Coverage Operational Concept Development Stakeholder Benefits and Business Case Analysis Requirement Negotiation and Definition NDI and Legacy System Analysis and Tailoring Object Oriented Analysis and Design Concurrent Increment Development and V&V; Nextincrement Rebaselining Design Patterns Object Oriented Development NDI-Interoperability Traceability Matrix Software Testing Quality Management New-System Cutover, Installation, and Training Design-Code Review User Acceptance Test Quality Management System Transition and Support Evolutionary requirements Regression Test Package Cost Estimation Business Case Analysis Project Management, Team Collaboration Project Contracting ICSM Project Activities Coverage WinWin Negotiation Client/Team Meetings Prototyping Requirements Negotiation Tool Training NDI Evaluation Architecture Tool Training System and Software Architecture Workshop Concurrent Engineering Requirements-Design-Code Peer Reviews, V&V Core Capabilities Drive through Requirements-Design-Code Peer Reviews Integrated Independent Verification and Validation (IIV&V) Core Capabilities Drivethrough Integrated Independent Verification and Validation (IIV&V) Cutover and Training Core Capabilities Drivethrough Transition Coordination Cost Estimation Training Project Planning Training Mentoring sessions Architecture Review Board

8 4. Feedback from course staff Feedback enables the students to learn about their status and to improve their performance. As project progresses, student teams will receive feedback from course staff and other stakeholders in many different ways. Architecture Review Board (ARB) ARB is one of the best practices developed by AT&T, and it is a widely used practice in systems engineering [14]. There are 2 ARBs which mark as a milestone review in each semester. At each ARB, student teams show the project feasibility evidence and project progress in the form of the agreed requirements, operational concepts, prototypes, design and architecture, project plan, business case analyses, test cases, test results, risks and mitigation plan, and strengths, weaknesses of the team. Student teams will present all their work in front of the architecture review board, which are course instructors, clients, teaching assistants, graders, and related stakeholders. The two ARBs in each semester can be compared to mid-term and final examinations. The ARB will check their evidence based on the exit criteria of each milestone and provide feedback to the team for future improvement. Mentorship Mentors are PhD students who took software engineering class and perform research in software engineering. Student teams can ask for additional explanation or suggestions on problems they are facing. With peer-like suggestions, students are eager to ask more questions, hence gain more knowledge and experience in their project development. Workshops Besides lectures, there are special workshops on topics that can be learned better in an interactive session. Students are encouraged to bring in their project artifacts, current problems and concerns. Teaching staff provides feedback or extra support. Focus group Outside the class room, teaching assistants organize special sessions for students to speed up the learning curve for certain roles such as system and software architect, shaper, or feasibility analyst. The discussion in the focus group includes case studies, previous project examples, and exception case discussion. Artifact grading Teaching assistants and graders grade project artifacts and homeworks with detailed explanations. Additionally, the teaching assistants and faculty provide feedback on their core capabilities drivethrough and design-code review results. Verification and Validation Personnel Off-campus students are usually full-time employees and generally have more experience in system and software development. Offcampus students take a project V&V role by evaluating the project artifacts and filing defects and concerns in the defect tracking system, and by virtually participating in ARB reviews. Discussion Board - An online discussion board allows students to post their questions and concerns or discuss class-related topics, and get feedback from course staff, team members and classmates. ICSM EPG The Incremental Commitment Spiral Model Electronic Process Guide [10] is a major source of information for the software development guidelines. The ICSM EPG provides information about each role and responsibilities, tasks, work products, practices, checklist, and artifact templates. Student teams can check their progress or their work based on the guidelines and examples provided in the ICSM EPG. 5. Project Results and Feedback from Students and Clients One way to learn whether the project is successful is to look at the feedback from the stakeholders. Two key stakeholders of the real-client project software engineering class are the students themselves and the clients. Based on the feedback from students in their end-ofsemester reflection-in-action individual critiques, the students are largely very satisfied with

9 the knowledge and experiences they learned from the class, but also helpful in identifying future course improvements. With their role-playing responsibilities and real-life clients, they learn how to determine and handle stakeholders wants and needs from hardware, software and peopleware perspectives and translate them into a successfully operating system. Another most important stakeholder is the clients, who closely participate in the whole development life cycle. They learn to give their win conditions, to compromise to gain mutually satisfactory sets of plans and specifications, and to discover new concepts and knowledge about the system. At the end of semester, we ask the clients about their satisfaction on the team performance in various activities. The results, in Table 5, have shown that the clients are overall very satisfied with the team in all system engineering perspectives. Trends are generally upward, but scores vary from year to year based on course content evolution, client satisfaction thresholds, and student preparedness levels. Table 5. Clients' evaluation ( ) Questions Average (Scale of 5) 1. Operational concepts: How well did the team capture your ideas of what the new system should do? Stakeholders' value propositions: Did the team suggest new idea about your project? Did the team support you in any project complexity? Team responsiveness: Was the team responsive to you and to your requirements? Did the team answer any questions you might have had? How successful was the requirements negotiation between you and your team? Project satisfactory: How satisfied are you with the prototype? How well does the proposed system meet the need identified by your project? Project feasibility: Do you think this project should be carried forward for development in CS577b? Has the team adequately identified and managed the potential risks and complications associated with the project? Do you foresee difficulties in transitioning this project into the development and implementation phase? Team communication: How effective was the team in communicating with you? Do you feel you had enough meetings with them? Did the team make effective use of your time and expertise? What means did you use to communicate? Conclusions A successful software engineer should know not only how to engineer software, but also how to participate in engineering the other system elements. The increasing importance of, and rapid changes in information and software technology make systems-oriented software engineers critical to the success of most future system developments. To design a course for them, it is very important to supply the knowledge, improve their skills, and provide experiences for them to be ready to participate in industry projects on day one. Section 3.1 indicates that students can learn the key skills required for a systems engineer from assorted class activities. Section 3.2 shows the compatible mapping between systems engineering roles and software engineering course roles. Section 3.3 presents the match of

10 key systems engineering activities to project artifacts and tools usage in the software engineering class. Section 3.4 provides evidence that knowledge about systems engineering activities is considerably covered by lectures, presentations, and activities in the software engineering class. Section 4 explains how feedback from teaching staff helps students to acquire system engineering and development skills. Section 5 reports the satisfaction feedback about systems engineering elements from the students and the clients. Generally, our USC software engineering course has integrated the concept of systems engineering, both in the lectures, and in student teams project development activities, artifacts, and supporting tools. However, with the limitation of the class schedule that the semester ends slightly after the students deliver their final product, students usually do not have a chance to fully experience system operation, maintenance and disposal, which are also key activities in systems engineering although clients often hire students to help them evolve the systems that they have developed. 6. References [1] Boehm, B., "A Spiral Model of Software Development and Enhancement," IEEE Computer, Volume 21, Issue 5, May 1988, pp [2] Boehm, B., "Unifying Software Engineering and System Engineering," Computer, Volume 3, Number 3, March 2000, pp [3] Boehm, B., "The Future of Software and Systems Engineering Processes," SSCI Member Forum, 2005 [4] Boehm, B. and J.A. Lane, Using the Incremental Commitment Model to Integrate Systems Acquisition, Systems Engineering, and Software Engineering, Cross Talk, October 2007, pp.4-9 [5] Chrissis, M., B., Konrad, M., Shrum, S.: CMMI. Addison Wesley (2003) [6] CSCI577ab Software Engineering Class Website, [7] Dahmann, J., G. Rebovich Jr., J.A. Lane, "Systems Engineering for Capabilities," CrossTalk Journal, Volume 21, Number 11, pp. 4-9 [8] Dahmann, J., G. Rebovich Jr., J.A. Lane, "System Engineering Artifacts for SoS," IEEE Systems 2010 Conference [9] GSwE 2009 curriculum guidelines [10] Instructional ICSM-Software Electronic Process Guide, [11] ISO (International Standards Organization).: Standard for Information Technology Software Life Cycle Processes. ISO/IEC (1995) [12] ISO (International Standards Organization).: Systems Engineering System Life Cycle Processes. ISO/IEC (2002) [13] Klappholtz, D., J. McDonald, A. Pyster, "The Graduate Software Engineering Reference Curriculum (GSwERC)," cseet, pp , nd Conference on Software Engineering Education and Training, 2009 [14] Maranzano, J., et.al, "Architecture Reviews: Practice and Experience," IEEE Software, vol. 22, no. 2, pp , Mar./Apr [15] Pew, R. W. and A.S. Mavor, Human-System Integration in the System Development Process: A New Look. 2007, National Academy Press. [16] SEBoK 2010, Systems Engineering Body of Knowledge Version [17] Sheard, Sarah A. Twelve Systems Engineering Roles, Proceedings of INCOSE, [18] SWEBoK 2004 Guide to the Software Engineering Body of Knowledge [19] Walden, D., Systems Engineering: Fundamental Skills and Why They are Important, Seventeenth Annual Professional Development Seminar, (St. Paul, MN, USA, March, 2010), The Society of Women Engineers Minnesota (SWE MN), 2010.