Software Engineering and Scientific Computing

Transcription

1 Software Engineering and Scientific Computing Barbara Paech, Hanna Valtokari Institute of Computer Science Im Neuenheimer Feld Heidelberg, Germany RUPRECHT-KARLS-UNIVERSITÄT HEIDELBERG

2 AG Software Engineering, Uni HD Profile Quality Engineering Requirements Engineering Rationale Management Quality Assurance Prof. Dr. Barbara Paech Since 7 years in HD before Fh IESE, Kaiserlautern 6 PhD students 2 children Hanna Valtokari Master in Computer Science Worked 7 years as a software developer PhD student since april

3 Who are you? Your name What are you doing at Uni HD? What do you know about and what do you practise in software engineering? What are YOUR biggest problems with software engineering? What do you want to learn about software engineering and scientific computing? 3

4 Schedule Monday 9:00 Introduction to each other, Introduction to Software Engineering 10:30 Break 11:00 Version management concepts 12:00 Lunch 13:00 Incl. a short break Tools, Exercises Version Management Issue Tracking Build Management End 4

5 Software development is complex Programming Software Product Distributed Development Project management Quality assurance Cost, Schedule, Quality Requirements documents Design documents Knowledge management Evolution Version management 5

6 Goals of this course Understand the complexity of software development Know the interests and responsibilities of the project team Know the basic practices of software development Main contents Version management Build management Issue tracking Project management Testing Documentation and Modeling Knowledge Management 6

7 Overview Software Engineering 7

8 Overview 1. Terminology 2. Motivation 3. General Structure 1. What to do? (Activities) 2. What to produce? (Results, Products) 3. Who? (Actors) 4. How? (Methods, Tools, Best Practices) 8

9 What is Software Engineering? Software Software is a collection of computer programs, procedures, rules, corresponding documentation and data (ISO/IEC 12207:2008) Engineering Systematic process and prodcuct Adherence to standards, consideration of quality and cost Usage of models 9

10 [Standish group] SE is difficult! Just 32 % of the projects successful, 25 % without result, 44 % not within schedule Time overrun up to 63%, cost overrun up to 45 % What is important fo success? Management support 18 User involvement 16 Experienced project managers 14 Clear business goal 12 Reduced Scope 10 Standard SW Infrastructure 8 Fixed Requirements 6 Formal Methods 6 Reliable estimation 5 10

11 Joint understanding of all stakeholders As proposed by the project sponsor As specified in the project request As designed by the senior analyst As produced by the programmers As installed at the user's site What the user wanted 11

12 SW is big Enterprise-Ressource-Planning Software R/3 von SAP Year Lines of Code Number components Million (Rel. 3.1.) 30 Million (Rel. 4.5.) 50 Million => Team work Communication Knowledge management Project management 12

13 Rapid Technology Change DaimlerChrysler

14 Quality is difficult Error rates (M. Cusumano, MIT 1990) 1977: 7-20 errors in 1000 LOC 1994: 0,05-0,2 errors in 1000 LOC Increase factor 100 in 17 years But: complexity increase factor 10 in 5 years 0,1 errors means: 18 plan crashes per day money mistransfers per hour 14

15 SW characteristics depend on goals [Weinberg,Schulmann, 1974] Quality in small programs Goal Effort LOC Memory Understandability of the code Effort Understandability of the output LOC Memory Understandability of the code Understandability of the output

16 Quality in big programs Well-known example: Ariane 501 Ariane5 successor of Ariane4-family with over 100 successful starts 6-12t carriage (vs. 2-5t A4) First start

17 Result Rocket destroys itself after a few minutes High damage Carriage lost, cost > 500 M 3 year delay of further missions Investigation committee Report from (only 14 days after!!!) see 17

18 Causes Main problem: conversion error, missing exception handling (programming error). Missing exception handling to save execution time (cost). Un-documented assumptions about value ranges (distributed development). Planned travel route not included in the requirements specification (management). Shut-down in case of errors typical for hardware problems (culture). Unnecessary code copied from A4 (re-use). Copied code not tested (testing, re-use). Missing Review (quality assurance) 18

19 Software Engineering (SE) Development Of big programs With high quality By many team members With cost and time limits using Engineering principles - Planning - Work distribution - Methods and Tools (Standardisation, Quality) - Models to support knowledge management 19

20 SE is a Process actors (WHO) activities (WHAT) results (WHAT) guidelines (HOW) context (HOW) 20

21 Responsibilities Documentation Knowledge management Development Context Requirements Engineering Architecture Design Implementation Version management Quality management Product (Testing, Inspection, Metrics) Process (Metrics, Improvement) Evolution Re-use Enhancement Reengineering Change management Project management Team cost schedule Risks Customer/ Contractor 21

22 Development Decisions Context Business goals Business processes Requirements Engineering Architecture Functionality Quality constraints User Interface Project constraints Technolgy Subsystems Design Software components Interfaces Implementation Programming language 22

23 Quality assurance Context Requirements Engineering Inspection Acceptance test Inspection Usage test System test Metrics Used System Usable System Architecture Design Inspection Integration test Metrics Integrated System Implementation Inspection Component test Metrics Implementated component 23

24 Documentation Context Problem description Contract Acceptance test plan Requirements Engineering Architecture Design Customer requirements Software specification Architecture definition Subsystem specification Component specification Usage test plan Systemtest plan Integration test plan Implementation Code Component test plan 24

25 Modeling Context Text Activity Diagram Requirements Engineering Structured Text, Use Case Entity-Relationship-Diagram Rationale Architecture Deployment Diagram, Component diagram Design Class diagram, Object diagram, Interaction diagram, state diagram Implementation Programming language 25

26 Software Quality ISO 9126/DIN66272 Context Human Work Customer Satisfaction Requirements Engineering Architecture Design Usability Accuracy, Security, Safety Reliability Reliability Changeability Efficiency Changeability Efficiency Implementation Efficiency Portability 26

27 Project Participants Customer SW User Contractor 27

28 Software Engineering Methods In the Past Idee!!! First Algorithms, Data base, Hardware Configuration GOTO oder nicht?? IF THENELSE... User System Design Coding Problem definition, funktianal Analysis System Analyse Modularization, Adaptation of Data base and Configuration, usw. Programming [Yourdon/Constantine 1979] 28

29 Quelle: Software Engineering Methods Today 29

30 Summary SE creates socio-technical Systems SE focuses on quality, cost und effort SE shapes product / system and process 30

31 Literature J. Ludewig, H. Lichter, Software Engineering, dpunkt 2007 L. A. Macaulay, Requirements Engineering, Springer, 1995 I. Sommerville, Software Engineering, Addison Wesley, 2008 G. Weinberg, E. Schulmann, Goals and Performance in Computer Programming, Human Factors 16, p.70-77,1974 E. Yourdon, L.L. Constantine, Structured Design Fundamentals of a Discipline of Computer Program and System Design, Prentice Hall,

32 In this course: Programming in a small team Documentation Knowledge management Development Context Requirements Engineering Architecture Design Implementation Version management Quality management Product (Testing, Inspection, Metrics) Process (Metrics, Improvement) Evolution Re-use Enhancement Reengineering Change management Project management Team cost schedule Risks Customer/ Contractor 32

33 Programming in a small team What is Ron doing? Project management Issue Tracking I want to explain my ideas to Hermione Modeling Knowledge Management I want to change Ginnys code Version management, Build management I want to check Harrys changes Quality assurance Testing 33

34 Version Management 34

35 Terminology Product Line (different products) Releases (product configuration) K1.Vn K2.Vm K3.Vk Configuration K1.V1 K2.V1 K3.V1 Version 35

36 [ Problem #1: Collaboration What if two or more people want to edit the same file at the same time? Option 1: make them take turns But then only one person can be working at any time And how do you enforce the rule? Option 2: patch up differences afterwards Requires a lot of re-working Stuff always gets lost 36

37 The right solution is to use a version control system Keep the master copy of the file in a central repository Each author edits a working copy. When they're ready to share their changes, they commit them to the repository Other people can then do an update to get those changes [ Solution: Version management 37

38 [ When working alone This is also a good way for one person to manage files on multiple machines Keep one working copy on your personal laptop, the lab machine, and the departmental server No more mailing yourself files, or carrying around a USB drive (and forgetting to copy things onto it) This by itself is reason enough to use version control even when you are the only author 38

39 [ Problem #2: Undoing Changes Often want to undo changes to a file Start work, realize it's the wrong approach, want to get back to starting point Like "undo" in an editor......but keep the whole history of every file, forever Also want to be able to see who changed what, when The best way to find out how something works is often to ask the person who wrote it 39

40 [ Solution: Version Control (again) Have the version control system keep old revisions of files And have it record who made the change, and when Authors can then roll back to a particular revision or time (again) This by itself is reason enough to use version control even when you are the only author 40

41 General Features of Version Control Systems Storage of and Search for Versions Change Control Composition of Components (Build Management) Work Distribution (Issues Tracking) 41

42 Storage and Change Control Component Repositoy List or Data base Unique identification Linear: Doc1, Doc2, Doc3 Hierarchical: LSB.BDS.ENT.Doc1 (Project LSB, Component BDS, Phase Design) Document Changes Log change history Save memory: only store deltas to recover changes 42

43 [F.Houdek: Vorlesung Projektmanagement WS2002/2003] Versions Branching possible! 43

44 [F.Houdek: Vorlesung Projektmanagement WS2002/2003] Configurations Branching possible! 44

45 [F.Houdek: Vorlesung Projektmanagement WS2002/2003] Change control => status management Check environment Work environment To be changed To be repaired Repository Being checked Accepted Rejected Production environment 45

46 F.Houdek: Vorlesung Projektmanagement WS2002/2003 Build management Original (z.b. Requirements, Code, Project plan) Derivates (z.b. Object-Files, Executable, Cross- Reference-List) Build procedure (Makefiles, Compiler, etc) 46

47 Work Distribution Define and check access rights Define and check parallel access Element based: a developer can access a certain element whenever s/he wants Role based: a developer can access a certain element whenever s/he performs a certain task (role) 47

48 [ Basic Use Ron and Hermione each has a working copy of the solarsystem project repository Ron wants to add some information about Jupiter's moons Runs svn update to synchronize his working copy with the repository Goes into the jupiter directory and creates moons.txt Ron then: Runs svn add moons.txt to bring it to [Subversion]'s notice Runs svn commit to save his changes in the repository - Repository is now at revision 121 That afternoon, Hermione runs svn update on her working copy [Subversion] sends her Ron's changes 48

49 [ Resolving Conflicts Back to the problem of conflicting edits (or, more simply, conflicts) Option 1: only allow one person to have a writeable copy at any time This is called pessimistic concurrency Used in Microsoft Visual SourceSafe Option 2: let people edit, and resolve conflicts afterward by merging files Called optimistic concurrency "It's easier to get forgiveness than permission" Most modern systems (including [Subversion]) do this 49

50 [ Starvation But what happens if Ginny commits another set of changes while Hermione is resolving? And then Harry commits yet another set? Starvation: Hermione never gets a turn because someone else always gets there first This is a management problem, not a technical one Break the file(s) up into smaller pieces Give people clearer responsibilities The version control system is trying to tell you that people on your team are working at cross purposes If you are doing things right, you will probably never (or rarely) encounter this 50

51 [ Reverting After doing some more work, Ron decides he's on the wrong path svn diff shows him which files he has changed, and what those changes are He hasn't committed anything yet, so he uses svn revert to discard his work I.e., throw away any differences between his working copy and the master as it was when he started Synchronizes with where he was, not with any changes other people have made since then (the base revision, not latest revision in the repository) If you find yourself reverting repeatedly, you should probably go and do something else for a while... 51

52 [ Rolling Back Now Ron decides that he doesn't like the changes Harry just made to moons.txt Wants to do the equivalent of "undo" svn log shows recent history Current revision is 157 He wants to revert to revision 156 svn merge -r 157:156 moons.txt will do the trick The argument to the -r flag specifies the revisions involved Merging allows him to keep some of Harry's changes if he wants to Revision 157 is still in the repository Remember, this affects Ron's local copy, he still needs to commit this undo if he wishes to commit these changes 52

53 [ Summary Version control is one of the things that distinguishes professionals from amateurs And successful projects from failures Everything that a human being had to create should be under version control You'll see the benefits almost immediately You will want to put all your work (even solo work) under version control once you experience the benefits 53

54 Literature Software carpentry ( N. Ford: Produktiv Programmieren, O Reilly,

55 Binary Files Subversion can only merge conflicts in text files Source code, HTML---basically, anything you can edit with Notepad, Vi, or Emacs But images, video clips, Microsoft Word, and other formats aren't plain text When there's a conflict, Subversion saves your copy and the master copy side by side in your working directory Up to you to resolve the differences It's not Subversion's fault Most creators of non-text formats don't provide a way to find or merge differences between files 55