Modern practices TIE-21100/

Transcription

1 Modern practices

2 Today s lecture Learn what some modern SW engineering topics are about A peek to some research topic of our department 2

3 3

4 4

5 5

6 6

7 How the lectures continue? Modern practices Continuous Integration, Continuous Deployment, DevOps, No lectures - exam week 17.3 Improving Quality: review practices, testing and quality assurance Quality in general; Quality management systems 23.3 Dependable and safety-critical systems 30.3 Other methods; RUP, XP, Lean; Kanban. 6.4 Easter - no lectures 13.4 Role of software architecture; software maintenance; software evolution 20.4 Software business, software start-ups 27.4 Guest lecture Guest lecture 2 7

8 General matters About the tools All information should be on the Web page (if something is missing, please report me) Canvas is a collaboration tools s (POP-messages) are for quick updates and reminders Sprint reviews Evaluation is based on several factors (overall management, scrum process, implemented user stories) Weekly exercises We had some technical problems that delayed finalization of Jenkins exercise => postponed to 16.3 This week we look at Traditional, Agile and Lean through a scientific article 8

9 Agenda for today Recap of the effort estimation Global SW development And related research Optimized pipeline from requirements to use feedback And related research 9

10 10 STILL ABOUT EFFORT ESTIMATION

11 Estimates improve as the project progresses Picture 12.8 in Haikala&Mikkonen, 23.9 in Sommerville 11 Prestudy Early prototyping Early design Requirement specification Architecture design Post architecture Detailed design Implementation Real value

12 Effort estimation in Scrum hours Planning Initial estimation Sprint planning Sprint backlog Sprint 2-4 weeks Go / no-go decision Resource allocation

13 Effort estimation 13 Necessary for planning: timetable and budget Factors Complexity of the software often hard to know in advance Productivity of the team Skills Experience Spirit Required quality Timetable (recall previous slides)

14 14 GSD - GLOBAL SOFTWARE DEVELOPMENT

15 Reasons for distributed development 15 Size of the project One team cannot finish in time Not enough resources available otherwise Need for additional competences HW, databases, UI Domain knowledge Business In some places development is cheaper Presence close to customer needed for the business

16 Global SW development 16 SW developed by a geographically distributed organization Reasons like above Often main reason is lower salary costs Also called off-shore development Real savings debated

17 Business set-ups 17 Scale own development By resources from foreign company Buy a project Handle management

18 GSD problems 18 Communication distance: (timezone, language cultural..). Serious issue since communication is major challenge in SW Spirit, motivation, committment They and us attitude No feeling of ownership

19 Guidelines for planning distributed projects 19 Design for partitioning Or it will happen anyways Conway s law: organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations Facilitate communication Add extra effort for Communication Solving problems due to bad communication in your plan

20 How to use Genetic Algorithm to propose planning of a global SW project Work packages & their relations Teams & their relations GA Work distribution Scheduling 20 12/09/2014

21 For our algorithm we need a GSD Work Distribution Model Skill -area -level * * Available resources Team -cost -capacity -performance 0..* Communication distance * Slot Component -estimated effort 1..* * * System requirements 21 System information -skill Responsibility 12/09/2014

22 Genetic Algorithms Genetic Algorithms (GA) are adaptive heuristic search algorithms inspired by the natural selection and genetics. The system runs through several generations A solution is encoded as genes and altered through mutations New solutions found also by combining parts of two existing solutions (parents) The best individuals survive from each generation. For a more detailed description see the paper and its references /09/2014

23 12/09/2014 Genetic algorithm simplified! 23 A B C D A B C D A B C D A B C D A I C D A B E D A H C D O B C D A I C D A B E D A H C D O B C D A I C D A H C D A I E D A H C D A I C D A B E D A H C D O B C D Mutations Selection

24 Applying genetic algorithm to work distribution and scheduling in GSD (1/3) Team and WP characteristics encoded in genes Initial work distribution and schedule Genetic Algorithm Result Set of optimal work distributions and schedules - Change team - Change development order Mutations Cost Duration Fitness function 24 12/09/2014

25 Applying genetic algorithm to work distribution and scheduling in GSD (3/3) Crossover exchanges genes between two parent chromosomes The fitness function evaluates the goodness of the solution (individual in a generation). Two factors: cost and lead-time determine the fitness Pareto optimality approach is used for selecting the survivors In practice we seek points where improvement of one criteria (e.g. cost) can only be optimized by compromising another (e.g. time). Cost Time 25 12/09/2014

26 Results London is faster but Bangalore is cheaper. Using the Pareto fronts, the project manager can analyze different work distributions for a given project based on available cost and time /09/2014

27 Time vs Cost 27 Cost Time is expensive! Minimum cost Time Minimum time

28 Partitioning of the project 28 Impossible to partition Easy to partition Cost/people

29 29 CONTINUOUS INTEGRATION, DELIVERY, DEPLOYMENT

30 Continuous integration ( 30 Suggested by many agile methods, for example XP. The team integrates latest changes to complete system frequently even several times a day. Often combined with test-driven development Benefits Reduced risk (and earlier discovery) Helps in getting rid of bugs Avoid chaos at the end (integration hell) Changes behavior of programmers, more carefully written and simpler code

31 Practices of continuous integration 31 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment

32 Continuous integration, when not practical 32 Target platform is too different for example embedded SW System is big, and building takes long (e.g. hours) => Daily builds (often nightly builds) By agreed time of the day every developer needs to deliver a functional version of his/her component Build server compiles and links the system Automated tests are run and/or system is sent to test team

33 Continuous integration 33 Version mgmt 1. Check in 4. Check out Build servers App Developes 2. Signal change 3. Init build 5. Report results Other stakeholders 6. Feedback CI server

34 Figure in Sommerville Version Management Private Workspace fail Checkout Mainline Build and Test System Make Changes Build and Test System Check-in to Build Server Build and Test System Commit changes Build Server Version Management 34

35 Practices of continuous integration 35 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Limit number of branches

36 Practices of continuous integration 36 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Manage the dependencies for the source code Builds are based on a schedule or triggered by source code changes Possibly run the build as distributed processes Generating build notes and other documentation Reporting the build status.

37 Practices of continuous integration 37 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Challenges: increased work effort needed to create the automated tests need to test user interaction or external interfaces that are difficult to automate the amount of processing power needed to run the tests repeatedly.

38 Practices of continuous integration 38 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment NOTE: this is not always possible Prevents potential problems or conflicts to be hidden in the local copies of the software. The rule of thumb in CI is to commit software changes every day. In practice it is often beneficial to commit even more frequently.

39 Practices of continuous integration 39 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Potential solutions to the problem: Staged build or build pipeline Everyone Commits To the Mainline Staged Every testing Day Distributed build and testing Raw processing power. Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment

40 Practices of continuous integration 40 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Do the best you can

41 Practices of continuous integration 41 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build Even the Mainline managers! on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment

42 Practices of continuous integration 42 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Status displays in separete and public screens (but maybe not in cafeteria)

43 Practices of continuous integration 43 Maintain a Single Source Repository. Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build Discussed the Mainline later today on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment

44 An example tool for CI: Jenkins 44 Very commonly used open source tool Run build (e.g. Ant) and automatic test (e.g. Junit) scripts Integrates with very many version management systems Can be triggered automatically by version management Can be triggered by sending Build running in batch mode users can see the status Let look at tutorial at: There will be a weekly exercise (16.3)!

45 Continuous delivery and deployment ( 45

46 A/B Testing 46 Should our project have A or B? Implement a way to collect statistics Usage statistics Compare Usage statistics Implement B Implement A Deploy B Deploy A

47 Background info what is DevOps 47 DevOps is about faster deployment of new innovations by combining technology, culture, practices and processes Some factors (sources: wikipedia, Use agile methodologies Demand for an increased rate of production releases Cloud infrastructure from internal and external providers Unified tools / unified processes Business need Development Operation Agile DevOps See: Marketing video: (Nicole Forsgren - DevOps and the Bottom Line)

48 Summary of the demo Integrated pipeline from coding to deployment and operation Edit Build Test Deploy 48 Operate The developer just needs a browser. No SW installation needed. Deployment to brokered cloud: system finds the most suitable provider and deploys there

49 App Developer Architecture that delivers from the developers to users 49 Develop code (using web-based IDE) App users Commit Source code repository Build Test Public or Private PaaS runtimes Package Project configuration Reporting Cloud-based Continuous Integration COAP S Deploy App

50 50

51 Concrete example Need-For-Speed How to ensure fast pipeline for continuous deployment? We have worked on real data/projects/companies Does the agile process work as it should? Nature of changes, changing role of developers? Project mgmt, Issue tracking Codin g Version mgmt Build Testing Deployment Use Are there particular reasons for problems (delays) Are right tests done in right place to maximize speed? 51

52 Example visualization 52

53 Material 53 Martin Fowler: Continuous Integration Ashish Kumar: Development at the Speed and Scale of Google (Presentation + Video) Google#.TuYqU_kvSuI. Tero Piirainen: Continuous Delivery or: How I Learned to Stop Worrying about Releasing and started to Love my Customers Keynote by prof Jan Bosch, Web page of need for speed project