Agile Methodologies and Quality Certification

Transcription

1 Agile Methodologies and Quality Certification Keynote speech, XP2003 Michele Marchesi DIEE University of Cagliari Agile Group

2 What is Quality? The totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs. (ISO) In the end, quality is the ability to consistently satisfy customer s expectations, in a way profitable also for the supplier 2

3 Why Software Quality Management? A standard and audited quality system underwrites trust and competence An effective quality system can provide the customer with documented, third party assurance that the supplier is competent So, it can enhance the level of trust that exists between a supplier and a customer Sometimes, this trust is enforced: you cannot get a contract with some organizations if you have no QC 3

4 Quality Standards The software industry has been developing voluntary standards since the early 1960s In 1987, the ISO published ISO 9000 Version 1.0 of the SEI CMM was released in 1991 The ISO/IEC Technical Report series was published in 1998 Others: ISO TickIT BOOSTRAP 4

5 ISO ISO 9000 standards are a set of international quality management system standards and guidelines The basis for establishing quality management systems ISO 9000 identifies the minimal criteria for a quality management system ISO 9000 doesn t address the process of improvement ISO 9000 is not specific for software 5

6 Principles of ISO 9000 DECLARE WHAT YOU DO Standards & Procedures DEMONSTRATE IT Certification DO WHAT YOU DECLARE Records Responsibility RECORD WHAT YOU DID All these principles look very sensible! 6

7 ISO Quality Guidelines Management responsibility Quality system Contract review Software development and design Document and data control Purchasing requirements Customer-supplied products Process control requirements Product inspection and testing Control of inspection equipment Inspection and test status of products Control of nonconforming products Corrective and preventive action Handling, storage, and delivery Control of quality records Internal quality audit requirements Training requirements Servicing requirements Statistical techniques 7

8 SEI CMM Software-specific CMM identifies the need for continuous process improvement 5 LEVELS LEVEL 1 INITIAL LEVEL 2 REPEATABLE LEVEL 3 DEFINED LEVEL 4 MANAGED LEVEL 5 OPTIMIZING KEY process areas KEY practices 8

9 ISO/IEC (SPICE) Provides a common framework and language for software process assessment reference model (processes and process capabilities) rating process capabilities Specifies requirements which must be met in order for an assessment to be considered conformant reference model compatibility assessment process Defined on a six point nominal scale (0 to 5) 9

10 The truth about quality certification! SQC audits and best practices have been first applied to very big projects, and retain their culture SQC principles are good and acceptable In practice, though, SQC certification is very heavy Auditors require a lot of documentation, meetings and records that may hinder the process The driving force behind efforts to develop a QS is certification itself, not the perception that currently existing processes need to be improved. Many certified software firms actually do not follow their certification 10

11 Testimonial of a developer (from c2.com Wiki on XP) I've only seen a few CMM audits, but the ones I've seen have been uniformly an exercise in deception: No way was the development team doing what team leaders told the auditors! When a more through audit was expected, team leads coached the developers as to what to tell the auditors. This, is real life. As I see it, CMM audits have generally been something "top management" subjected teams to, to further some apparently unrelated political objective. 11

12 Testimonial of a manager (personal communication) Software development owes a lot to SQC. In fact, developers become much smarter, since they have to devise continuously new ways to overcome and cheat the Quality System, to be able to continue doing what they were accustomed to do. 12

13 ISO 94 issues Too many standards in the series ISO 9001:1994, ISO 9002:1994, ISO 9003:1994 Too many elements (20) not well structured Standard too related to manufacturing Too much emphasis on documentation rather than results Customer satisfaction is not specifically addressed Continual improvement is not specifically addressed 13

14 An answer: ISO The number of guidelines has been reduced ISO is easier to use for services and small-medium size organizations From a procedurally based approach to management (HOW you control your activities) to a process based approach (WHAT you do) MORE Emphasis on: Continual Improvement Customer satisfaction 14

15 ISO Principles Customer focus Leadership Involvement of people Process approach System approach to management Continual improvement Factual approach to decision making Mutually beneficial supplier relationships Many of them look like the Agile Manifesto principles! 15

16 Customer focus ISO It s important to: Understand customer needs and expectations Manage customer relationship Link customer needs with objectives of organization Measuring customer satisfaction Communicate customer needs through the organization Agile Methodologies : Our highest responsability is to satisfy the customer through early and continuous delivery of software Welcome change requirements, even late in development. Agile processes harness change for customer s competitive advantage 16

17 Leadership ISO It motivates, encourages,inspires and recognizes people s contribution (employees, suppliers,owners, customers) considering all they needs Establish a clear vision of the organization (actual and future) Establish trust and eliminate fear Improve communication between levels of organization Agile Methodologies: Business people and developers work together daily throughout the project Technical people need very close contact with business managers Communication is fundamental in an adaptive development where things change quickly 17

18 ISO Involvement of people People are the essence of an organization They have to be motivated and involved within the organization People accepts ownership of problems and responsability for solving them People share knowledge and experience and improve their competence Agile Methodologies: Build projects around motivated individuals, give them the enviroment and support they need and trust them to get the job done The most efficient and effective method of conveyng information with and within a development team is face-to-face conversation Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a costant peace indefinitely 18

19 Continual improvement ISO It s important to make continual improvement of products, services, processes, activities and system. This brings the organization to a more mature level of flexibility to react quickly to new requirements and opportunities Agile Methodologies: The best architectures, requirements and designs emerge from selforganizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjust its behavior accordingly Continuos attention to technical excellence and good design enhances agility 19

20 Factual approach to decision making ISO All effective decisions are based on the analysis of data and information Agile Methodologies: Working software is the primary measure of progress In XP: the Planning Game drives the project according to estimates of project velocity In XP: the percentage of passed acceptance tests denotes the advancement of the project. 20

21 However The problem of quality certification for software firms using XP or another AM remains While the principles of AM are very similar to those of recent quality standard, the implementation of these standards still requires more than simply saying that we follow XP In the last part of this talk, we ll briefly review some issues and possible answers We ll specifically refer to XP 21

22 Recall ISO 9000 DECLARE WHAT YOU DO Standards & Procedures DEMONSTRATE IT Certification DO WHAT YOU DECLARE Records Responsibility RECORD WHAT YOU DID 22

23 XP and ISO 9000 Declare what you do: No problem to declare XP process and practices Do what you declare: Sincerity is key to all XP values Record what you did: XP mainly does it in the code. For ISO 9000, clearly it is not enough! Demonstrate it: Obtain certification for an XP shop. This is the problem! 23

24 Records are the key issue! User requirements (user stories and acceptance tests) must be recorded. A requirement document for non-functional reqs. should also be provided PG meetings must be verbalized and recorded Task cards should be recorded Unit/Acceptance tests runs must be periodically recorded A CMS should record the story of system coding 24

25 How to record all this information without hindering the process? Make use of automated tools for user stories and project tracking Such tools should have pretty-printing capabilities Such tools could also be used for gathering metrics, including time-dependent ones, on the project Gathering metrics, to be used for process improvement, is mandatory for many new standards! Add to X-Unit the capability to record and print test results Use tools like Together and Javadoc to extract documentation from the code 25

26 Who should be in charge of that? XP was created to be lightweight to programmers, and puts extra work on the shoulders of other people The tracker could be in charge of record management and project tracking All should be made in an agile way, minimizing the recorded documentation and being able to easily access it, when needed 26

27 Possible tasks of the tracker Record US, AC, TC using an automated tool Write a short requirement doc and a data dictionary of the system Write a short architectural doc of the system (from CRC analysis and from the code itself) Periodically record UT/AT runs Manage CMS and integration records Keep minutes of meetings Collect process and product metrics 27

28 Another example: XP and CMM L2 Following H. Glazer, most XP projects that truly follow the XP practices could be assessed at CMM Level 2 if they could demonstrate having a process for the following : Ensuring that the XP Practices are taught to new developers on the project. Ensuring that the XP Practices are followed by everyone. Escalating to decision makers when the XP Practices are not followed and not resolved within the project. Measuring the effectiveness of the XP Practices. Providing visibility to management via appropriate metrics from prior project QA experience. Knowing when the XP Practices need to be adjusted. Having an independent person doing the above. 28

29 Conclusions XP and AMs are closer to quality standards, and in particular to the most recent ones, than heavyweight software development processes To obtain certification, however, some extra burden must be imposed to a classical XP process like that of the C3 project These quality-related activities should be made by an independent person (the Tracker?) Having (synthetic and up-to-date) requirements document, data dictionary, and architecture document could also be useful for many teams Metrics gathering on the process and the software are more and more important 29

30 References Mark C. Paulk, Extreme Programming from a CMM Perspective, IEEE Software vol. 18, No. 6, 2001 Hillel Glazer, Dispelling the Process Myth: Having a Process Does Not Mean Sacrificing Agility or Creativity, CROSSTALK The Journal of Defense Software Engineering, November J. R. Nawrocki, M. Jasiñski, B. Walter, and A. Wojciechowski, Combining Extreme Programming with ISO 9000, Lecture Notes in Computer Science, vol. 2510, p. 786 ff., Springer,