Success Factors to Achieve Excellent Quality

Transcription

1 Success Factors to Achieve Excellent Quality - CMMILevel5OrganizationsResearchReport- Naomi Honda NEC Corporation Page 1

2 Business Domains and Their Chief Products and Services IT Services Cloud-Oriented Service Platform Solutions Platform Super Computer Personal Solutions Carrier Network Server Integrated Operation/ Management Middleware Unified Communication Social Infrastructure Personal Computers Long Term Evolution Network Systems Unity Cable Systems WiMAX Network Systems Compact Microwave Communications Systems Digital Terrestrial TV Transmitters Others Asteroid Explorer "HAYABUSA" (provided by Japan Aerospace Exploration Agency) Mobile Terminals Electron Devices Lithium-ion Batteries Liquid Crystal Displays Page 2

3 Previous study Beyond CMMI level 5 CMMI level 5 doesn t necessarily guarantee excellent quality Beyond CMMI level 5 = Achieving real excellent quality What s the keys to achieve real excellent quality? Benchmarking using process data between CMMI level 5 organizations Superior abilities for defect root-cause analysis QCC: Quality-Centric software engineering Culture The important Idea : The quality is the highest priority in the organization Behaviors of the developers based on the idea Page 3

4 Organization A and Organization B Similar development conditions Business area, Shipment volume Development size Number of engineers (2,000 engineers each) Software process with CMMI level 5 V-model, V & V Development and Management techniques Only organization B had troubled with Large number of post-release defects! Page 4

5 Comparison of the number of post-release defects Organization A before Kaizen Mean for Org. A Before Kaizen = 100 (N = 11) Number of products Organization A after Kaizen (3 years later) Mean for Org. A after Kaizen = 59 (N = 11) Organization B before Kaizen Mean for Org. B before Kaizen = Organization B after Kaizen (3 years later) Mean for Org. B after Kaizen = (N = 8) (N = 7) 0 Page 5 Bette r 199 Post-release defects

6 Kaizen activities Analytical strategy Benchmarking using process data Benchmarking Quality management system Kaizen Activities 1.Reinforcing defect detection during design or code review 2. Increasing the success rate of 1+n procedure 3. Implementation of independent QA testing 4. Quantitative management on a weekly basis using face to face communication Page 6

7 Data items Category No. Data item Unit 1 Total effort Person-hours/KL Effort 2 Design and coding effort Person-hours/KL 3 Review effort Person-hours/KL 4 Testing effort Person-hours/KL 5 Total defect Number of defects/kl 6 Defect during review Number of defects/kl Defect 7 Defect during testing Number of defects/kl Testing item Preventive action 8 Upstream defect detection rate 9 Testing item 10 Success rate of 1+n procedure % Number of testing items/kl % Page 7

8 Descriptive statistics on the data Organization A Organization B No. Before Kaizen After Kaizen Before Kaizen After Kaizen Data item Std. N Mean Deviation N Mean Std. Deviation N Mean Std. Deviation N Mean Std. Deviation 1 Total effort Design and coding effort Review effort Testing effort Total defect Defect during review Defect during testing Test item Upstream defect detection Success rate of 10 1+n procedure Note: 1. All values are summed up for one year of each product. 2. All values are shown using relative values, assuming the mean value for Organization A as 100. Page 8

9 Process data of organization B (before Kaizen) 1.Totalefort 2.Designandcodingefort Low! 3.Reviewefort 4.Testingefort 5.Totaldefect 6.Defectduringreview 7.Defectduringtesting 8.Testingitem 9.Upstream defectdetectionrate 10.Successrateof1+nprocedure 100=meanvaluefororganizationA =10 High! Page 9

10 Experiences in organization A Upstream defect detection rate(%) <Trend chart showing the change of upstream defect detection rate> <Trend chart showing decrease in number of post-release defects> 100% 80% Rate of defects detected while still in upstream processes has become greater than 80% 80% 60% % 20% Number of post-release defects lowered to less than 1/20 0% Thischartshowsrelativeratioagainstthenumberofpost-releasedefectsof1985 Page 10

11 Process data of organization B (after Kaizen) 100=meanvaluefororganizationA 1.Totalefort 2.Designandcodingefort 3.Reviewefort 4.Testingefort 5.Totaldefect 6.Defectduringreview 7.Defectduringtesting 8.Testingitem 9.Upstream defectdetectionrate 10.Successrateof1+nprocedure BeforeKaizen AfterKaizen Note: All values are shown using relative values, assuming the mean value for Organization A as 100. Page

12 Result of defects during review and testing <Comparing of defects during review and testing> Note: All values are shown using absolute values. Defectduringreview Defectduringtesting OrganizationA (before kaizen) OrganizationB (beforekaizen) Upstream defect detection rate is. more than 80% Page 12 OrganizationB (afterkaizen) Early detection more than 80% of defect during design or code review is a key to achieve excellent quality less than 80% almost 80%

13 Lessons Learned 1 Success factors to achieve excellent quality Early detection more than 80% of defect during design or code review Superior abilities for defect root-cause analysis Page 13

14 Kaizen activities Analytical strategy Benchmarking using process data Benchmarking Quality management system Kaizen Activities 1.Reinforcing defect detection during design or code review 2. Increasing the success rate of 1+n procedure 3. Implementation of independent QA testing 4. Quantitative management on a weekly basis using face to face communication Page 14

15 Comparison of Quality management system Item Organization A Organization B Software process V model V model V & V etc. V & V etc. Quality checking through development Based on deliverables Based on process data Independent QA testing for final products Weekly basis Discussing on weekly Project management meeting Not applicable On completion of each process Confirmation in writing Page 15

16 Effects and Lessons learned Kaizen Activity Implementation of independent QA testing Effects 4% of total defects were detected Shipment of defective software products were reasonably postponed Lessons learned Quality assurance from both process quality and product quality has a good effect on reduction of post-release defects. Quantitative management on a weekly basis using face to face communication Problems were timely figure out through development <Example> Checking whether actual value of the review effort reaches the target value Quantitative management with hands-on approach has a good effect on reduction of postrelease defects. Page 16

17 Hands-on approach (Triple Actualities) Actual spot Visiting the location of the trouble Actual object looking at the actual objects there Actual phenomenon Observing what is really happening Instead of sitting at one s desk theorizing! Page 17

18 Behavioral changes in organization B Participants in Quality meeting Before Kaizen Very few words After Kaizen Lively discussion Address of the Top management in the year beginning No words about quality procedure Quality-centric software engineering culture is being built up now! Explaining the importance of product quality Declaration about the quality target Holding of Quality enhancement event Page 18

19 Conclusion Success factors to achieve excellent quality 1. Early detection more than 80% of defect during design or code review 2. Superior abilities for defect root-cause analysis 3. Quality assurance from both side of process quality and product quality 4. Quantitative management with hands-on approach 5. Quality-centric software engineering culture Page 19

20 NEC Group Vision 2017 To be a leading global company leveraging the power of innovation to realize an information society friendly to humans and the earth Established in 2008 Page 20

21 Page 21