Open Source Software Quality Assurance - Lessons Learnt

Size: px

Start display at page:

Download "Open Source Software Quality Assurance - Lessons Learnt"

Jesse Wilkins
8 years ago
Views:

1 1 Open Source Software Quality Assurance - Lessons Learnt Imed Hammouda, adjunct professor Alexander Lokhman, researcher Tampere University of Technology Credits to Prof. Ernesto Damiani, University of Milan, Italy

Lokhman, researcher Tampere University of Technology

2 Definitions Software Assurance the level of confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at anytime during its lifecycle, and that the software functions in the intended manner National Information Assurance Glossary Software Quality The degree to which a system, component, or process meets specified requirements. The degree to which a system, component or process meets customer or user needs or expectations. IEEE Standard Glossary

manner National Information Assurance Glossary Software Quality The degree to which a system, component, or process meets

3 Software Quality Attributes

4 Traditional Assurance Tasks - Security Security requirements Abuse cases External Review Risk analysis Risk-based Security Tests Code Review (Tools) Risk analysis Penetration Testing Security Operations Requirements and Use Cases Architecture and Design Test Plans Code Tests and Test Results Feedback

Risk analysis Penetration Testing Security Operations Requirements and Use

5 FLOSS Development Contributor Committer Trusted Repository Bug reports Source code Distributor User

6 Multi-Tiered Assurance Process of FLOSS

7 Quality by Access Availability of source code from an easily accessible medium No authorization is required Source code is made available in a format that is suitable and workable for the purpose of review and development, and appropriate for free distribution "Given enough eyeballs, all bugs are shallow" Linus s Law

suitable and workable for the purpose of review and development, and

8 Quality by Development Precise and explicit understanding of the software goals and requirements Choice of methodologies for testing, debugging, and error and bug reporting Tools to provide effective communication, coordination and the overall management of the project Facilitation of rapid frequency of beta releases Release early, release often

reporting Tools to provide effective communication, coordination and the overall

9 The Role of Community Maemo Karma Product Karma Features/components are rated by contributors (+1/-1) A certain karma level has to be reached for the component to be promoted Member Karma Comments, commenting a news item or a Maemo app in testing: 2 * sqrt(# comments) Favourites, voting on news items: 0.25 * sqrt(# thumbs) Package testing, promoting or demoting packages in Extras Testing: 0.5 * # thumbs Blog post, 1-10 per post depending on votes it has received Products, applications in Maemo Downloads: 7 * (1 + stars per application) Discussion, posts to Maemo mailing lists: 2 * sqrt(# posts) Brainstorm: 5 for idea under vote or in development, 10 for implemented idea, 2 for solution under vote or in development, 10 for implemented solution Groups, 3 * # garage project memberships Bugzilla_reported: 6 * sqrt(# bugs reported) Bugzilla_comments: sqrt(# bug comments) Itt_thanks, thanks for TMO posts: 8 * sqrt(# thanks) Itt_posts, posts on TMO: sqrt(# posts) Mediawiki_edits: 6 * sqrt(# edits)

5 * # thumbs Blog post, 1-10 per post depending on votes it has received Products, applications in Maemo Downloads: 7 * (1 + stars per application) Discussion, posts to Maemo mailing lists: 2 *

10 The Role of Tools [

11 The Role of Reviews Projects go through six distinct phases All Projects are required to have at least one review per year. [

12 The Role of Processes Contribution Untrusted Trusted Under Test Tested Repository User

13 The Role of Version Control Systems Central Repository Repository checkout commit checkout checkout User A patch patch User C User B Centralized Version Control User C pull User D User A pull pull pull commit maintain User B Distributed Version Control

patch User C User B Centralized Version Control User C pull

14 The Role of Forges Infrastructure for collaboration and communication Maintaining project data and statistics Requirements for hosting projects Visibility of projects to other communities Access to a database of past experience

Requirements for hosting projects Visibility of projects

15 The Role of Conventions and Guidelines Naming Conventions - How to name things like packages, classes, and methods Coding Conventions - How to make source code readable Documentation - How to write documentation comments, especially for API User Interface Guidelines - How to achieve user interface consistency Version Numbering - How to evolve plug-in version numbers

Documentation - How to write documentation comments, especially for API User Interface

16 Checklist Objectives Critical success factors Roles and responsibilities Plans Technical architecture and good practice Risks OSS challenges User-related Technology-related

17 Thank You! Questions?

ECLIPSE FOUNDATION, Inc. INTELLECTUAL PROPERTY POLICY Effective as of July 21, 2011 (the Effective Date )

ECLIPSE FOUNDATION, Inc. INTELLECTUAL PROPERTY POLICY Effective as of July 21, 2011 (the Effective Date ) 0. DEFINITIONS Content is copyrightable material, including without limitation software, documentation,