Course Information. 1. Application Guidelines and Regulations. 2. Course Objective. 3. Learning Outcomes

Transcription

1

2 Specification Author: Course Information Nikita Nalyutin, Associate Professor, PhD (Science and Technology Studies: Software Engineering) Subject Title in English: Software testing and quality assurance 1. Application Guidelines and Regulations This specification presents a detailed description of the educational purpose, curriculum, and assessment methods for the discipline Software testing and Quality Assurance. This course is delivered to students of master s program Software Engineering, computer science faculty, HSE. The specification is intended to be used as a source of information by: students and potential students; lecturers delivering lectures and conducting practical classes on the course or related disciplines; professional and statutory regulatory bodies when carrying out accreditation. 2. Course Objective Primary course objective is to develop student professional competences in software testing and verification techniques and to give them practical skills for software testing tools usage. Course topics set is based on enterprise testing approaches and contains topics on test environment managements, test planning, coverage analysis and improvement, integration and load testing, UI testing. Also, course covers project documentation verification and formal inspection process. At the end of the course the overview of Quality Assurance process is given. During the course, the students will: 3. Learning Outcomes Study the basic terms, definitions and principles of software testing and quality assurance; Study state-of-the-art technologies and products for software testing; Study regulatory documents in the area of software testing; Master methods and tools for development of robust test cases for different types of software systems; Master industrial standards for software quality assurance; Master models of testing process in terms of lifecycle phases, role based project structure, test effort management, test coverage.; Acquire practical skills in test analysis and test development; Acquire practical skills in test management and test planning; The course contributes to the development of the following professional competencies [3]: 2

3 1. Research activities The ability to manage both self-paced and team work on research projects (ПК-3). 2. Project activities The ability to approach projects in a systematic way, to build and apply descriptive and forecasting models, to conduct qualitative and quantitative analysis (ПК-6); 3. Technical activities The ability to apply state-of-the-art development technologies for the development of complex software systems, to take the advantage of automated job scheduling and control tools, and to make use of quality assurance techniques (ПК-9). Curriculum: 4. Discipline in the Educational Program The course length is 80 academic hours, including 20 hours of lectures, 20 hours of practice, and 40 hours of self-study. Academic control forms are one home assignment checked at the final practical seminar. It is a part of specialized curricula unit, and it is delivered in module 1 of the first academic year. Number of credits is 3. Prerequisites: The course is based on the knowledge of foundations of general technical disciplines, mathematics, computer science and fundamentals of the decision theory. 3

4 5. Subject Structure and Contents No Topic name Software testing techniques and approaches. Testing in software development life cycle. Test analysis and test development. Test coverage. Repeatability of testing. Regression testing Course hours, Total Audience Hours Lectures Practical studies Self-study Test process documentation: from test strategy to test coverage reports. 4. Formal inspections 5. Unit testing Integration testing User interface testing 8. System testing 9. Robust software development techniques 10. Enterprise approaches for test process organization. Quality Assurance. Total: Grading and Assessment Type Form 1 year Notes Progress check Homework assignment * 1) Research paper 2) ISTQB mock tests 3) Defect reports case study Final check Oral exam Talk on research paper topic and several ISQTB-like tasks Homework assignment 6.1. Evaluation criteria 4

5 Homework consists of 3 parts: 1) Research paper development and analysis. The students select the subject based on their interests. Once approval of the topic is obtained from course instructor, the student writes the research paper based on the information from public sources, personal experience and imagination. 2) Passing of 2 mock tests for International Software Testers Qualification Board (ISTQB) certification. Link to the tests is given by course instructor. 3) Preparation of several defect reports for Web sites selected by students. Evaluation criteria for student s homework assignment are presented in the table below. A maximum score of 10 can be achieved. Development research paper Relevancy of information Real-life story basis Structure of text Sufficiency of selected material Freshness of ideas Accuracy of problem statement Author s contribution and analytical processing of information from public sources Analysis of research paper Demonstration of good command of theoretical knowledge Ability to identify both common and specific problems Exploring various solutions Openness to different perspectives Being persuasive in argumentation Application of risk management principles and decision making methods Oral exam Final exam is in the form of homework review. Student obtain 3 questions on research paper topic, 2 tasks similar to ISTQB test and 2 questions on defect report. Evaluation criteria for student s homework assignment are presented in the table above. A maximum score of 10 can be achieved. Penalties Should plagiarism be identified in the student s homework, disciplinary measures are applied as appropriate per the HSE Charter Overall Score Overall score on the course Ofinal is determined using the following formula: Ofinal = (Ohome<4 Oexam<4)? min (Ohome, Oexam): 0,5*Ohome+0,5Oexam where - Ohome- score achieved by the student for the homework assignment; - Oexam - score achieved by the student for the written exam. Should the student fail to either pass the written exam or submit the case before the end of the course, overall score for the next attempts is determined using the following formula: Ofinal = (Ohome<4 Oexam<4)? min (Ohome, Oexam): 0,8*(0.5Ohome + 0.5Oexam) 5

6 A score of 4 or higher means successful completion of the course ( pass ). A score of 3 or lower means failure to complete the course ( fail ). 7. Detailed Curriculum Plan Topic 1. Software testing techniques and approaches. Testing in software development life cycle. 1. Software development life cycle 2. Life cycle models Waterfall Spiral Prototyping Rapid 3. Software development process frameworks: Microsoft Solutions Framework Rational Unified Process extreme Programming Agile Scrum 4. Roles in software development team, role interaction for different processes 5. Goals of software testing. Difference between testing and quality assurance. Difference between testing, verification and validation. 6. Types of testing process and test integration into the lifecycle: Unit testing Integration testing System testing Load testing Static testing techniques 7. Testing of mission-critical software and the difference with business software. Topic 2. Test analysis and test development. Test coverage. Repeatability of testing. Regression testing. 1. Test approaches: Black box White box Model checking Static checking 2. Test environment Drivers and stubs Testing classes Signal generators 3. Test cases Requirements based test case development Explorative test case development 6

7 Equivalence partitioning Test plan Test traceability Test execution statistics 4. Test coverage Requirements and code coverage Types of code coverage: lines, branches, condition, decision, condition/decision, modified condition/decision. Equivalence partitioning Boundary conditions Robustness checks Coverage completeness checks 5. Repeatability of tests in enterprise environment, regression testing 6. Test pre-condition, test dependencies, test environment management, types of test environments Topic 3. Test process documentation: from test strategy to test coverage reports. 1. Test processes and test documentation for each process. 2. Test strategy 3. Test plan 4. Test requirements. Traceability, completeness, consistency, testability. 5. Test scenarios. Types of test scenarios (sequences, checklists, tables, automata). Test generators. 6. Test execution reports. Types of test execution reports. Execution reports analysis. 7. Coverage reports. Types of coverage reports. Coverage reports analysis. 8. Defect reports. Types of defect reports. Defect reports analysis, release planning based on defect reports. Topic 4. Formal inspections. 1. Types of static techniques. Types of project reviews. Difference of formal inspection from other types of review. 2. Formal inspection roles, life cycle 3. Code formal inspection: role setting, preparation, results documentation 4. Requirements formal inspection: role setting, preparation, results documentation Topic 5. Unit testing. 1. Goals of unit testing 2. Unit boundaries 3. Test environment architectural approaches 4. Data for unit testing 5. Unit testing repeatability, continuous integration 7

8 Topic 6. Integration testing. 1. Goals of integration testing 2. Integration testing approaches: Early integration Big bang integration Layer-based integration Hierarchical integration 3. Module interface testing 4. Requirements limits, testing missing integration requirements. Topic 7. User interface testing. 1. Goals of user interface testing 2. User interface testing approaches: manual testing, scenarios, event simulation 3. Interface testing automation, test supportability 4. Completeness criteria for user interface testing Topic 8. System testing. 1. Goals of system testing 2. System testing planning 3. Test environment set-up 4. Load testing 5. Mission critical software system tesing Topic 9. Robust software development. 1. Critical points 2. Assertions 3. Exception handling 4. Automated code robustness analysis Topic 10. Enterprise approaches for test process organization. Quality Assurance. 1. Quality assurance. Goals. Quality management (ISO 9000, CMMI). Quality audits. Corrections and corrective actions. Process optimization 2. Configuration management. Goals. Identification. Version control. Life cycle control. Traceability. Baselines. 3. Enterprise-level quality management 8

9 Sample topics: 8. Assessment Methods Research paper 1. Pros and cons for explorative testing and scenario-based testing 2. Pros and cons of waterfall vs. agile, testing in these lifecycles 3. Project risks for software testing 4. Configuration management for software testing 5. Testing of high-load internet sites 6. Test tool selection strategy for software projects 7. Testing vs. quality assurance 8. Context testing school 9. Test effort estimation techniques 10. Move from manual to automated testing: benefits and problems 11. Test team management: goals, tasks, risks 12. UX and usability testing techniques 13. Regression testing as project health indicator 14. Test end criteria and their impact to software quality 15. Test effectiveness estimation 16. Types of test coverage 17. Test documentation flows 18. Testing without requirements - pros and cons 19. Test stragegy 20. Outsourced test management 21. Project risks for outsourced testing 22. Mobile applications testing Assesment: Evaluation of research paper text as a homework is performed using academic-wide set of criteria to evaluate research papers (completeness, innovation, original material etc), giving maximum of 6 points to Ohome. On oral exam student should answer to 3 questions based on his research paper topic, giving maximum 6 points to Oexam (2 points each question). Mock tests Samples of mock tests questions: 1) An input field takes the year of birth between 1900 and The boundary values for testing this field are: A. 0,1900,2004,2005 B. 1900, 2004 C. 1899,1900,2004,2005 D. 1899, 1900, 1901,2003,2004,2005 9

10 2) Pick the best definition of quality A. Quality is job one B. Zero defects C. Conformance to requirements D. Work as designed Assesment: During the homework student should pass minimum 2 mock tests giving him up to 4 point (2 points each test) to Ohome. On oral exam student should answer to 2 questions similar to questions in mock test, giving him up to 2 points to Oexam. Defect reports During the homework student should prepare minimum 2 defect reports for his favorite Web site, giving him no extra points to Ohome. On oral exam student should answer to 2 questions on his defect reports, giving him up to 2 points to Oexam. 9. Learning Resources (in Russian and English) 1. Блэк Р. Ключевые процессы тестирования. Планирование, подготовка, проведение, совершенствование. М.: Лори, // Rex Black. Critical Testing Processes: Plan, Prepare, Perform, Perfect 2. Савин Р. Тестирование Дот Ком, или пособие по жестокому обращению с багами в интернет-стартапах. М.: Дело, Липаев В.В. Тестирование компонентов и комплексов программ. М.:Синтег, Криспин Л., Грегори Д. Гибкое тестирование. Практическое руководство для тестировщиков ПО и гибких команд. М.:Вильямс, // Lisa Crispin and Janet Gregory. Agile Testing: A Practical Guide for Testers and Agile Teams 5. Бейзер Б. Тестирование черного ящика. Технологии функционального тестирования программного обеспечения и систем. СПб.:Питер, // Boris Beizer. Black-Box Testing: Techniques for Functional Testing of Software and Systems 6. Канер С., Фолк Д., Нгуен Е.К. Тестирование программного обеспечения. Фундаментальные концепции менеджмента бизнес-приложений. М.:ДиаСофт, // Cem Kaner et al. Testing Computer Software, 2nd Edition 7. Дастин Э., Рашка Д., Пол Д. Автоматизированное тестирование программного обеспечения. М.:Лори, // Elfriede Dustin et al. Automated Software Testing: Introduction, Management, and Performance: Introduction, Management, and Performance 8. Тамре Л. Введение в тестирование программного обеспечения. М.:Вильямс, // Louse Tamres. Introducing Software Testing 9. Котляров В.П., Коликова Т.В. Основы тестирования программного обеспечения. М.:ИНТУИТ, Синицын С.В., Налютин Н.Ю. Верификация программного обеспечения. М.:ИНТУИТ, Синицын С.В., Налютин Н.Ю. Верификация программного обеспечения. 10

11 12. ГОСТ Р ИСО Системы менеджмента качества. Требования М.: ИПК Изд-во стандартов, // ISO 9000 Quality Systems Handbook 13. RTCA/DO-178B. Software considerations in airborne system and equipment certification RTCA Inc, IEEE IEEE Standard for Software Verification and Validation Institute of Electrical and Electronics Engineers. 01-May IEEE IEEE Standard for Software Test Documentation Institute of Electrical and Electronics Engineers. 01-May IEEE IEEE Standard for Software Unit Testing. Institute of Electrical and Electronics Engineers. 01-May ГОСТ Надежность в технике. Основные понятия. Термины и определения. М.: Издательство стандартов, ГОСТ Надежность в технике. Термины и определения. М.: Изд-во стандартов, IEEE IEEE Guide to Software Configuration Management. Institute of Electrical and Electronics Engineers. 10-Sep Майерс Г. Искусство тестирования программ. М.:Финансы и статистика, 1982, 176 с // Glenford Myers. The Art of Software testing 21. ISO/IEC :2001. Software engineering -- Product quality -- Part 1: Quality model International Organization for Standardization/International Electrotechnical Commission. 01-Jun Microsoft Solutions Framework. Методология создания программных решений Special Equipment Practical studies are conducted in a computerized laboratory equipped with an overhead projector and audio portables. Every student has a PC with high speed internet connection and access to the HSE electronic library. References 1. Master curriculum of MSc educational programme Software Engineering, specialization Software development management. Moscow, HSE, Curriculum of 1 st year MSc educational programme Software Engineering, specialization Software development management. - Moscow, HSE, Educational standard of the Federal State Autonomous Educational Institution for higher professional education "National Research University "The Higher School of Economics" (HSE) / educational course area code "Software Engineering", MS degree level. The author of the program: Nalyutin N. 11