TYPES OF TESTING CONTENTS. Software Testing

Transcription

1 TYPES OF TESTING CONTENTS I. White Box Testing: Classification of White Box. i. Static Testing- Inspections, Structured Walkthroughs, Technical Review. ii. Structural Testing -Code Functional Testing, Code Coverage Testing, Code Complexity Testing. II. Black Box Testing: Techniques for Black Box Testing i. Requirement Based Testing. ii. Positive and Negative Testing. iii. Boundary Value Analysis. iv. Decision Tables. v. Equivalence Partitioning. vi. User Documentation Testing. vii. Graph Based Testing. viii. Sample Examples on White and Black Box Testing.

2 I. White Box Testing: Classification of White Box. (Question: Explain white box testing. 4 marks) 1. This is also known as glass box, clear box, and open box testing. 2. In white box testing, test cases are created by looking at the code to detect any potential failure scenarios. 3. The suitable input data for testing various APIs and the special code paths that need to be tested by analysing the source code for the application block. 4. Therefore, the test plans need to be updated before starting white box testing and only after a stable build of the code is available. 5. White box testing assumes that the tester can take a look at the code for the application block and create test cases that look for any potential failure scenarios. 6. During white box testing, analyze the code of the application block and prepare test cases for testing the functionality to ensure that the class is behaving in accordance with the specifications and testing for robustness. 7. A failure of a white box test may result in a change that requires all black box testing to be repeated and white box testing paths to be reviewed and possibly changed. Figure 1: White Box T esting i. Static Testing- Inspections, Structured Walkthroughs, Technical Review. 1. Inspection (Question: Explain the concept of inspection = 4 marks)

3 i. Inspections are the most formal type of reviews. ii. They are highly structured and require training for each participant. iii. Inspections are different from peer reviews and walkthroughs in that the person who presents the code, the presenter or reader, isn t the original programmer. iv. These forces someone else to learn and understand the material being presented, potentially giving a different slant and interpretation at the inspection meeting. v. The other participants are called inspectors. vi. Each is tasked with reviewing the code from a different perspective, such as a user, a tester, or a product support person. vii. This helps bring different views of the product under review and very often identifies different bugs. viii. One inspector is even tasked with reviewing the code backward that is, from the end to the beginning to make sure that the material is covered evenly and completely. 2. Walkthrough (Question: Explain the concept of Walkthrough = 4 marks) i. Walkthroughs are the next step up in formality from peer reviews. ii. In a walkthrough, the programmer who wrote the code formally presents (walks through) it to a small group of five or so other programmers and testers. iii. The reviewers should receive copies of the software in advance of the review so they can examine it and write comments and questions that they want to ask at the review. iv. Having at least one senior programmer as a reviewer is very important. 3. Technical Review (Question: Explain Technical review- 6 Marks) i. Formal Review A formal review is the process under which static white-box testing is performed. A formal review can range from a simple meeting between two programmers to a detailed, rigorous inspection of the code. There are four essential elements to a formal review

4 1. Identify Problems: -.The goal of the review is to find problems with the software not just items that are wrong, but missing items as well. All criticism should be directed at the code, not the person who created it. Participants shouldn t take any criticism personally. Leave your egos, emotions, and sensitive feelings at the door. 2. Follow Rules: - A fixed set of rules should be followed. They may set the amount of code to be reviewed (usually a couple hundred lines), how much time will be spent (a couple hours), what can be commented on, and so on. This is important so that the participants know what their roles are and what they should expect. It helps the review run more smoothly. 3. Prepare: - Each participant is expected to prepare for and contribute to the review. Depending on the type of review, participants may have different roles. They need to know what their duties and responsibilities are and be ready to actively fulfill them at the review. Most of the problems found through the review process are found during preparation, not at the actual review. 4. Write a Report: - The review group must produce a written report summarizing the results of the review and make that report available to the rest of the product development team. It s imperative that others are told the results of the meeting how many problems were found, where they were found, and so on. ii. Peer Reviews (Question: Explain the concept of peer review = 4 marks) The easiest way to get team members together and doing their first formal reviews of the software is through peer reviews, the least formal method. Sometimes called buddy reviews, this method is really more of a discussion. Peer reviews are often held with just the programmer who wrote the code and one or two other programmers or testers acting as reviewers. Small group simply reviews the code together and looks for problems and oversights. To assure that the review is highly effective all the participants need to make sure that the four key elements of a formal review are in place: Look for problems, follow rules, prepare for the review, and write a report. As peer reviews are informal, these elements are often scaled back. Still, just getting together to discuss the code can find bugs.

5 ii. Structural Testing-Code Functional Testing, Code Coverage Testing, Code Complexity Testing. 1. Data Flow (Code Functional Testing) i. Data flow coverage involves tracking a piece of data completely through the software. ii. At the unit test level this would just be through an individual module or function. iii. The same tracking could be done through several integrated modules or even through the entire software product although it would be more timeconsuming to do so. iv. During data flow, the check is made for the proper declaration of variables declared and the loops used are declared and used properly. For example 1. #include<stdio.h> 2. void main() 3. { 4. int i, fact= 1, n; 5. printf( enter the number ); 6. scanf( %d,&n); 7. for(i =1 ;i <=n;i++) 8. fact = fact * i; 9. printf ( the factorial of a number is %d, fact); 10. } 2. Data Coverage (Code Coverage Testing) (Question: Explain the concept of data coverage in software testing- 4 Marks) i. The logical approach is to divide the code just as you did in black-box testing into its data and its states (or program flow). ii. By looking at the software from the same perspective, you can more easily map the white-box information you gain to the black-box cases you ve already written. iii. Consider the data first. Data includes all the variables, constants, arrays, data structures, keyboard and mouse input, files and screen input and output, and I/O to other devices such as modems, networks, and so on. For example 1. #include<stdio.h> 2. void main()

6 3. { 4. int i, fact= 1, n; 5. printf( enter the number ); 6. scanf( %d,&n); 7. for(i =1 ;i <=n;i++) 8. fact = fact * i; 9. printf ( the factorial of a number is %d, fact); 10. } The declaration of data is complete with the assignment statement and the variable declaration statements. All the variable declared are properly utilized. 3. Program Statements and Line Coverage (Code Complexity Testing) (Question: Explain the concept of code complexity testing with example- 6 Marks) i. The most straightforward form of code coverage is called statement coverage or line coverage. ii. If you re monitoring statement coverage while you test your software, your goal is to make sure that you execute every statement in the program at least once. iii. With line coverage the tester tests the code line by line giving the relevant output. For example 1. #include<stdio.h> 2. void main() 3. { 4. int i, fact= 1, n; 5. printf( enter the number ); 6. scanf( %d, &n); 7. for(i =1 ;i <=n; i++) 8. fact = fact * i; 9. printf ( the factorial of a number is %d, fact); 10. } 4. Branch Coverage (Code Complexity Testing) i. Attempting to cover all the paths in the software is called path testing. ii. The simplest form of path testing is called branch coverage testing. iii. To check all the possibilities of the boundary and the sub boundary conditions and it s branching on those values.

7 iv. Test coverage criteria requires enough test cases such that each condition in a decision takes on all possible outcomes at least once, and each point of entry to a program or subroutine is invoked at least once. v. Every branch (decision) taken each way, true and false. vi. It helps in validating all the branches in the code making sure that no branch leads to abnormal behavior of the application. 5. Condition Coverage (Code Complexity Testing) i. Just when you thought you had it all figured out, there s yet another complication to path testing. ii. Condition coverage testing takes the extra conditions on the branch statements into account.

8 II. Black Box Testing: Techniques for Black Box Testing (Question: Explain the black box testing technique 4 marks) 1. This approach tests all possible combinations of end -user actions. 2. Black box testing assumes no knowledge of code and is intended to simulate the end-user experience. 3. The tester can use sample applications to integrate and test the application block for black box testing. 4. Planning for black box testing immediately after the requirements and the functional specifications are available. 5. Black box testing should be conducted in a test environment close to the target environment. Figure 2. Black Box Testi ng 1. Requirement Based Testing. (Question: Explain requirement based testing stages - 4 marks) Requirements-based testing is a testing approach in which test cases, conditions and data are derived from requirements. It includes functional tests and also nonfunctional attributes such as performance, reliability or usability. i. Stages in Requirements based Testing: Defining Test Completion Criteria - Testing is completed only when all the functional and non-functional testing is complete.

9 Design Test Cases - A Test case has five parameters namely the initial state or precondition, data setup, the inputs, expected outcomes and actual outcomes. Execute Tests - Execute the test cases against the system under test and document the results. Verify Test Results - Verify if the expected and actual results match each other. Verify Test Coverage - Verify if the tests cover both functional and nonfunctional aspects of the requirement. Track and Manage Defects - Any defects detected during the testing process goes through the defect life cycle and are tracked to resolution. Defect Statistics are maintained which will give us the overall status of the project. ii. Requirements testing process: Testing must be carried out in a timely manner. Testing process should add value to the software life cycle, hence it needs to be effective. Testing the system exhaustively is impossible hence the testing process needs to be efficient as well. Testing must provide the overall status of the project, hence it should be manageable. 2. Positive and Negative Testing. i. Positive Testing: Positive Testing is testing process where the system validated against the valid input data. In this testing tester always check for only valid set of values and check if an application behaves as expected with its expected inputs. The main intention of this testing is to check whether software application not showing error when not supposed to & showing error when supposed to. Such testing is to be carried out keeping positive point of view & only execute the positive scenario. Positive Testing always tries to prove that a given product and project always meets the requirements and specifications. Under Positive testing is test the normal day to day life scenarios and check the expected behaviour of application. Example of Positive testing: To test the text box for integer numbers only. As we are testing for positive testing the input range is valid numbers only.

10 Figure 3: Positive Testing ii. Negative Testing: Negative Testing is testing process where the system validated against the invalid input data. A negative test checks if an application behaves as expected with its negative inputs. The main intention of this testing is to check whether software application not showing error when supposed to & showing error when not supposed to. Such testing is to be carried out keeping negative point of view & only execute the test cases for only invalid set of input data. Negative testing is a testing process to identify the inputs where system is not designed or un-handled inputs by providing different invalid. The main reason behind Negative testing is to check the stability of the software application against the influences of different variety of incorrect validation data set. 3. Boundary Value Analysis. (Question: Explain the concept of boundary conditions and sub boundary condition s 2 marks each - 4 marks) i. Boundary conditions are special because programming, by its nature, is susceptible to problems at its edges. ii. The boundary conditions are defined as the initial and the final data ranges of the variables declared. iii. If an operation is performed on a range of numbers, odds are the programmer got it right for the vast majority of the numbers in the middle, but maybe made a mistake at the edges. iv. The edges are the minimum and the maximum values for that identifier. 1. #include<stdio.h>

11 2. void main() 3. { 4. int i, fact=1, n; 5. printf( enter the number ); 6. scanf( %d,&n); 7. for(i =1 ;i <=n;i++) 8. fact = fact * i; 9. printf ( the factorial of a number is %d, fact); 10. } The boundary condition in the above example is for the integer variable. Sub-Boundary Conditions i. They re the ones defined in the specification or evident when using the software. ii. Some boundaries, though, that are internal to the software aren t necessarily apparent to an end user but still need to be checked by the software tester. iii. These are known as sub-boundary conditions or internal boundary conditions. iv. In the given example the sub boundary condition is the value of factorial For example 1. #include<stdio.h> 2. void main() 3. { 4. int i, fact=1, n; 5. printf( enter the number ); 6. scanf( %d,&n); 7. for(i =1 ;i <=n;i++) 8. fact = fact * i; 9. printf ( the factorial of a number is %d, fact); 10. } 4. Decision Tables. i. Decision table testing is black box test design technique to determine the test scenarios for complex business logic. ii. iii. Decision tables provide a systematic way of stating complex business rules, which is useful for developers as well as for testers. Decision tables can be used in test design whether or not they are used in specifications, as they help testers explore the effects of combinations of

12 iv. different inputs and other software states that must correctly implement business rules. It helps the developers to do a better job can also lead to better relationships with them. v. Testing combinations can be a challenge, as the number of combinations can vi. vii. viii. often be huge. Testing all combinations may be impractical if not impossible. We have to be satisfied with testing just a small subset of combinations but making the choice of which combinations to test and which to leave out is also important. If you do not have a systematic way of selecting combinations, an arbitrary subset will be used and this may well result in an ineffective test effort. 5. Equivalence Partitioning. i. Type of black box testing that divides that divides the input domain of the program into classes of data from which the test cases can be divided. ii. This method is typically used to reduce the total number of test cases to a finite set of testable test cases, still covering maximum requirements. iii. It is the process of taking all possible test cases and placing them into classes. One test value is picked from each class while testing. For example: If you are testing for an input box accepting numbers from 1 to 1000 then there is no use in writing thousand test cases for all 1000 valid input numbers plus other test cases for invalid data. i. Using equivalence partitioning method above test cases can be divided into three sets of input data called as classes. Each test case is a representative of respective class. ii. For the above example we can divide our test cases into three equivalence classes of some valid and invalid inputs. Test cases for input box accepting numbers between 1 and 1000 using Equivalence Partitioning: 1. One input data class with all valid inputs. Pick a single value from range 1 to 1000 as a valid test case. If you select other values between 1 and 1000 then result is going to be same. So one test case for valid input data should be sufficient. 2. Input data class with all values below lower limit. i.e. any value below 1, as an invalid input data test case.

13 3. Input data with any value greater than 1000 to represent third invalid input class. 4. So using equivalence partitioning all possible test cases is partitioned into three classes. Test cases with other values from any class should give you the same result. 5. We have selected one representative from every input class to design our test cases. Test case values are selected in such a way that largest number of attributes of equivalence class can be exercised. 6. Equivalence partitioning uses fewest test cases to cover maximum requirements. 6. User Documentation Testing. i. Documentation testing is a non-functional type of software testing. ii. iii. iv. It is a type of non-functional testing. Any written or pictorial information describing, defining, specifying, reporting, or certifying activities, requirements, procedures, or results. Documentation is as important to a product s success as the product itself. If the documentation is poor, non-existent, or wrong, it reflects on the quality of the product and the vendor. v. As per the IEEE Documentation describing plans for, or results of, the testing vi. vii. viii. ix. of a system or component, Types include test case specification, test incident report, test log, test plan, test procedure, test report. Hence the testing of all the above mentioned documents is known as documentation testing. This is one of the most cost effective approaches to testing. If the documentation is not right: there will be major and costly problems. The documentation can be tested in a number of different ways to many different degrees of complexity. These range from running the documents through a spelling and grammar checking device, to manually reviewing the documentation to remove any ambiguity or inconsistency. Documentation testing can start at the very beginning of the software process and hence save large amounts of money, since the earlier a defect is found the less it will cost to be fixed.

14 7. Graph Based Testing. i. Each and every application is buildup of some objects. ii. All such objects are identified and graph is prepared. iii. From this object graph each object relationship is identified and test cases written accordingly to discover the errors. 8. State Testing (Question: Explain the concept state testing in dynamic black box testing 4 marks) i. The data gets tested on the numbers, words, inputs, and outputs of the software. ii. The product or the software should be tested for the program s logic flow through its various states. iii. A software state is a condition or mode that the software is currently in. iv. Consider Figures 4, which illustrates the software mode of the paint software in airbrush mode. Figure 4: Paint Mode Testing Software s Logic Flow 1. Testing the software s states and logic flow has the same problems. It s usually possible to visit 2. Check for the all possible states for test to pass condition 3. The complexity of the software, especially due to the richness of today s user interfaces, provides so many choices and options that the number of paths grows exponentially.

15 Creating a State Transition Map (Question: Explain the concept of creating a state transition map. Diagram 1 mark, Explanation 3 marks 4 marks) i. There are various techniques for state transition diagrams. ii. Figure 5 shows two examples. One uses boxes and arrows and the other uses circles (bubbles) and arrows. iii. The sate transition diagram should be easily understandable and should be able to explain clearly about the various stages the software passes through. Figure 5: State Transition Map A state transition map should show the following items: Each unique state that the software can be in. A good rule of thumb is that if the state is unsure whether something is a separate state, it probably is. Always collapse it into another state if you find out later that it isn t. The input or condition that takes it from one state to the next. This might be a key press, a menu selection, a sensor input, a telephone ring, and so on. A state can t be exited without some reason. The specific reason is what you re looking for here. Set conditions and produced output when a state is entered or exited. This would include a menu and buttons being displayed, a flag being set, a printout occurring, a calculation being performed, and so on. It s anything and everything that happens on the transition from one state to the next.

16 9. Sample Examples on White and Black Box Testing. A simple website application as an example to explain the white and the black box testing.. The end user is simply access the website, Login & Logout, this is very simple and day 2 days life example. As end users point of view user able to access the website from GUI but inside there are lots of things going on to check the internal things are going right or not, the white box testing method is used. To explain this we have to divide this part in two steps. This is all is being done when the tester is testing the application using White box testing techniques. Step 1) UNDERSTAND OF THE SOURCE CODE The first & very important steps is to understand the source code of the application is being test. As tester should know about the internal structure of the code which helps to test the application. Better knowledge of Source code will helps to identify & write the important test case which causes the security issues & also helps to cover the 100% test coverage. Before doing this the tester should know the programming language what is being used in the developing the application. As security of application is primary objective of application so tester should aware of the security concerns of the project which help in testing. If the tester is aware of the security issues then he can prevents the security issues like attacks from hackers & users who are trying to inject the malicious things in the application. Step 2) CREATE TEST CASES AND EXECUTE In the second step involves the actual writing of test cases based on Statement/Decision/Condition/Branch coverage & actual execution of test cases to cover the 100% testing coverage of the software application. The Tester will write the test cases by diving the applications by Statement/Decision/Condition/Branch wise. Other than this tester can include the trial and error testing, manual testing and using the Software testing tools as we covered in the previous article.