1 Topic relevant selected content from the highest rated entries, typeset, printed and shipped. Combine the advantages of up-to-date and in-depth knowledge with the convenience of printed books. A portion of the proceeds of each book will be donated to the Wikimedia Foundation to support their mission: to empower and engage people around the world to collect and develop educational content under a free license or in the public domain, and to disseminate it effectively and globally. The content within this book was generated collaboratively by volunteers. Please be advised that nothing found here has necessarily been reviewed by people with the expertise required to provide you with complete, accurate or reliable information. Some information in this book maybe misleading or simply wrong. The publisher does not guarantee the validity of the information found here. If you need specific advice (for example, medical, legal, financial, or risk management) please seek a professional who is licensed or knowledgeable in that area. Sources, licenses and contributors of the articles and images are listed in the section entitled References. Parts of the books may be licensed under the GNU Free Documentation License. A copy of this license is included in the section entitled GNU Free Documentation License All used third-party trademarks belong to their respective owners.
2 Contents Articles Testing as a service 1 Application software 2 Mob4Hire 7 Web testing 10 Software testing 12 Portal:Software Testing 26 Acceptance testing 28 Ad hoc testing 32 Agile testing 33 All-pairs testing 34 American Software Testing Qualifications Board 35 API Sanity Autotest 36 Association for Software Testing 37 Attack patterns 38 Augmented Reality-based testing 42 Australian and New Zealand Testing Board 44 Automated Testing Framework 45 Avalanche (dynamic analysis tool) 46 Bebugging 47 Behavior Driven Development 47 Black-box testing 52 Block design 53 Boundary case 56 Boundary testing 56 Boundary-value analysis 56 Browser speed test 57 BS BS Bug bash 60 Build verification test 60 CA/EZTEST 61 Cause-effect graph 62 Centercode 63 Characterization test 65
3 Cloud testing 66 Code coverage 67 Code integrity 72 Codenomicon 73 Compatibility testing 76 Component-Based Usability Testing 77 Conference Room Pilot 79 Conformance testing 80 Core Security 82 Corner case 84 Daikon (system) 85 Data-driven testing 85 Decision table 87 Decision-to-decision path 90 Design predicates 90 Development, testing, acceptance and production 92 DeviceAnywhere 93 Dry run (testing) 94 Dynamic program analysis 95 Dynamic testing 96 Edge case 97 Endeavour Software Project Management 98 Equivalence partitioning 100 Error guessing 101 Exploratory testing 102 Fagan inspection 104 Fault injection 107 Financial tester 112 Framework for Integrated Test 112 Functional testing 113 Functionality assurance 114 Fuzz testing 115 Game testing 118 Google Guice 124 Graphical user interface testing 125 Hybrid testing 128 IBM Product Test 129 IBM Rational Quality Manager 131
4 IEEE Independent software verification and validation 136 Installation testing 137 Integration testing 138 Integration Tree 139 International Software Testing Qualifications Board 140 International Software Testing Qualifications Board Certified Tester 141 JSystem 143 Keyword-driven testing 148 Learnability 150 Lightweight software test automation 151 Load testing 152 Localization testing 155 Manual testing 156 Matrix of Pain 158 Mauve (test suite) 159 Metasploit Project 160 Microsoft Reaction Card Method (Desirability Testing) 163 Mobile Device Testing 166 Mockito 167 Model-based testing 168 Modified Condition/Decision Coverage 172 Modularity-driven testing 173 Monkey test 174 Month of bugs 175 Mutation testing 176 National Software Testing Laboratories 179 NMock 180 Non-functional testing 180 Non-Regression testing 181 Operational Acceptance Testing 184 Oracle (software testing) 185 Original Software 186 Oulu University Secure Programming Group 187 Pair Testing 188 Parameter validation 189 Partial concurrent thinking aloud 190 Penetration test 192
6 STAR (Conference) 261 Stream X-Machine 262 Stress testing 264 Stress testing (software) 266 System integration testing 268 System testing 270 Tessy (Software) 271 Test Anything Protocol 272 Test automation 274 Test automation framework 278 Test automation management tools 279 Test bench 280 Test case 281 Test data 283 Test design 284 Test Double 285 Test effort 286 Test execution engine 287 Test harness 289 Test management 290 Test Management Approach 291 Test plan 295 Test script 297 Test strategy 298 Test stubs 301 Test suite 302 Test Template Framework 303 Test Vector Generator 307 Test-driven development 307 Test-Driven Development by Example 314 Testbed 314 Tester driven development 315 Tester forum 316 Testing Maturity Model 317 Testware 318 Think aloud protocol 319 Tiger team 320 Tosca (Software) 321
7 TPS report 324 TPT (Software) 326 Traceability matrix 329 Tree testing 331 TTCN Twist (software) 333 Unit testing 334 Unusual software bug 339 Usability testing 343 Utest 348 Verification and Validation (software) 351 Volume testing 353 Vulnerability (computing) 353 White-box testing 361 Windmill (testing framework) 362 X-Machine Testing 362 References Article Sources and Contributors 368 Image Sources, Licenses and Contributors 377 Article Licenses License 378
8 Testing as a service 1 Testing as a service Testing as a Service (TaaS, typically pronounced 'tass') is a model of software testing whereby a provider undertakes the activity of software testing applications/solutions for customers as a service on demand. Testing as a Service involves the on-demand test execution of well-defined suites of test material, generally on an outsourced basis. The execution can be performed either on client site or remotely from the outsourced providers test lab/facilities. Aims and objectives One of the main objectives of TaaS is to allow the organisation to focus on core business activities while keeping costs down, thus allowing them to address the current (2008/9) trend of reduced IT budgets while not getting distracted on non-core competencies.  Key characteristics and delivery In order for TaaS to work effectively it should be delivered on either a fixed price or known cost basis with clearly defined schedules of work, this will allow organisations to deal with the high levels of change that might be inherent in their solutions/organisations while allowing them to smooth any resource peaks and troughs.  There is no clear evidence to support which is the most effective delivery model for TaaS, whether that be onshore, near shore or indeed offshore, it is however clear that for most organisations speed of testing is important and so the use of people versus technology must be considered very carefully. Implementation While it is not true for every system under test (SUT) or application under test (AUT), it is true that in certain circumstances the use of test automation can massively increase the speed and efficiency of test execution.  So any TaaS offering, if delivered using test automation will allow organisations to benefit from reduced time to market without suffering the learning curve and headaches involved in undertaking test automation.  References  Value versus cost: governing IT on a reduced budget, Retrieved, 2010/01/04 / www. computerweekly. com/ Articles/ 2002/ 02/ 08/ / value-versus-cost-governing-it-on-a-reduced-budget. htm  Agile Requirements Change Management, Retrieved 2010/01/04, / www. computer. org/ portal/ web/ csdl/ doi/ / EURMIC  Importance of Software Test Automation using tools such as QTP, Retrieved , / qualitypoint. blogspot. com/ 2009/ 04/ importance-of-software-test-automation. html  Cost Benefits Analysis of Test Automation, Retrieved 2010/01/04, / www. softwarequalitymethods. com/ Papers/ Star99%20model%20Paper. pdf
9 Testing as a service 2 External links nfocus Testing as a Service (http:// www. nfocus. co. uk/ Services/ ManagedServices/ TestingasaService. aspx) Initto Testing as a Service (http:// www. initto. com/ test-service. html) Qutesys Testing as a Service (http:// www. qutesys. com/ 2011/ 01/ testing-as-service-taas. html) Manual Intervention in an Automated Test (http:// blog. nfocus. co. uk/ 2009/ 11/ manual-intervention-in-automated-test. html) Using Non-Functional Tests Tools with Axe and WatiN (http:// blog. nfocus. co. uk/ 2009/ 10/ using-non-functional-tests-tools-with. html) Practical Experience in Automated Testing (http:// www. methodsandtools. com/ archive/ archive. php?id=33) Test Automation: Delivering Business Value (http:// www. applabs. com/ internal/ app_whitepaper_test_automation_delivering_business_value_1v00. pdf) Guidelines for Test Automation framework (http:// info. allianceglobalservices. com/ Portals/ 30827/ docs/ test automation framework and guidelines. pdf) Tieto Testing as a Service (http:// www. tieto. com/ default. asp?path=1,127,41042) TestLab² Testing as a Service (http:// www. testlab2. com/ ) Using Cloud Computing to Automate Full-Scale System Tests (http:// www. youtube. com/ watch?v=atyq-41gnjc) The Cloud's Next Big Thing: Software Testing (http:// www. informationweek. com/ cloud-computing/ blog/ archives/ 2009/ 06/ the_clouds_next. html) Application software Application software, also known as an application or an "app", is computer software designed to help the user to perform singular or multiple related specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with documents. Application software is contrasted with system software and middleware, which manage and integrate a computer's capabilities, but typically OpenOffice.org Writer word processor. OpenOffice.org is a popular do not directly apply them in the performance of example of open source application software tasks that benefit the user. A simple, if imperfect, analogy in the world of hardware would be the relationship of an electric light bulb (an application) to an electric power generation plant (a system). The power station merely generates electricity, not itself of any real use until harnessed to an application like the electric light that performs a service that benefits the user. Application software applies the power of a particular computing platform or system software to a particular purpose. Some apps such as Microsoft Office are available in versions for several different platforms; others have narrower requirements.
10 Application software 3 Terminology In information technology, an application is a computer program designed to help people perform an activity. An application thus differs from an operating system (which runs a computer), a utility (which performs maintenance or general-purpose chores), and a programming language (with which computer programs are created). Depending on the activity for which it was designed, an application can manipulate text, numbers, graphics, or a combination of these elements. Some application packages offer considerable computing power by focusing on a single task, such as word processing; others, called integrated software, offer somewhat less power but include several applications.  User-written software tailors systems to meet the user's specific needs. User-written software include spreadsheet templates, word processor macros, scientific simulations, graphics and animation scripts. Even filters are a kind of user software. Users create this software themselves and often overlook how important it is. The delineation between system software such as operating systems and application software is not exact, however, and is occasionally the object of controversy. For example, one of the key questions in the United States v. Microsoft antitrust trial was whether Microsoft's Internet Explorer web browser was part of its Windows operating system or a separable piece of application software. As another example, the GNU/Linux naming controversy is, in part, due to disagreement about the relationship between the Linux kernel and the operating systems built over this kernel. In some types of embedded systems, the application software and the operating system software may be indistinguishable to the user, as in the case of software used to control a VCR, DVD player or microwave oven. The above definitions may exclude some applications that may exist on some computers in large organizations. For an alternative definition of an app: see Application Portfolio Management. Application software classification Application software falls into two general categories; horizontal applications and vertical applications. Horizontal Application are the most popular and its widely spread in departments or companies. Vertical Applications are designed for a particular type of business or for specific division in a company. There are many types of application software: An application suite consists of multiple applications bundled together. They usually have related functions, features and user interfaces, and may be able to interact with each other, e.g. open each other's files. Business applications often come in suites, e.g. Microsoft Office, OpenOffice.org and iwork, which bundle together a word processor, a spreadsheet, etc.; but suites exist for other purposes, e.g. graphics or music. Enterprise software addresses the needs of organization processes and data flow, often in a large distributed environment. (Examples include financial systems, customer relationship management (CRM) systems and supply-chain management software). Note that Departmental Software is a sub-type of Enterprise Software with a focus on smaller organizations or groups within a large organization. (Examples include Travel Expense Management and IT Helpdesk) Enterprise infrastructure software provides common capabilities needed to support enterprise software systems. (Examples include databases, servers, and systems for managing networks and security.) Information worker software addresses the needs of individuals to create and manage information, often for individual projects within a department, in contrast to enterprise management. Examples include time management, resource management, documentation tools, analytical, and collaborative. Word processors, spreadsheets, and blog clients, personal information system, and individual media editors may aid in multiple information worker tasks. Content access software is software used primarily to access content without editing, but may include software that allows for content editing. Such software addresses the needs of individuals and groups to consume digital entertainment and published digital content. (Examples include Media Players, Web Browsers, Help browsers and Games)
11 Application software 4 Educational software is related to content access software, but has the content and/or features adapted for use in by educators or students. For example, it may deliver evaluations (tests), track progress through material, or include collaborative capabilities. Simulation software are computer software for simulation of physical or abstract systems for either research, training or entertainment purposes. Media development software addresses the needs of individuals who generate print and electronic media for others to consume, most often in a commercial or educational setting. This includes Graphic Art software, Desktop Publishing software, Multimedia Development software, HTML editors, Digital Animation editors, Digital Audio and Video composition, and many others.  Mobile applications ("Mobile apps") run on hand-held devices such as smart phones, tablet computers, portable media players, personal digital assistants and enterprise digital assistants : see mobile application development. Product engineering software is used in developing hardware and software products. This includes computer aided design (CAD), computer aided engineering (CAE), computer language editing and compiling tools, Integrated Development Environments, and Application Programmer Interfaces. A command-line interface is one in which you type in commands to make the computer do something. You have to know the commands and what they do, and type them correctly. DOS and Unix are examples of command-driven interfaces. A graphical user interface (GUI) is one in which you select command choices from various menus, buttons and icons using a mouse. It is a user-friendly interface. Microsoft Windows and Mac OS are both graphical user interfaces. A third party server side application that the user may choose to install in his or her account on a social media site or other Web 2.0 web site, for example a facebook app. Applications can also be classified by computing platform. Information worker software Enterprise Resource Planning Accounting software Task and Scheduling Field service management Data Management Contact Management Spreadsheet Personal Database Documentation Document Automation/Assembly Word Processing Desktop publishing software Diagramming Software Presentation software Blog Reservation systems Financial Software Day trading software Banking systems Clearing systems
12 Application software 5 arithmetic software Content access software Electronic media software Web browser Media Players Hybrid editor players Entertainment software Digital pets Screen savers Video Games Arcade games Emulators for console games Personal computer games Console games Mobile games Educational software Classroom Management Learning/Training Management Software Reference software Sales Readiness Software Survey Management Enterprise infrastructure software Business workflow software Database management system (DBMS) software Digital asset management (DAM) software Document Management software Geographic Information System (GIS) software Simulation software Computer simulators Scientific simulators Social simulators Battlefield simulators Emergency simulators Vehicle simulators Flight simulators Driving simulators Simulation games Vehicle simulation games
13 Application software 6 Media development software Image organizer Media content creating/editing 3D computer graphics software Animation software Graphic art software Image editing software Raster graphics editor Vector graphics editor Video editing software Sound editing software Digital audio editor Music sequencer Scorewriter Hypermedia editing software Web Development Software Game development tool Product engineering software Hardware Engineering Computer-aided engineering Computer-aided design (CAD) Finite Element Analysis Software Engineering Computer Language Editor Compiler Software Integrated Development Environments Game creation software Debuggers Program testing tools License manager References  Ceruzzi, Paul E. (2000). A History of Modern Computing. Cambridge, Mass.: MIT Press. ISBN  Campbell-Kelly, Martin; Aspray, William (1996). Computer: A History of the Information Machine. New York: Basic Books. ISBN
14 Mob4Hire 7 Mob4Hire Mob4Hire is a privately held Canadian Web 2.0 company that focuses on crowd sourced mobile application testing services     and market research in the global wireless telephony space. The company facilitates the relationship between mobile application developers and professional users who test the functionality and assess the usability of the application(s) under consideration for a negotiated fee.  The services are based on a bidding model with a fixed-fee structure. As the broker for these transactions, Mob4Hire derives revenue as a flat percentage of each transaction agreed to by the developer and tester. The company also accrues income from up-front licensing fees, ongoing subscriptions, managed accounts and other services. The business model also incorporates enterprise customers. Mob4Hire As A Platform [MAAP] effectively re-brands the Mob4Hire model. O2 Litmus, a British division of the global carrier Telefonica has implemented Mob4Hire technology and launched their own community of developers and testers.  General overview Mob4Hire brokers relationships between mobile application developers and a global testing community. Developers post pre-release applications and associated test plans to the community, and specify which handsets and networks are under consideration. Individuals (or groups) registered in the testing community (Mob) that are able to fulfill the functional requirements of the test bid for the contract, and developers select testers based on such criteria as the community ranking of the tester, turn-around time for delivered test reports, the tester s desired pay-scale and other metrics.  Mob4Hire is responsible for holding payment in escrow until the developer is satisfied. Once the developer has approved the test report, monies are then released to the tester via PayPal.  The developer is expected to grade the quality of each tester. This feedback establishes a rating hierarchy among the testing community; the intent is to continually improve the overall quality the application test reports and grade the relative skill and efficacy of the individual testers.  Mob4Hire is also concerned with the user experience of mobile applications. The process involved is similar to the functionality testing except that the testers are focused on the usability of the application being considered and the overall user experience rather than a purely functional assessment.  A paradigm of software design is that developers are the least desirable group to test their own releases and, in general, developer bias can be mitigated by employing a testing group completely independent of the design process. The testers are locally based on the actual networks on which the applications will be launched and as such, their assessment more closely emulates genuine market feedback than in-house testing conducted remotely from the network.  Developers conduct mobile surveys in the global space by paying testers to download applications and complete a survey with demographic information relevant to the developer. The intent is to collect key market research from a community of unbiased early adopters with minimized costs to the developer.  Examples of typical survey questions may include: Was the application intuitive? How would you describe the user experience? Would you pay for the application and if so, how much? Were there features that should be included or enhanced? How did the interface appear on the handset? Would you recommend the application? Mob4Hire builds and maintains a global community of application developers and testers. There are currently almost 1,000 registered developers and market research firms in 111 countries and 42,000 registered testers in 142 countries. The testers represent 364 carrier networks. Mob4Hire is also associated with 20 professional entities that develop, test and port mobile applications. 
15 Mob4Hire 8 History Paul Poutanen, the founder and current president of Mob4Hire, credits the inspiration of the crowdsourced mobile application testing paradigm to an insight he had in the San Francisco airport. Poutanen and several colleagues had spent a week testing a mobile phone based game on a carrier in the Bay Area. They d purchased ten handsets with subscriptions for a demonstration to a local client, and Poutanen was considering the overall expenses of the network tests while awaiting a return flight. He saw a fellow traveler speaking on the identical handset and considered how much time and money he could have saved if that user had done his testing for him.  Poutanen wrote a business idea and presented it to the Cambrian House  community, a Calgary-based crowdsourcing community for software and business applications. Members from the community then helped build the original Mob4Hire platform. Mob4Hire was incorporated in September, 2007  and the beta version of the Mob4Hire platform was launched in March,  November, 2008: O2 Litmus, a U.K. subsidiary of Telefonica, became the first carrier to adopt and implement Mob4Hire technology [MAAP] as a white-label testing community.  November, 2008: Stephen King appointed CEO of Mob4Hire.  January, 2009: At the GSMA 2009 Mobile Innovation Global Competition in Barcelona, Spain, Mob4Hire was named Top 15 Innovator Chosen for Most Innovative True Mobile Startup Category in Global Tournament.    March, 2009: Mob4Hire partners with Wavefront. March, 2009: Mob4Hire enters marketing deal with Gamelion.  May, 2009: Mob4Hire awarded Red Herring 100 North America.  July, 2009: Mob4Hire collaborates with Navteq for LBS application testing.  November, 2009: Mob4Hire wins Investor Choice awards at Vancouver Angel Forum.    November, 2009: Mob4Hire and Flurry Analytics announce a joint collaboration.   December, 2009: Infostretch and Mob4Hire form partnership. February, 2010: Mob4Hire joins the BlackBerry ISV Alliance Program.  March, 2010: Mob4Hire releases preliminary Global Wireless Survey.  March, 2010: Perfecto Mobile and Mob4Hire form partnership.  April, 2010: Mob4Hire launches V4.0 platform.  April, 2010: Mob4Hire included in the 'Top 25 Canadian ICT Up and Comers' by the Branham Group  June, 2010: Mob4Hire awarded a 'Meffy' by the Mobile Entertainment Forum for 'Most Innovative Business Model'  Corporate affairs Executives Stephen King, CEO Paul Poutanen, President and Founder John Carpenter, CTO Allen Poutanen, VP Business Development Advisory board Randy Thompson, Venture Alberta Keith Cook, Device Anywhere Bob Hayes, Ph.D., Advocacy Expert
16 Mob4Hire 9 Tony Fish, AMF Ventures Lisa Whelan, socializemobilize.com David Craddock, MacLeod Dixon LLP Jonathan Kohl, Testing Evangelist Marc Kornak, Advisory Board Member, Genesis Technical Solutions Cameron Peters, Innovation Architect Stephen Nykolyn, Mobile Business Dev Marc Wachmann, growwireless.com References  / en. wikipedia. org/ wiki/ List_of_crowdsourcing_projects  / www. itworldcanada. com/ news/ mob4hire-taps-into-the-crowd-for-mobile-app-tests/  / www. mob4hire. com/ about. php  / developer. motorola. com/ fasttrack/ mob4hire/  / wirelessweek. com/ Articles/ 2009/ 10/ Putting-Apps-to-Test/  / www. o2litmus. co. uk/ ipb/ index. php?showtopic=612  / www. slideshare. net/ stebankag/ wi-tec-stephen-king-commercialization-challenges-of-mobile-software-development-in-a-fragmented-mobile-ecosystem-mob4hire?type=presentation/  / www. mob4hire. com/ developerhowitworks. php  / www. scribd. com/ doc/ / Application-Development-John-Carpenter-Mob4Hire  / www. littlespringsdesign. com/ blog/ blog/ 2008/ 07/ 21/ usability-testing-of-multiple-device-software-and-sites  / news. cnet. com/ _ html  / www. calgaryherald. com/ technology/ Calgary+ developer+ taps+ power+ crowd/ / story. html  / www. mob4hire. com/ about. php  / developer. sonyericsson. com/ site/ global/ newsandevents/ latestnews/ newsaug08/ p_crowdsourced_testing_mob4hire. jsp  / www. backbonemag. com/ Web2/ Cambrian_House_2008. asp  / www. crowdsourcingdirectory. com/?p=108  / finance. denverpost. com/ mng-denver?account=denverpost& GUID= & Page=MediaViewer& ChannelID=3191  / www. pcadvisor. co. uk/ news/ index. cfm?newsid=  / www. capterra. com/ ecommerce-software/ software_news/ prweb  / www. mob4hire. com/ mobile_innovation_global_award. pdf  / www. marketwire. com/ press-release/ Wavefront html  / www. bctechnology. com/ scripts/ display_news. cfm?id=27611  / www. annualaudiograms. com/ a mob4hire-signs-mobile-testing-marketing-deal. cfm  / www. techvibes. com/ blog/ mob4hire-wins-red-herring-top-100-award  / www. prweb. com/ releases/ 2009/ 07/ prweb htm  / mob4hire. blogspot. com/  / www. research-live. com/ news/ analytics/ mob4hire-and-flurry-team-up-for-mobile-app-tracking/ article  / www. earthtimes. org/ articles/ show/ mob4hire-and-flurry-team-to, shtml  / www. 1888pressrelease. com/ mobile-app-testing/ mobile-crowed-testing/ infostretch-and-mob4hire-form-partnership-to-offer-seamless-pr html  / www. research-live. com/ news/ new-business/ mob4hire-and-infostretch-team-up-to-test-mobile-apps/ article  / www. prweb. com/ releases/ 2010/ 02/ prweb htm  / www. techvibes. com/ blog/ mob4hire-releases-first-look-at-their-global-wireless-survey  / www. perfectomobile. com/ portal/ cms/ mob4hire_perfecto_mobile_solve_problem_fragmentation. html  / www. mrweb. com/ drno/ news htm  / www. branham300. com/ index. php?year=2010& listing=3  / www. meffys. com/ attend/ meffys-winners-showcase-dynamic-mobile-media-industry