A Case Study in Test Management
|
|
- Sharon Knight
- 8 years ago
- Views:
Transcription
1 A Case Study in Test Management Tauhida Parveen Scott Tilley George Gonzalez Dept. of Computer Sciences Dept. of Computer Sciences Software Quality Management Florida Institute of Technology Florida Institute of Technology Sabre Holdings Inc. fit.edu fit.edu ABSTRACT Testing is an essential but often under-utilized area of software engineering. A variety of software testing techniques have been developed to effectively identify bugs in source code, yet these techniques are not always fully employed in practice. There are numerous reasons for this, including the difficulty in mastering the complexity of managing all of the test cases for large-scale projects. Test case management involves organizing testing artifacts (e.g., requirements traceability data, test cases, and expected results) in a systematic manner. To be successful, test case management requires a high degree of discipline to accommodate the large volume of artifacts under consideration. This paper presents the results of a case study in centralizing test artifacts in an industrial setting to aid better test management. Several of the challenges in adopting this approach are discussed. In response to these challenges, recommendations on how to better leverage test case management are offered. Keywords Software testing, test management 1. INTRODUCTION Software testing is a process to identify the correctness, completeness, and quality of developed computer software. It is an integral part of software engineering. It encompasses the concepts of demonstrating the validity of software at each stage of the development life cycle and the authenticity of the final system with respect to the customer s requirements [1]. However, in application development projects, testing is often not given enough resources, time, and priority until initial development is completed. With competitive pressure and the increasing cost of downtime, organizations have started to introduce testing at earlier stages of software development while others are striving to find effective testing strategies. Few organizations have established the basis to measure the effectiveness of testing. Without testing standards and a proper test management strategy, the effectiveness of testing can not be measured or improved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ACMSE 2007, March 23-24, 2007, Winston-Salem, North Carolina, USA Copyright 2007 ACM /07/ $5.00 This paper focuses on the test management part of the testing process. Test management is a method of organizing test assets and artifacts such as test requirements, test cases, and test results to enable accessibility and reuse. This paper compares and contrasts the theory of test case management against what commonly happens in industry with respect to manual test cases. It describes a case study of the authors experience in applying academic best practices for software test case management in industry and the outcome of such best practices. The paper also outlines an approach to streamlining test case management and how to enhance productivity through test case management. The paper is organized as follows: Section 2 gives an overview of testing and the common techniques that are followed in the testing process. Section 3 describes a case study of the experiences of the authors in implementing a test management strategy. Section 4 illustrates the impacts of establishing test management, the challenges faced and some recommendations to aid in efficient test management process. Section 5 summarizes the paper with some direction for future research in this area. 2. SOFTWARE TESTING Research shows that software quality consumes more than fifty percent of the cost of the software and this number is even higher for safety critical software [2]. One way to support quality assurance is efficient software testing. Testing refers to the execution of software with an input (known as the test case) and observing the result of the input. The output data helps in understanding the behavior of the system and provides data to ensure the conformance to specification of the software. The goal of testing is to efficiently identify any high impact deviations from expected results, so that these deviations can be corrected or prevented before the software is released. The value of testing also encompasses goals such as the reduction of the number of defects and increase development productivity through continual feedback. 2.1 Testing techniques Software testing is a collection of techniques used in the process of verifying that a piece of software is free from technical errors. Three well-known testing techniques that are commonly used in the industry are black box, white box, and gray box testing [6]. Black box testing treats the system as if it were a black box. This testing technique requires no knowledge of the internal code of the system during the testing process and with little regard in 82
2 the internal logic of the system. Although black box testing is designed to identify errors, the technique is also used to demonstrate that the software meets functional requirements. White box testing of software involves a closer examination of the internal logic of the system. Logical paths through the software are tested by providing test cases that use particular sets of conditions and/or loops. The status of the system can be identified at diverse points to establish if the expected status matches the actual status. Gray box testing refers to a technique that examines and can manipulate the activity of back-end components to view the state of the application during test case execution. An example of gray box testing would be using database queries to seed data and check results. The testing process is not limited to the above-mentioned techniques only. It includes testing the software at different levels such as unit, system, integration, acceptance, regression, load, performance, usability, and reliability testing. It also involves writing well-described test cases, test plans, test results, and bug reports. 2.2 Test automation Testing can be manual or automated. Manual testing is performed by testers who use manual interaction to carry out all the step by step actions on the application and determine whether a particular step was accomplished successfully or not. Automated testing is assisted by tools that allow the software to be tested repeatedly and quickly without the need for the presence of a tester for every input, analysis, or evaluation. It is not always possible to fully automate testing. Therefore, manual testing is often an important part of almost any testing effort. Test automation is also not an easy task and requires a greater upfront investment than manual testing. It is expensive due to the need of its specialized tools to support automation as well as the need for specialized programming skills to develop test suites. 2.3 Testing in industry The general theory of software testing is not directly usable in industry. Commercial software testing is always under time pressure that makes it hard to follow theoretical best practices of software testing. Testing in practice is very dynamic, requires reconfiguration of tests from existing test cases, or the modification of basic assumptions from the existing test cases. In theory, testing should be introduced in every step of the software development cycle but in practice it is often introduced at the end of the software development process. Limited time, budget, and resources are put into the testing phase. There is often only one tester for a whole software development team executing mostly manual test cases and documenting the results. Therefore, a lot of critical bugs that should be caught before deployment of the product are never caught. Since the overall testing process in software development projects takes less priority, a sub-section of the testing process, test management, is often not even considered to be an area that should be paid attention to. Testing software applications requires hundreds to thousands of unique test cases, and the ability to efficiently manage the results that are produced from running these test cases. The test cases are collectively known as the test suite and can be manual or automated. With the test cases and the test results, other artifacts such as test plans, bug reports, and defect reports are also produced during the testing process. Often these artifacts are not maintained due to lack of a proper test management strategy and eventually become outdated and can not be reused. Test management is a method of organizing test assets and artifacts such as test requirements, test plans, test cases, test scripts, and test results to enable easy accessibility and reusability. It is one of the many ways the testing process can be made more efficient. There is a need to introduce test management in the testing process, but often the amount of time and effort that has to be put in implementing such a management strategy may keep one from taking such a step. 3. A CASE STUDY This case study describes the challenges that were faced when introducing test management for the first time in an industrial setting. The case study is based on experiences of the authors in implementing a centralized test management strategy. It emphasizes the need for a test management strategy with the right resources and effective use of centralized control. 3.1 The project The objective of the project was to gather all testing artifacts that were available between different testing teams into one central repository to streamline a more disciplined approach to test management. The company followed an agile software development methodology [12]. Software development at this company encompassed developing new products, upgrading of existing products, and software customization. Development was distributed among several offices around the world. Since development was agile, testers were involved in the project from the beginning and in every stage of the process. Every time changes were made to the product by the developers, the resulting build was sent to the testing team. The testing team executed regression tests to make sure that all the tests cases that the product passed before the changes were made, still worked after the new changes have been added. The regression tests were typically manual test cases in Microsoft Excel files. The Excel files containing the test cases were saved in the tester s workstation and when needed by others, were delivered via . Every time a tester was assigned to a product, the tester would make changes in the Excel files according to what he thought suited him. The progress in the testing process was handled by from the testers stating a percentage count of the test coverage. It was getting very hard for management as well as offshore testers to have access to the test cases. Predictably, a large percentage of test cases were lost over time due to hardware failures, employee turnover, and simple changes to test case formats. Other artifacts such as test plans and automated test scripts were also maintained in this manner. After managing the testing artifacts in the above-mentioned adhoc manner, it was decided that a global test management strategy needed to be in place. Therefore, a project to centralize the testing artifacts started. The project had two major goals: (1) to manage test cases globally so that the testers at various locations (in this case Poland, India and the U.S.) could use the same test cases to test the product; and (2) to minimize the loss of testing artifacts 83
3 that occurs over time due to software upgrades, bug fixes, and constant evolution of the software. 3.2 Test case centralization phases The centralization of testing artifacts involved two phases: the discovery phase, and the migration phase. The discovery phase was a challenging phase that involved finding, collecting, and assessing the quality of existing test cases. Since the test management strategy of the testing teams was generally ad-hoc, test cases were written but were not used frequently. This was mostly due to the test cases not being accessible, which made them difficult to find for purposes of execution and modification. The fact that they were difficult to modify generally outdated them over time even though they often included valuable business logic that was needed to evaluate the functionality of the software. Since most of the test cases resided on the local machine of the testers assigned to each product group, predictably, some test cases were lost and needed to be recreated. For the migration process, massive amounts of test cases that were stored in Excel and Word files were ported to a centralized test management tool: TestDirector. Mercury TestDirector (now a part of Mercury Quality Center) [4] was chosen to be the test management tool to centralize the test artifacts and mange them globally. The migration process required changing the Excel files into a format accepted by TestDirector. A TestDirector add-in was used to import the Excel files to Testdirector. If needed, the test cases were then modified to fit the feature that the test case was written for. TestDirector supports more than just test case management, but the company focused on the tool for test case management alone because of existing significant investments in other tools for specification, release management, and error reporting. 3.3 The TestDirector tool TestDirector (TD) is a web-based application published by Mercury Interactive that can manage the essential aspects of test management, requirements management, and defect management. It is a global test management tool. Since it is Web- enabled, it supports collaboration among distributed teams whether they are in different parts of the world or at different places within the same organization. With TestDirector, testers can design test plans, develop test cases and can run the test cases locally or remotely, on any available machine on the network. Automated test scripts generated by other testing tools (e.g., Mercury s QTP, LoadRunner) can be launched using TestDirector. The testers can run both manual and automated tests, report execution results, and enter defects. Business analysts can define application requirements and testing objectives. Project managers can generate reports on project status and above all, TestDirector allows teams to access test cases anytime, from any location via a web browser The choice of TestDirector was driven by requirement for more widespread sharing of test cases between teams. At the time, the company was undergoing a rapid shift to distributed software development, with growing offices around the world. TestDirector focused test case management and was intended to preserve business logic associated with requirements specifications (for those teams that developed from fixed requirements) and test cases. 4. DISCUSSION Based on experience from the case study, an initial impact assessment was preformed to determine the results of using the centralized test case management approach. A follow-on assessment was performed one year later as well, to ascertain the longer-term impact of the approach in the project as it evolved over time. Based on these assessments, a number of challenges were identified, and preliminary recommendations suggested. 4.1 Impact assessment Initial impact Initially, projects in active development were not directly impacted by a demand to migrate test cases to Test Director, as centrally funded resources (summer interns) were loaned to each product team to migrate legacy regression tests. Later, there were impacts, as testers had to spend time learning the tool and discovering ways to adapt it for effective use on their projects. As project teams were told to use TestDirector for storing all test cases, and execute the test cases using the tool, differences in adoption rates were encountered and limitations of the tool were found. It turned out that migration of the test cases from Excel files to TestDirector was not sufficient to encourage its usage. Limitations of the tool discouraged testers to adopt the usage of it. For example, the tool had restrictions on the number of columns that could be used to describe the test cases. For some products, the test cases were shipped to the customers with the products. The test cases exported from TestDirector were not in the format preferred by the customers. TestDirector could only export to Word and text file formats whereas the customers preferred Excel spreadsheets. This discouraged testers to adopt the usage of TestDirector since they had to find an alternative solution to meet customer demand. TestDirector also had limitations of Software Configuration Management (SCM) and it did not match the test cases to the version and branch management of the source code. For example, if version 1.1 of a product needed to be released, the test cases for that version of the product could be labeled as 1.1 to match the SCM label for the release. If, at some point, a branch release (i.e., a customized variation of the actual release) needed to be made, a version 1.1a could be created, the test cases could be updated and then labeled as 1.1a. Any fixes or enhancements and the test cases used in 1.1a would be preserved for future releases by merging to future branches of this version, for example, release 2.0 source code branch. But this type of functionality is not implemented in TestDirector and limited the usage of the tool as a repository for automated test cases. Although the immediate impact of the conversion effort was not 100% successful, it was beneficial in most respect. The immediate positive result was the drop in the rate of test case loss. Moreover, test cases and test plans became more transparent, enabling each product team to see how other products were tested. A greater positive impact of the conversion was seen after a year of usage of the tool One year later An assessment of the usage of TestDirector was done one year after the test cases were ported from the Excel spreadsheet to TestDirector. It was found that some teams adapted the usage of the tool but others still maintained test cases outside of 84
4 TestDirector. There were three reasons why these teams could not adopt the tool: (1) some could not use the tool because of its limitations; (2) some would not use it because they were insufficiently bought-in to the tool; (3) and some found successful alternative tools that they were familiar with and that did not have limitations that TestDirector appeared to have from their perspective. Some teams would not adopt the tool because of its technical limitations. After a year of usage, it was found that teams that had large quantities of automated test cases did not want to store test cases in TestDirector due to its poor performance in execution of the automated test cases from TestDirector. The test scripts and object repositories were very large and TestDirector took an unacceptable amount of time to load them before execution. Such teams abandoned the use of TestDirector as a repository for automated test cases favoring storage in the same version control system that was being used for the project s source code. Most of the test cases for such teams were automated and there existed very few manual test cases. Therefore, the utility of TestDirector was greatly diminished for such teams. Even after one year of usage, there were teams that could not overcome the limitations of TestDirector for representation of test case scenarios. For example, some teams had test cases that were written with more columns than are allowed in TestDirector, and the customer had specified the test case format, so it could not be changed. Other teams also had formats deliverable to customers that could not be changed quickly to match TestDirector s native output. Another important issue that was holding back the widespread adoption of the tool was its price. The software was rather expensive and yet specification and documentation information needed to be widely shared within the company. If each employee at the company needed to access specifications required a license to use TestDirector, the tool itself would become prohibitively expensive. Nevertheless, the usage of the tool has grown steadily since the completion of the project. Test cases are now shared with other teams, and details of test cases are now visible to upper management as well as to team members on other projects. This enables reuse of test cases and test management strategies. 4.2 Challenges Some of the challenges that were faced during the project in the porting task are described below Testing terminology The importance of having standard testing terminology was not realized until the start of this project. As mentioned above, TestDirector had a specific format to represent the test cases. For manual test cases, a simple format was needed where each test case would have the steps of the test execution, expected results and any necessary comments. It was not obvious until the start of this project how testing terminology could result in confusion and delay. Simple terms such as test case, test steps, and test scripts had different meanings to each tester. There were no standards set in place for testing terminologies in the company or even if there were, no one followed them. This was exposed during the implementation of this project. Some referred to the default 1, 2, 3 test executions steps as test cases and to the Excel files as test scripts where in reality, the test scripts are referred to automated test cases File format In order to import the test cases from excel files to TestDirector, the Excel files needed to be in a particular format. Converting the Excel files to fit the format that TestDirector would accept was a challenge for this project and took the largest amount of time. Just as the testing terminologies did not have any standards across the board, so were creations of the Excel files. Every tester had his own file format. TestDirector was set up to have four columns of information. The first column indicated what step is being executed in the testing process and is indicated by a number, the second column is the steps required to perform the test, the third column about the expected result that the test is supposed to have, and the fourth column is used for comments. The Excel files that the testers were maintaining their test cases in had more than four columns. Table 1 shows the format of the file that TestDirector would accept and Table 2 shows the actual Excel file that was maintained by testers. In order to port the test cases to TestDirector, the Excel files needed to be setup in a way that TestDirector could map the columns in the Excel file to the columns in TestDirector. It was very difficult to assess what information was important and what could be thrown away. Some of the test cases were written for a previous version of the product, by previous testers who were no longer in the company, but the test cases were never updated. It was a time consuming and difficult task to compress the many columns of information into four columns only. Many hours of manual work, interactions and brainstorming with testers and business analysts, use of Word and Excel macros, parsing algorithms in Visual Basic for Applications (VBA), and rewriting many of the test cases were required to port the test cases into TestDirector. The process took more effort than expected. There were no consistencies in the file format between testers. Therefore, it was not possible to figure out one solution that would convert all the Excel files to the correct format accepted by TestDirector and import the test cases. Multiple solutions needed to be in place and some solutions were specific and could be applicable to only a few Excel files Embracing the change With an extended amount of effort and time put on this project, thousands of test cases from various products across the company were ported into TestDirector successfully. Any tester could now open a test case and read the requirements for that particular feature to facilitate their testing activities. But, the successful completion of porting the test cases to TestDirector did not end the test management process. Only a global repository was found from where test cases could be maintained. But the maintenance and management mostly depended on the testers that were writing and updating the test cases and adopting this change. As challenges were faced from converting test cases stored in Excel files to TestDirector, similar challenges were faced when testers were told to change their work habits. The testers were already familiar with the news of porting test cases that was going on. The message was sent across the testing team that all the test cases have been successfully ported and they no longer can use Excel or Word files or any other utility at their convenience to 85
5 store test cases. Nor could they be stored in their local machines anymore. Several training sessions were held and presentations were conducted to get the employees familiar with using TestDirector. Adoption to change was not taken easily by the testers. Among some of the challenges that were faced were employees not having enough trust on the managerial decision. Questions such as How long will TestDirector be around before we move to some other tool? were raised. The issue of investing time and effort to adopt a new tool and finding out that it may not be around for long was a concern. 4.3 Recommendations The key to effective test management is communication among different parties involved in the process. Before reporting mechanisms can be put into place, the testing team needs to set the ground rules, such as defining the terminologies and agreeing upon the general testing terminologies, what is considered severity of the bugs and agreeing on what information must be included in test cases as well as defect reports. This terminologies may or may not follow a known standard but it should be an agreed upon standard that is accepted and followed by all the testers across the board. The key to making a good test case is supplying testers with clear and concrete information so they can follow and test the product. A test case should include a list of steps needed to test the product, expected result, and any relevant information that is necessary for the testers. But this format of creating the test cases should also be agreed upon and standardized for all testers. It is critical that everyone follows the standard so there is no conflict when a new person joins the team. The creation of test cases should also be prioritized according to highest value (domain testing) and highest impact (extreme negative impact from likely accidental inputs). This may require that the tester receive user-level training on the application or even on the business area. Understanding high impact actions will likely require a good understanding of the product architecture. In any case, this may not lead to portable test cases, but it will likely lead to reusable testing patterns. Test cases should also be divided into commonly used subset (sanity test) and deep functionality components. Manual testing typically requires that a large number of test cases be run against new functionality. It will likely not be feasible to run all of those manual test cases against the software on successive regression tests, so a basic subset of the functionality is typically targeted for inclusion in a sanity test [3]. Adoption of the test case management tool should not be left to each employee. They should be sent to training classes, evangelized, supported and their progress should be measured. A tool should be chosen with an eye in the future. The goal for a tool might be preservation of domain-specific business knowledge, but the choice (or available choices) must allow for sufficient information-sharing, test automation and release branching. When purchasing tools, all aspect of the functionality of a tool and the limitations of the tool should be accessed thoroughly. Limitations of tools discourage employees from adopting the usage of the tool which in this project did take place. Enough research on the tool before investing money in it may eliminate this. 5. SUMMARY There is little doubt that testing can be an effective means of improving the quality of software applications. The problem is that testing rarely receives the proper attention that it deserves. Test case management is an excellent example of an activity that is extremely difficult to master particularly when manual intervention is needed at all stages of the process. This paper presented the results of a case study in test case management in an industrial setting. The approach was intended to address some of the shortcomings of real-world management test case management, such as variations in terminology, file format interchange problems, and overall centralization through automation. The approach has been in use for over a year and, although there are still issues to be resolved (e.g., managing change and getting buy-in from all the engineers with the new tools introduced into the testing process), the result can be seen as moderately successful. The experience in executing the case study has opened up several avenues for future work. For example, the recommendations outlined in the paper should be empirically validated in further studies. There is an ever-present need to increased automation the test case management activities, since automation is one way of codifying best practices in tools that can better aid less experiences software testers. Finally, there remain a number of opportunities for academics to learn from industrial practice, and perhaps incorporate some of the lessons learned in case studies such as those presented here into the curricula. REFERENCES [1] Perry, W.; Effective Methods for Software Testing (2 nd Ed.), Wiley, [2] Harrold, M. J. Testing: A Roadmap. In the International Conference on Software Engineering, (Limerick, Ireland, 2000). [3] Kaner, C.; Bach J.; and Pettichord, B. Lessons Learned in Software Testing: A Context Driven Approach. New York: Wiley & Sons Inc., [4] Mercury, Last accessed - November 22, [5] Miller K. and Voas J., Software Test Cases: Is One Ever Enough? IT Pro, January-February 2006, pp [6] Kaner, C. Testing Computer Software (2 nd Ed). John Wiley & Sons, [7] G. J. Myers, The Art of Software Testing. New York, NY: Wiley-Interscience, [8] J. A. Whittaker, How to Break Software: A Practical Guide to Testing: Pearson Addison Wesley, [9] Kaner, C. Architectures of Test Automation. Software Testing, Analysis & Review Conference (Star) West, San Jose, CA, Oct [10] B. Korel, Automated Test Data Generation. IEEE Transactions on Software Engineering, vol. SE-16, NO. 8, August 1990, pp [11] Kaner, C. Improving the Maintainability of Automated Test Suites, Software QA, 4(4),
6 [12] Beck K., Test-Driven Development: By Example, Addison- Wesley Professional, Table 1. Format of the Excel file that TestDirector would accept Table 2. Format of the actual Excel file that the testers followed 87
BUSINESS RULES AND GAP ANALYSIS
Leading the Evolution WHITE PAPER BUSINESS RULES AND GAP ANALYSIS Discovery and management of business rules avoids business disruptions WHITE PAPER BUSINESS RULES AND GAP ANALYSIS Business Situation More
More informationKeywords document, agile documentation, documentation, Techno functional expert, Team Collaboration, document selection;
Volume 4, Issue 4, April 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Document Driven
More informationLevels of Software Testing. Functional Testing
Levels of Software Testing There are different levels during the process of Testing. In this chapter a brief description is provided about these levels. Levels of testing include the different methodologies
More informationAnalysis of Object Oriented Software by Using Software Modularization Matrix
Analysis of Object Oriented Software by Using Software Modularization Matrix Anup 1, Mahesh Kumar 2 1 M.Tech Student, 2 Assistant Professor, Department of Computer Science and Application, RPS College,
More informationAGILE SOFTWARE TESTING
AGILE SOFTWARE TESTING Business environments continue to rapidly evolve, leaving many IT organizations struggling to keep up. This need for speed has led to an increased interest in the Agile software
More informationJOURNAL OF OBJECT TECHNOLOGY
JOURNAL OF OBJECT TECHNOLOGY Online at http://www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2007 Vol. 6, No. 1, January-February 2007 CM Configuration Change Management John D.
More information101-301 Guide to Mobile Testing
101-301 Guide to Mobile Testing Perfecto Mobile & Toronto Association of System and Software Eran Kinsbruner & Joe Larizza 2014 What To Do? Great News Your first Mobile Project has arrived! You have been
More informationWeb Application Regression Testing: A Session Based Test Case Prioritization Approach
Web Application Regression Testing: A Session Based Test Case Prioritization Approach Mojtaba Raeisi Nejad Dobuneh 1, Dayang Norhayati Abang Jawawi 2, Mohammad V. Malakooti 3 Faculty and Head of Department
More informationBasic Unified Process: A Process for Small and Agile Projects
Basic Unified Process: A Process for Small and Agile Projects Ricardo Balduino - Rational Unified Process Content Developer, IBM Introduction Small projects have different process needs than larger projects.
More informationPicasso Recommendation
Picasso Recommendation Mission The School of Dentistry for the University at Buffalo (UB) uses a custom application called Picasso to manage their clinic operations. Developed by two engineers in 1997,
More informationSQA Labs Value Assured
Q SQA Labs Value Assured QUALITY ASSURANCE TESTING TOOLS QUALITY ASSURANCE TESTING TOOLS Quality Assurance refers to the steps taken to make sure that a company s products or services are of sufficiently
More informationSOFTWARE TESTING TRAINING COURSES CONTENTS
SOFTWARE TESTING TRAINING COURSES CONTENTS 1 Unit I Description Objectves Duration Contents Software Testing Fundamentals and Best Practices This training course will give basic understanding on software
More informationRealizing business flexibility through integrated SOA policy management.
SOA policy management White paper April 2009 Realizing business flexibility through integrated How integrated management supports business flexibility, consistency and accountability John Falkl, distinguished
More informationCo-Presented by Mr. Bill Rinko-Gay and Dr. Constantin Stanca 9/28/2011
QAI /QAAM 2011 Conference Proven Practices For Managing and Testing IT Projects Co-Presented by Mr. Bill Rinko-Gay and Dr. Constantin Stanca 9/28/2011 Format This presentation is a journey When Bill and
More informationCentercode Platform. Features and Benefits
Centercode Platform s and s v1.2 released July 2014 Centercode s and s 2 Community Portal Host a secure portal for your candidates and participants Your Own Private Beta Portal Centercode provides your
More informationImplementing Continuous Integration Testing Prepared by:
Implementing Continuous Integration Testing Prepared by: Mr Sandeep M Table of Contents 1. ABSTRACT... 2 2. INTRODUCTION TO CONTINUOUS INTEGRATION (CI)... 3 3. CI FOR AGILE METHODOLOGY... 4 4. WORK FLOW...
More informationApplying Agile Methods in Rapidly Changing Environments
Applying Agile Methods in Changing Environments 7/23/2002 1 Applying Agile Methods in Rapidly Changing Environments Peter Kutschera IBM Unternehmensberatung GmbH Am Fichtenberg 1, D-71803 Herrenberg Steffen
More informationInformation Systems Development Process (Software Development Life Cycle)
Information Systems Development Process (Software Development Life Cycle) Phase 1 Feasibility Study Concerned with analyzing the benefits and solutions for the identified problem area Includes development
More informationSuccessful Test Case Management: It Takes More than a Spreadsheet
Successful Test Case Management: It Takes More than a Spreadsheet Testing software applications can require hundreds to thousands of unique test cases, weeks or even longer to execute them, and the ability
More informationQuality Assurance - Karthik
Prevention is better than cure Quality Assurance - Karthik This maxim perfectly explains the difference between quality assurance and quality control. Quality Assurance is a set of processes that needs
More informationOracle Insurance Policy Administration System Quality Assurance Testing Methodology. An Oracle White Paper August 2008
Oracle Insurance Policy Administration System Quality Assurance Testing Methodology An Oracle White Paper August 2008 Oracle Insurance Policy Administration System Quality Assurance Testing Methodology
More informationBuilding Lab as a Service (LaaS) Clouds with TestShell
Building Lab as a Service (LaaS) Clouds with TestShell Introduction Test labs represent a large ongoing investment and an opportunity for significant savings. Many technology organizations are seeking
More informationBusiness white paper. Best practices for implementing automated functional testing solutions
Business white paper Best practices for implementing automated functional testing solutions Table of contents Contents 3 Introduction 3 Functional testing versus unit testing 4 The pros and cons of manual
More informationAgile Software Development Methodologies and Its Quality Assurance
Agile Software Development Methodologies and Its Quality Assurance Aslin Jenila.P.S Assistant Professor, Hindustan University, Chennai Abstract: Agility, with regard to software development, can be expressed
More informationCustom Software Development Approach
Custom Software Development Approach Our approach to custom software development combines benefits from several standard development process models. We tend to have a well-defined, predictable and highly
More informationTesthouse Training Portfolio
Testhouse Training Portfolio TABLE OF CONTENTS Table of Contents... 1 HP LoadRunner 4 Days... 2 ALM Quality Center 11-2 Days... 7 HP QTP Training Course 2 Days... 10 QTP/ALM Intensive Training Course 4
More informationSuccessfully managing geographically distributed development
IBM Rational SCM solutions for distributed development August 2004 Successfully managing geographically distributed development Karen Wade SCM Product Marketing Manager IBM Software Group Page 2 Contents
More informationA Knowledge Management Framework Using Business Intelligence Solutions
www.ijcsi.org 102 A Knowledge Management Framework Using Business Intelligence Solutions Marwa Gadu 1 and Prof. Dr. Nashaat El-Khameesy 2 1 Computer and Information Systems Department, Sadat Academy For
More informationTable of contents. Performance testing in Agile environments. Deliver quality software in less time. Business white paper
Performance testing in Agile environments Deliver quality software in less time Business white paper Table of contents Executive summary... 2 Why Agile? And, why now?... 2 Incorporating performance testing
More informationRegression Testing Based on Comparing Fault Detection by multi criteria before prioritization and after prioritization
Regression Testing Based on Comparing Fault Detection by multi criteria before prioritization and after prioritization KanwalpreetKaur #, Satwinder Singh * #Research Scholar, Dept of Computer Science and
More informationCHAPTER 7 Software Configuration Management
CHAPTER 7 Software Configuration Management ACRONYMS CCB CM FCA MTBF PCA SCCB SCI SCM SCMP SCR SCSA SEI/CMMI SQA SRS USNRC INTRODUCTION Configuration Control Board Configuration Management Functional Configuration
More informationEnterprise Frameworks: Guidelines for Selection
Enterprise Frameworks: Guidelines for Selection Mohamed E. Fayad, University of Nebraska, Lincoln David S. Hamu, TRW fayad@cse.unl.edu, dhamu@acm.org An Enterprise Framework (EF) is a software architecture.
More informationSmarter Balanced Assessment Consortium. Recommendation
Smarter Balanced Assessment Consortium Recommendation Smarter Balanced Quality Assurance Approach Recommendation for the Smarter Balanced Assessment Consortium 20 July 2012 Summary When this document was
More informationSoftware Testing. Knowledge Base. Rajat Kumar Bal. Introduction
Software Testing Rajat Kumar Bal Introduction In India itself, Software industry growth has been phenomenal. IT field has enormously grown in the past 50 years. IT industry in India is expected to touch
More informationBest Practices for Improving the Quality and Speed of Your Agile Testing
A Conformiq White Paper Best Practices for Improving the Quality and Speed of Your Agile Testing Abstract With today s continually evolving digital business landscape, enterprises are increasingly turning
More information(Refer Slide Time: 01:52)
Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This
More informationImplementation Process Ensuring Effective Development and Deployment
Implementation Process Ensuring Effective Development and Deployment Highlights Our spiral process facilitates rapid and accurate development Iterative software deployment, which captures and incorporates
More informationElite: A New Component-Based Software Development Model
Elite: A New Component-Based Software Development Model Lata Nautiyal Umesh Kumar Tiwari Sushil Chandra Dimri Shivani Bahuguna Assistant Professor- Assistant Professor- Professor- Assistant Professor-
More informationPractical Experiences of Agility in the Telecom Industry
Practical Experiences of Agility in the Telecom Industry Jari Vanhanen 1, Jouni Jartti 2, and Tuomo Kähkönen 2 1 Helsinki University of Technology, Software Business and Engineering Institute, P.O. Box
More informationSKILL DEVELOPMENT IN THE ERA OF QUALITY ASSURANCE MANAGEMENT WITH RESPECT TO PRODUCTS & SERVICES BASED SOFTWARE IT ORGANIZATIONS
Tactful Management Research Journal ISSN: 2319-7943 Impact Factor : 2.1632(UIF) SKILL DEVELOPMENT IN THE ERA OF QUALITY ASSURANCE MANAGEMENT WITH RESPECT TO PRODUCTS & SERVICES BASED SOFTWARE IT ORGANIZATIONS
More informationCRM SUCCESS GUIDELINES
CRM SUCCESS GUIDELINES Provided to You By: Integrated Sales Management, Inc. Helping You Grow! CRM Success Guidelines Customer Relationship Management (CRM) has evolved dramatically recently as many companies
More informationone Introduction chapter OVERVIEW CHAPTER
one Introduction CHAPTER chapter OVERVIEW 1.1 Introduction to Decision Support Systems 1.2 Defining a Decision Support System 1.3 Decision Support Systems Applications 1.4 Textbook Overview 1.5 Summary
More informationYour Software Quality is Our Business. INDEPENDENT VERIFICATION AND VALIDATION (IV&V) WHITE PAPER Prepared by Adnet, Inc.
INDEPENDENT VERIFICATION AND VALIDATION (IV&V) WHITE PAPER Prepared by Adnet, Inc. February 2013 1 Executive Summary Adnet is pleased to provide this white paper, describing our approach to performing
More informationTop ten reasons to transition your IT lab environments to the cloud
Top ten reasons to transition your IT lab environments to the cloud WHITE PAPER BROUGHT TO YOU BY SKYTAP 2 Top ten reasons to transition your IT lab environments to the cloud Contents The Promise of Cloud
More informationIncreasing Business Efficiency and Agility for ATGbased. Systems. the business challenge: upgrading the development pipeline
Increasing Business Efficiency and Agility for ATGbased ecommerce Systems This case study follows a Tier 1 retailer migrating to an ATG-based ecommerce platform and upgrading its software development process
More informationLearning More About Load Testing
Welcome to this introduction to application performance testing and the LoadRunner load testing solution. This document provides a short overview of LoadRunner s features, and includes the following sections:
More informationSA Tool Kit release life cycle
Release management Release management process is a software engineering process intended to oversee the development, testing, deployment and support of software releases. A release is usually a named collection
More informationA Configuration Management Model for Software Product Line
A Configuration Management Model for Software Product Line Liguo Yu 1 and Srini Ramaswamy 2 1 Computer Science and Informatics Indiana University South Bend South Bend, IN 46634, USA ligyu@iusb.edu 2 Computer
More informationAutomated Testing Best Practices
Automated Testing Best Practices This document includes best practices to consider before implementing automated software testing. These best practices are strategic and are applicable regardless of the
More informationBasic Unix/Linux 1. Software Testing Interview Prep
Basic Unix/Linux 1 Programming Fundamentals and Concepts 2 1. What is the difference between web application and client server application? Client server application is designed typically to work in a
More informationComparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications
Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications White Paper Table of Contents Overview...3 Replication Types Supported...3 Set-up &
More informationQuality Management. Lecture 12 Software quality management
Quality Management Lecture 12 Software quality management doc.dr.sc. Marko Jurčević prof.dr.sc. Roman Malarić University of Zagreb Faculty of Electrical Engineering and Computing Department of Fundamentals
More informationOptimizing Your Software Process
Optimizing Your Software Process Top 5 Software Development Process Challenges Executive Summar ry A process framework is a combination of project management, technical practices, and supporting tools.
More informationALM Solutions using Visual Studio TFS 2013 ALMI13; 5 Days, Instructor-led
ALM Solutions using Visual Studio TFS 2013 ALMI13; 5 Days, Instructor-led Course Description This five day course is designed to get your entire team working efficiently with Microsoft s Application Lifecycle
More informationSoftware Continuous Integration & Delivery
November 2013 Daitan White Paper Software Continuous Integration & Delivery INCREASING YOUR SOFTWARE DEVELOPMENT PROCESS AGILITY Highly Reliable Software Development Services http://www.daitangroup.com
More informationIntroduction site management software
Web Testing Introduction Making a web site does not end with putting all the media and software together. Actually, web site work never ends. When all the design is done, you have to test the site first
More informationContinuous Integration, Delivery and Deployment. Eero Laukkanen T-76.5613 - Software Testing and Quality Assurance P 20.11.2015
Continuous Integration, Delivery and Deployment Eero Laukkanen T-76.5613 - Software Testing and Quality Assurance P 20.11.2015 System Integration In engineering, system integration is defined as the process
More informationRequirements Management
REQUIREMENTS By Harold Halbleib Requirements Management Identify, Specify, Track and Control Requirements Using a Standard Process About the author... Harold Halbleib has a degree in Electrical Engineering
More informationNASCIO EA Development Tool-Kit Solution Architecture. Version 3.0
NASCIO EA Development Tool-Kit Solution Architecture Version 3.0 October 2004 TABLE OF CONTENTS SOLUTION ARCHITECTURE...1 Introduction...1 Benefits...3 Link to Implementation Planning...4 Definitions...5
More informationExample Software Development Process.
Example Software Development Process. The example software development process is shown in Figure A. The boxes represent the software development process kernels. The Software Unit Testing, Software Component
More informationT14 Thursday, November 4, 1999 1:30 PM
Presentation Notes Paper Bio Return to Main Menu PRESENTATION T14 Thursday, November 4, 1999 1:30 PM SYSTEM TESTING STRATEGIES FOR HIGHLY AVAILABLE CLUSTERED SYSTEMS Subbarao Jagannatha Sun Microsystems
More informationImplementing Hybrid Cloud at Microsoft
Implementing Hybrid Cloud at Microsoft Published September 2013 The following content may no longer reflect Microsoft s current position or infrastructure. This content should be viewed as reference documentation
More informationDatabase Marketing, Business Intelligence and Knowledge Discovery
Database Marketing, Business Intelligence and Knowledge Discovery Note: Using material from Tan / Steinbach / Kumar (2005) Introduction to Data Mining,, Addison Wesley; and Cios / Pedrycz / Swiniarski
More informationFive Reasons why Agile Won t Scale Without Automation
Five Reasons why Agile Won t Scale Without Automation September 2011 June 2011 Introduction The Agile software development methodology is composed of a number of techniques designed to help deliver incremental
More informationTest Data Management Concepts
Test Data Management Concepts BIZDATAX IS AN EKOBIT BRAND Executive Summary Test Data Management (TDM), as a part of the quality assurance (QA) process is more than ever in the focus among IT organizations
More informationa new generation software test automation framework - CIVIM
a new generation software test automation framework - CIVIM Software Testing is the last phase in software development lifecycle which has high impact on the quality of the final product delivered to the
More informationNEOXEN MODUS METHODOLOGY
NEOXEN MODUS METHODOLOGY RELEASE 5.0.0.1 INTRODUCTION TO QA & SOFTWARE TESTING GUIDE D O C U M E N T A T I O N L I C E N S E This documentation, as well as the software described in it, is furnished under
More informationTop Ten Reasons to Transition Your IT Sandbox Environments to the Cloud
Top Ten Reasons to Transition Your IT Sandbox Environments to the Cloud WHITE PAPER BROUGHT TO YOU BY SKYTAP 2 Top Ten Reasons to Transition Your IT Sandbox Environments to the Cloud Contents Executive
More informationCoverity White Paper. Effective Management of Static Analysis Vulnerabilities and Defects
Effective Management of Static Analysis Vulnerabilities and Defects Introduction According to a recent industry study, companies are increasingly expanding their development testing efforts to lower their
More informationQuality Assurance Training Program
Quality Assurance Training Program Introduction/Summary: This 5-day course focuses on understanding and developing various skills required by QA Developer, preparing to use various tools and techniques
More informationMetadata Quality Control for Content Migration: The Metadata Migration Project at the University of Houston Libraries
Metadata Quality Control for Content Migration: The Metadata Migration Project at the University of Houston Libraries Andrew Weidner University of Houston, USA ajweidner@uh.edu Annie Wu University of Houston,
More informationThe Worksoft Suite. Automated Business Process Discovery & Validation ENSURING THE SUCCESS OF DIGITAL BUSINESS. Worksoft Differentiators
Automated Business Process Discovery & Validation The Worksoft Suite Worksoft Differentiators The industry s only platform for automated business process discovery & validation A track record of success,
More informationTable of contents. HP Software customer perspective: using HP TestDirector for Quality Center software to report and resolve software defects
HP Software customer perspective: using HP TestDirector for Quality Center software to report and resolve software defects White paper Table of contents About the author................................................2
More informationKnowledge Infrastructure for Project Management 1
Knowledge Infrastructure for Project Management 1 Pankaj Jalote Department of Computer Science and Engineering Indian Institute of Technology Kanpur Kanpur, India 208016 Jalote@iitk.ac.in Abstract In any
More informationService Virtualization:
Service Virtualization: Reduce the time and cost to develop and test modern, composite applications Business white paper Table of contents Why you need service virtualization 3 The challenges of composite
More informationWindows 7 Upgrade Risk Mitigation Planning: Ensuring Windows 7 Upgrade Success
November 2010 Windows 7 Upgrade Risk Mitigation Planning: Ensuring Windows 7 Upgrade Success By: Robert Schaper, Laplink Software Enterprise Solutions Contents Introduction 2 Risk and Cost 2 Upgrade Strategies
More informationAdvancements in the V-Model
Advancements in the V-Model Sonali Mathur Asst. Professor, CSE Dept. ABES Institute of Technology Ghaziabad, U.P-201009 Shaily Malik Lecturer, CSE Dept. Maharaja Surajmal Institute of Tech. Janakpuri,
More informationComparing Methods to Identify Defect Reports in a Change Management Database
Comparing Methods to Identify Defect Reports in a Change Management Database Elaine J. Weyuker, Thomas J. Ostrand AT&T Labs - Research 180 Park Avenue Florham Park, NJ 07932 (weyuker,ostrand)@research.att.com
More informationCAN DISTRIBUTED SOFTWARE DEVELOPMENT BE AGILE?
By BALASUBRAMANIAM RAMESH, LAN CAO, KANNAN MOHAN, and PENG XU CAN DISTRIBUTED SOFTWARE DEVELOPMENT BE AGILE? Three organizations studied here suggest the answer is yes, when the unique characteristics
More informationRequirements-Based Testing: Encourage Collaboration Through Traceability
White Paper Requirements-Based Testing: Encourage Collaboration Through Traceability Executive Summary It is a well-documented fact that incomplete, poorly written or poorly communicated requirements are
More informationThe Power of Process, People, and Tools When Testing a Complex Integration Landscape for a Very Large Initial Retail ERP Implementation
The Power of Process, People, and Tools When Testing a Complex Integration Landscape for a Very Large Initial Retail ERP Implementation Nicki Nicolo Nike, Inc. Nicki.Nicolo@nike.com Abstract Anyone who
More informationKey Benefits of Microsoft Visual Studio Team System
of Microsoft Visual Studio Team System White Paper November 2007 For the latest information, please see www.microsoft.com/vstudio The information contained in this document represents the current view
More informationThe Quality Data Warehouse: Solving Problems for the Enterprise
The Quality Data Warehouse: Solving Problems for the Enterprise Bradley W. Klenz, SAS Institute Inc., Cary NC Donna O. Fulenwider, SAS Institute Inc., Cary NC ABSTRACT Enterprise quality improvement is
More informationWe are live on KFS Now What? Sameer Arora Director Strategic Initiatives, Syntel
We are live on KFS Now What? Sameer Arora Director Strategic Initiatives, Syntel Agenda Introduction Application Management Testing Kuali Financial System (KFS) using itap Syntel Fast Facts 2 Agenda Introduction
More informationA Tool for Mining Defect-Tracking Systems to Predict Fault-Prone Files
A Tool for Mining Defect-Tracking Systems to Predict Fault-Prone Files Thomas J. Ostrand AT&T Labs - Research 180 Park Avenue Florham Park, NJ 07932 ostrand@research.att.com Elaine J. Weyuker AT&T Labs
More informationBasic Trends of Modern Software Development
DITF LDI Lietišķo datorsistēmu programmatūras profesora grupa e-business Solutions Basic Trends of Modern Software Development 2 3 Software Engineering FAQ What is software engineering? An engineering
More informationComparing Agile Software Processes Based on the Software Development Project Requirements
CIMCA 2008, IAWTIC 2008, and ISE 2008 Comparing Agile Software Processes Based on the Software Development Project Requirements Malik Qasaimeh, Hossein Mehrfard, Abdelwahab Hamou-Lhadj Department of Electrical
More informationContinuous integration for databases using
Continuous integration for databases using Red Wie Sie Gate die tools Microsoft SQL An overview Continuous integration for databases using Red Gate tools An overview Contents Why continuous integration?
More informationA Closer Look at BPM. January 2005
A Closer Look at BPM January 2005 15000 Weston Parkway Cary, NC 27513 Phone: (919) 678-0900 Fax: (919) 678-0901 E-mail: info@ultimus.com http://www.ultimus.com The Information contained in this document
More informationWhat Is Specific in Load Testing?
What Is Specific in Load Testing? Testing of multi-user applications under realistic and stress loads is really the only way to ensure appropriate performance and reliability in production. Load testing
More informationAn Increase in Software Testing Robustness: Enhancing the Software Development Standard for Space Systems
An Increase in Software Robustness: Enhancing the Software Development Standard for Space Systems Karen Owens and Suellen Eslinger Software Engineering Subdivision 15 th Ground System Architectures Workshop
More informationWeb-based Reporting and Tools used in the QA process for the SAS System Software
Web-based Reporting and Tools used in the QA process for the SAS System Software Jim McNealy, SAS Institute Inc., Cary, NC Dawn Amos, SAS Institute Inc., Cary, NC Abstract SAS Institute Quality Assurance
More informationExploratory Testing Dynamics
Exploratory Testing Dynamics Created by James Bach, Jonathan Bach, and Michael Bolton 1 v2.2 Copyright 2005-2009, Satisfice, Inc. Exploratory testing is the opposite of scripted testing. Both scripted
More informationThe George Washington University
PMLC Project Management Life Cycle The George Washington University eexpense System Implementation Project Test Plan & Procedures Prepared By: Jeff Pearson Version: 1 Date: August 13, 2012 Project Owners:
More informationEFFECTIVE APPROACH FOR DYNAMIC TEST CASE GENERATION FOR LOAD TESTING OF HTTP WEB SERVER
EFFECTIVE APPROACH FOR DYNAMIC TEST CASE GENERATION FOR LOAD TESTING OF HTTP WEB SERVER Shweta Ahuja M.Tech. Research Scholar Computer Science and Engineering Guru Nanak Institute of Technology Mullana,
More informationExploratory Testing Dynamics
Exploratory Testing Dynamics Created by James and Jonathan Bach 1 v1.6 Copyright 2005-2006, Satisfice, Inc. Exploratory testing is the opposite of scripted testing. Both scripted and exploratory testing
More informationBanking Application Modernization and Portfolio Management
Banking Application Modernization and Portfolio Management Key Challenges and Success Factors As part of their long-term strategic plans, banks are seeking to capitalize on their legacy applications. Acquired
More informationCT30A8901 Chapter 10 SOA Delivery Strategies
CT30A8901 Chapter 10 SOA Delivery Strategies Prof. Jari Porras Communications Software Laboratory Contents 10.1 SOA Delivery lifecycle phases 10.2 The top-down strategy 10.3 The bottom-up strategy 10.4
More information