Towards Checking the Usefulness of Verification Tools
|
|
- Bethany Bradford
- 8 years ago
- Views:
Transcription
1 Towards Checking the Usefulness of Verification Tools Willem Visser and Jaco Geldenhuys Computer Science Division Department of Mathematical Sciences University of Stellenbosch, South Africa Abstract. Building verification tools is what researchers in the field are good at, but evaluating if they are actually useful and useable is often a much harder problem. We propose a very simple idea to address this problem, namely, a framework in which we capture what a programmer does during a programming exercise and mechanisms to then allow us to incorporate verification (including testing) tools to analyze the resulting code snapshots. The ultimate idea is to build up a very diverse set of code benchmarks on which verification tools can be evaluated. However unlike current benchmarks that are somewhat cherry-picked for showing off certain tools strengths, this one will be based on actual code being captured during programming. 1 Introduction This work stems from our desire to capture the errors that programmers make during the actual code development; more precisely we want to know exactly when an error is introduced and when is removed. In addition we want to capture the type of the errors. This requires a way to closely observe programmers as they develop code. For example, looking at revision repositories is not at a fine enough level of granularity: by the time code is committed many errors may already have been removed and we don t really know how much time was wasted on finding and repairing the code. We therefore take a code snapshot whenever a developer saves a file. Once we have the snapshots we can analyze each snapshot to see which errors occurs. Of course one needs to know what the code is supposed to do. In this first iteration of our approach we fix the domain to observe students doing well-specified programming puzzles, akin to Google Code Jam or the ACM programming competitions. We can thus construct what we refer to as the perfect oracle, which concretely consists of a large test suite that the code must pass to be considered correct. The code snapshots plus the test suite makes it possible to classify each snapshot in terms of which snapshots are correct and which ones have errors. A natural extension of this approach is to not just run the code to determine which errors are present, but to also run tools on the code to try and find the errors
2 2 Willem Visser and Jaco Geldenhuys automatically. Our current framework only supports one type of tool, namely static analysis tools for runtime error detection. However the results obtained already seem to be in stark contrast to common beliefs about the usefulness of these kinds of tools: we found that even well-respected tools like FindBugs [2, 8, 6] didn t find a single one of the errors that actually occurred (we also tried Lint4J [4] and Tpgen [10]). In the following we describe our current framework in more detail and also discuss the evaluation of the static analysis tools. Note that although our current system only works within the Eclipse IDE, it can trivially be extended to other languages and IDEs. We currently focus on bug finding tools, but nothing prevents us from also adding verification tools to analyze the snapshots to check which ones are easy to prove correct or invalid, the level of annotations required, the need for human intervention, etc. 2 Framework The framework for our experiments consists of two parts: Intlola, a data-gathering IDE plug-in, and Impendulo, a data analysis and visualisation tool [11]. As they are used during two distinct phases, the components are entirely independent of each other. 2.1 Gathering the Data We have implemented a plug-in for the Eclipse IDE platform that operates in the background. All that is needed is for the user to switch on recording for a particular project. As the user develops their program, the plug-in takes a complete snapshot of the project whenever code is saved, either explicitly by the user or implicitly by the IDE. This happens, for instance, when the user compiles or executes the code. The program code is recorded along with a small amount of meta-data. Collecting the data is fast and therefore unobtrusive, and requires a reasonably small amount of space. The data-gathering plug-in is not tied inexorably to the Eclipse platform, and, in fact, we plan to develop similar plug-ins for other environments. Once the experiment is completed, the stored snapshots are collected in an archive and passed to the next phase of the experiment. 2.2 Analysing the Data While individually small, the collected snapshots for a group of participants represent a large amount of raw data, and need to be organized and managed effectively. We have implemented a visualization tool to help with this. The tool has three components: (1) a central data manager that keeps track of different projects, different participants for each project, different snapshots for each participant, and the different files in each snapshot; (2) a modular architecture for managing different forms of analyses; and (3) a modular architecture for visualizing the output of the different analyses and the overall view on a participant or project.
3 Towards Checking the Usefulness of Verification Tools 3 The data manager and visualization modules are, if not trivial, at least straightforward, and the biggest challenge is integrating diverse forms of analyses into a uniform design that makes it easy to add new techniques to the tool. Some basic tools such as the compiler and unit tests are standard fixtures, but to be able to add different forms of analyses, we settled on the following plug-and-play style: When the user requests an analysis of a participant s work, the relevant snapshots are unpacked one by one. For each snapshot, a copy of the participant s work is reconstructed. The analysis may involve one or more tools, depending on how the system is configured. The location of the unpacked snapshot is passed to each tool in turn. The tool performs the analysis and captures and interprets the output. Mechanisms for persistent storage of results are provided, so that the same snapshot need not be analysed more than once by the same tool. Tools are expected to produce both verbose and summary output. As a simple example, the compiler is implemented as such a tool. Given a snapshot location, it invokes the Java compiler and records the compiler s output for the user to inspect. It also parses the output to collect information about the number and nature of the errors. 2.3 Visualization The visualization tools can be quite useful. For example, the graph in Figure 1 shows the progress of a particular student at a glance. In this context, each snapshot must pass three stages. First it is compiled, then it is subjected to a small set of tests ( Easy ), and finally to a large, complete set of tests ( All ). The smaller set is convenient for programmers to find initial bugs. For each of the two sets, a snapshot can either produce failures (terminate abnormally), or errors (terminate normally but produce the wrong answer). Note that the large set also checks the scalability of the code hence we additionally record if tests timeout. Each dot in the figure represents a snapshot: its horizontal position denotes the time it was taken, and its vertical position is its status. Those snapshots on the first (lowest) line do not compile at all, those on the next line compile but produce Easy failures, those on next line produce Easy errors, and so on. Those on the top line pass all the tests. If a snapshot appears at a certain level, this means that it has successfully passed all of the levels below it. 2.4 Extensibility The architecture allows users to develop and add their own visualizations, analysis tools and tool chains tools that invoke a sequence of other tools. Also, it remains flexible enough to accommodate almost any kind of analysis.
4 4 Willem Visser and Jaco Geldenhuys Fig. 1. Visualization of a participant s progress
5 3 Experimental Results Towards Checking the Usefulness of Verification Tools 5 We have conducted several trials runs of the system. An overview of the data collected is shown in Table 1. The S column shows the number of students, and the P shows the number of snapshots. This is followed by the average number of snapshots per student, the average number of seconds between snapshots, and the total time spent on the problem by all of the students combined. Table 1. Overview of collected data Average Total Problem S P P/S interval time welcome h12m kselect h47m triangle h22m watersheds h33m triangle h53m LRS h41m welcome h55m Total Problem Selection The choice of problem is a critical aspect of the experiments. The problems were chosen to be challenging but doable within one hour each. We have tried to avoid problems that require deep mathematical insight and concentrate on more traditional computer science problems. In the welcome problem the programmer is given an input string and asked to count how many times a second, fixed string appears as a non-contiguous substring of the first. kselect presents the programmer with an array of integers that belongs together in pairs (i.e., the first two numbers form the first pair, the next two numbers form the second pair, and so on); the task is to sort the pairs lexicographically. As an example of what a problem specification looks like we added the kselect specification to the Appendix. The triangle problem asks the programmer to find the longest path from a specified vertex in a triangle-shaped acyclic directed graph (with a potentially exponential number of paths). triangle2 refers to the same problem given to a different set of students. LRS is a problem in which the programmers needed to find the longest recurring substring. This problem was given to a class of first year students and contains the most snapshots of all.
6 6 Willem Visser and Jaco Geldenhuys Lastly, in the watersheds problem the programmer is given a two-dimensional integer array of function values; the task is to identify the local minima and the closest minimum for each array element according to a fixed definition of distance. watersheds2 refers to the same problem given to a different set of students. The welcome and watersheds problems were taken from the annual Google Code Jam programming competition, the triangle from a puzzle website, and the kselect problem is original. (It appears in the Appendix) Generally speaking, we tried to make the students think in a programmatic way, and to make them exercise their technical programming skills. For example, because the numbers in the kselect problem are paired, the students are not able to use a standard library sorting routine. We expected the students to make different kinds of errors both across the problems, and within a single problem, and we hoped that the analysis tools would be able to identify some of these errors. 3.2 Results To our surprise the results from the tools were quite dismal. Lint4J reported basically no errors on any of the examples, FindBugs reported some possible errors but not the ones that occurred when running the code, and Tpgen reported a few errors that actually occurred but also missed many. Note that the code fragments were relatively small and all the tools ran within seconds; we also tried to fine-tune each tool to make it perform to the best of its ability. One of the primitive visualizations provided by our tool allows the user to view the raw output of the analysis tools. A typical sample of code and the corresponding output is shown in Figures 2 and 3; both trimmed for the sake of brevity. FindBugs complains about a futile assignment in line 32, while Tpgen reports a null pointer exception on line 13 (a false positive) and an index-out-ofbounds error on line 46. In fact, most of the Easy tests produced errors because of an array-out-of-bounds error on line 33. Lint4J can be somewhat exonerated because the errors that did occur often, namely null pointer dereferences and index-out-of-bounds accesses, are not errors that it looks for. Still it seems that even these novice programmers didn t make the kind of silly mistakes Lint4J catches. FindBugs on the other hand should have found these errors but didn t. We believe it is due to FindBugs being tuned for large code bases and thus it doesn t report errors unless it is quite certain it will be triggered during execution. Tpgen, which is the tool that most closely follows real executions caught the most real bugs, but it did suffer from scalability issues since some of the errors occur at path depths beyond which it can scale to. However it is just a research prototype and we believe that it shows that path sensitive analyses are an important avenue for future work in applying static tools early in the development cycle.
7 Towards Checking the Usefulness of Verification Tools 7 5 public int kselect(int k, int[] V) { 6 int temp1 = 0, temp2 = 0; 10 if (k < 0) { k = k*(-1); } 13 if (k > ((V.length)/2)) { return 0; } 18 for (int i = 0; i < V.length; i += 2){ 19 for (int j=i+2; j < V.length; j += 2) 20 if (V[j] <= V[i]){ 21 temp1 = V[i]; 22 temp2 = V[i+1]; 23 V[i] = V[j]; 24 V[i+1] = V[j+1]; 25 V[j] = temp1; 26 V[j+1] = temp2; 27 } 28 } 31 for (int i = 0; i < V.length; i += 2){ 32 int j = i; 33 while (V[i] == V[i+2]) { 34 if (V[i+1] > V[i+3]){ 35 temp1 = V[i]; 36 temp2 = V[i+1]; 37 V[i] = V[i+2]; 38 V[i+1] = V[i+3]; 39 V[i+2] = temp1; 40 V[i+3] = temp2; 41 } 42 i = i+2; 43 } 44 } 46 return V[k]; 47 } Fig. 2. The source code corresponding to Figure 3
8 8 Willem Visser and Jaco Geldenhuys FindBugs: Tpgen: Dead store to $L6 in kselection.kselection.kselect(int, int[]) At KSelection.java:[line 32] Running 5 test(s)... 1) Solution (0) REAL? Caught expected exception: java.lang.nullpointerexception Occurred at kselection.kselection.kselect: ) Solution (2) REAL? Caught expected exception: java.lang.arrayindexoutofboundsexception: 0 Occurred at kselection.kselection.kselect:46 Easy tests: Time: There were 13 failures: 1) java.lang.arrayindexoutofboundsexception: 12 at kselection.kselection.kselect(kselection.java:33) 2) java.lang.arrayindexoutofboundsexception: 12 at kselection.kselection.kselect(kselection.java:33)... FAILURES!!! Tests run: 15, Failures: 13 Fig. 3. Typical output produced by FindBugs and Tpgen, and error output produced by the East tests.
9 4 Related Work Towards Checking the Usefulness of Verification Tools 9 There is a vast amount of related research to ours, but the closest is the Marmoset system [9]. As with our framework it can record programming iterations within Eclipse, but their goal is not to analyse the types of errors found, but rather to analyse students programming behavior and to provide a more stimulating learning environment for novice programmers. One should be able to extend Marmoset to do the same analysis we perform. 5 Conclusions The framework we have shown here and the initial experiments with static analysis tools indicate that the snapshots will be a fruitful source of benchmarks to evaluate testing and verification tools. The current focus has been on testing tools, and given that these tools can be evaluated on fault-detection capability, evaluating these tools are straightforward. Verification tools on the other hand will be much harder to evaluate. One nice feature though is that the programs are relatively small, and thus should not pose any scalability issues for the verification tools. References 1. Coverity Findbugs. findbugs.sourceforge.net 3. Klocwork Lint4j Polyspace N. Ayewah, W. Pugh, J. D. Morgenthaler, J. Penix, and YuQian Zhou. Evaluating static analysis defect warnings on production software. In Proc. 7th Workshop Program Analysis for Software Tools and Engineering, pages 1 8, A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler. A few billion lines of code later: using static analysis to find bugs in the real world. CACM, 53(2):66 75, D. Hovemeyer and W. Pugh. Finding bugs is easy. SIGPLAN Not., 39(12):92 106, J. Spacco, D. Hovemeyer, and W. Pugh. An eclipse-based course project snapshot and submission system. In Proc. 3rd Eclipse Technology Exchange Workshop (etx), A. Tomb, G. Brat, and W. Visser. Variably interprocedural program analysis for runtime error detection. In Proc Intl. Symp. Software Testing and Analysis, pages , W. Visser and J. Geldenhuys. Impendulo: Debugging the Programmer. To appear in Proc. Intl. Conf. Automated Software Engineering, 2010, tool paper.
10 10 Willem Visser and Jaco Geldenhuys 6 Appendix Given two pairs of integers (a, b) and (c, d) we can compare them by first comparing the first component and then the second. For example, (a, b) < (c, d) if and only if a < b or a = b c < d. The k-selection problem is to find the k-th smallest pair in a list of pairs. When k < 0, the task is to find the k-th largest pair. If k = 0, or if the absolute value of k is greater than the length of the list, we shall say that the answer is zero. Given the list (3, 1) (4, 1) (5, 9) (2, 6) (5, 3) (5, 8) then (2, 6) < (3, 1) < (4, 1) < (5, 3) < (5, 8) < (5, 9) and we know that the 1-th smallest pair (2, 6) is in position 4 4-th smallest pair (5, 3) is in position 5 1-th smallest pair (5, 9) is in position 3 and 4-th smallest pair (4, 1) is in position 2. Your task is to write a routine in the KSelection.java class that accepts two parameters: (1) the value of k and (2) the list of integer pairs, stored in a single array. Your routine must return the position of the k-smallest pair as an integer. public int kselect(int k, int V[]) For example, if A is an array containing the values [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8], then kselect( 1, A) should return 4 kselect(-3, A) should return 5 kselect( 7, A) should return 0 You may assume that the list will contain n pairs, where 1 n 10000, none of the pairs are equal, and each integer x in the list will satisfy 0 x 10 6.
Linux Kernel. Security Report
Linux Kernel Security Report September 25 Authors: Andy Chou, Bryan Fulton and Seth Hallem Coverity has combined two years of analysis work carried out in a commercial setting at Coverity with four years
More informationCS 2112 Spring 2014. 0 Instructions. Assignment 3 Data Structures and Web Filtering. 0.1 Grading. 0.2 Partners. 0.3 Restrictions
CS 2112 Spring 2014 Assignment 3 Data Structures and Web Filtering Due: March 4, 2014 11:59 PM Implementing spam blacklists and web filters requires matching candidate domain names and URLs very rapidly
More informationAnalyzing the Decision Criteria of Software Developers Based on Prospect Theory
Analyzing the Decision Criteria of Software Developers Based on Prospect Theory Kanako Kina, Masateru Tsunoda Department of Informatics Kindai University Higashiosaka, Japan tsunoda@info.kindai.ac.jp Hideaki
More informationTechnical paper review. Program visualization and explanation for novice C programmers by Matthew Heinsen Egan and Chris McDonald.
Technical paper review Program visualization and explanation for novice C programmers by Matthew Heinsen Egan and Chris McDonald Garvit Pahal Indian Institute of Technology, Kanpur October 28, 2014 Garvit
More informationNot agree with bug 3, precision actually was. 8,5 not set in the code. Not agree with bug 3, precision actually was
Task 1 Task 2 Task 3 Feedback Presence SUM Matrikkel Rühm [5] [1] [2] [1] [1] [10] Feedback to students A64129 1. rühm 0 0 No submission found A72068 1. rühm 5 1 2 1 1 For Bug 3. Actually the variable
More informationStatic Analysis for Software Verification. Leon Moonen
Static Analysis for Software Verification Leon Moonen Today s topics Software inspection it s relation to testing benefits and drawbacks Static (program) analysis potential benefits limitations and their
More information(Refer Slide Time: 01:52)
Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This
More informationAn Eclipse Plug-In for Visualizing Java Code Dependencies on Relational Databases
An Eclipse Plug-In for Visualizing Java Code Dependencies on Relational Databases Paul L. Bergstein, Priyanka Gariba, Vaibhavi Pisolkar, and Sheetal Subbanwad Dept. of Computer and Information Science,
More informationStatic Analysis of Dynamic Properties - Automatic Program Verification to Prove the Absence of Dynamic Runtime Errors
Static Analysis of Dynamic Properties - Automatic Program Verification to Prove the Absence of Dynamic Runtime Errors Klaus Wissing PolySpace Technologies GmbH Argelsrieder Feld 22 82234 Wessling-Oberpfaffenhofen
More informationDevelopment Testing for Agile Environments
Development Testing for Agile Environments November 2011 The Pressure Is On More than ever before, companies are being asked to do things faster. They need to get products to market faster to remain competitive
More informationMoving from CS 61A Scheme to CS 61B Java
Moving from CS 61A Scheme to CS 61B Java Introduction Java is an object-oriented language. This document describes some of the differences between object-oriented programming in Scheme (which we hope you
More informationA simple algorithm with no simple verication
A simple algorithm with no simple verication Laszlo Csirmaz Central European University Abstract The correctness of a simple sorting algorithm is resented, which algorithm is \evidently wrong" at the rst
More informationWhy? A central concept in Computer Science. Algorithms are ubiquitous.
Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online
More informationNo no-argument constructor. No default constructor found
Every software developer deals with bugs. The really tough bugs aren t detected by the compiler. Nasty bugs manifest themselves only when executed at runtime. Here is a list of the top ten difficult and
More informationIntroduction to Static Analysis for Assurance
Introduction to Static Analysis for Assurance John Rushby Computer Science Laboratory SRI International Menlo Park CA USA John Rushby Static Analysis for Assurance: 1 Overview What is static analysis?
More informationCrowdsourcing suggestions to programming problems for dynamic web development languages
Crowdsourcing suggestions to programming problems for dynamic web development languages Dhawal Mujumdar School of Information University of California, Berkeley 102 South Hall Berkeley, CA 94720 dhawal@ischool.berkeley.edu
More informationBaseline Code Analysis Using McCabe IQ
White Paper Table of Contents What is Baseline Code Analysis?.....2 Importance of Baseline Code Analysis...2 The Objectives of Baseline Code Analysis...4 Best Practices for Baseline Code Analysis...4 Challenges
More informationIntroduction to Automated Testing
Introduction to Automated Testing What is Software testing? Examination of a software unit, several integrated software units or an entire software package by running it. execution based on test cases
More informationSoftware quality is important, but often imperfect in practice. We can use many
focus s o f t w ar e development tools Using Static Analysis to Find Bugs Nathaniel Ayewah and William Pugh, University of Maryland David Hovemeyer, York College of Pennsylvania J. David Morgenthaler and
More informationComprehensive Static Analysis Using Polyspace Products. A Solution to Today s Embedded Software Verification Challenges WHITE PAPER
Comprehensive Static Analysis Using Polyspace Products A Solution to Today s Embedded Software Verification Challenges WHITE PAPER Introduction Verification of embedded software is a difficult task, made
More informationSoftware Quality Exercise 2
Software Quality Exercise 2 Testing and Debugging 1 Information 1.1 Dates Release: 12.03.2012 12.15pm Deadline: 19.03.2012 12.15pm Discussion: 26.03.2012 1.2 Formalities Please submit your solution as
More informationNetwork (Tree) Topology Inference Based on Prüfer Sequence
Network (Tree) Topology Inference Based on Prüfer Sequence C. Vanniarajan and Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology Madras Chennai 600036 vanniarajanc@hcl.in,
More informationSOFTWARE TESTING TRAINING COURSES CONTENTS
SOFTWARE TESTING TRAINING COURSES CONTENTS 1 Unit I Description Objectves Duration Contents Software Testing Fundamentals and Best Practices This training course will give basic understanding on software
More informationSecure Software Programming and Vulnerability Analysis
Secure Software Programming and Vulnerability Analysis Christopher Kruegel chris@auto.tuwien.ac.at http://www.auto.tuwien.ac.at/~chris Testing and Source Code Auditing Secure Software Programming 2 Overview
More informationClient-Side Validation and Verification of PHP Dynamic Websites
American Journal of Engineering Research (AJER) e-issn : 2320-0847 p-issn : 2320-0936 Volume-02, Issue-09, pp-76-80 www.ajer.org Research Paper Open Access Client-Side Validation and Verification of PHP
More informationBenefits of Test Automation for Agile Testing
Benefits of Test Automation for Agile Testing Manu GV 1, Namratha M 2, Pradeep 3 1 Technical Lead-Testing Calsoft Labs, Bangalore, India 2 Assistant Professor, BMSCE, Bangalore, India 3 Software Engineer,
More informationTowards a Framework for Generating Tests to Satisfy Complex Code Coverage in Java Pathfinder
Towards a Framework for Generating Tests to Satisfy Complex Code Coverage in Java Pathfinder Matt Department of Computer Science and Engineering University of Minnesota staats@cs.umn.edu Abstract We present
More informationNotes on Factoring. MA 206 Kurt Bryan
The General Approach Notes on Factoring MA 26 Kurt Bryan Suppose I hand you n, a 2 digit integer and tell you that n is composite, with smallest prime factor around 5 digits. Finding a nontrivial factor
More informationChronon: A modern alternative to Log Files
Chronon: A modern alternative to Log Files A. The 5 fundamental flows of Log Files Log files, Old School, are a relic from the 1970s, however even today in 2012, IT infrastructure monitoring relies on
More informationSoftware Developer Activity as a Source for Identifying Hidden Source Code Dependencies
Software Developer Activity as a Source for Identifying Hidden Source Code Dependencies Martin Konôpka, Mária Bieliková Slovak University of Technology, Faculty of Informatics and Information Technologies,
More informationMeasuring the Effect of Code Complexity on Static Analysis Results
Measuring the Effect of Code Complexity on Static Analysis Results James Walden, Adam Messer, and Alex Kuhl Department of Computer Science Northern Kentucky University Highland Heights, KY 41099 Abstract.
More informationEnterprise Service Bus
We tested: Talend ESB 5.2.1 Enterprise Service Bus Dr. Götz Güttich Talend Enterprise Service Bus 5.2.1 is an open source, modular solution that allows enterprises to integrate existing or new applications
More informationVisualizing Information Flow through C Programs
Visualizing Information Flow through C Programs Joe Hurd, Aaron Tomb and David Burke Galois, Inc. {joe,atomb,davidb}@galois.com Systems Software Verification Workshop 7 October 2010 Joe Hurd, Aaron Tomb
More informationSoftware Testing & Analysis (F22ST3): Static Analysis Techniques 2. Andrew Ireland
Software Testing & Analysis (F22ST3) Static Analysis Techniques Andrew Ireland School of Mathematical and Computer Science Heriot-Watt University Edinburgh Software Testing & Analysis (F22ST3): Static
More informationWelcome to the Force.com Developer Day
Welcome to the Force.com Developer Day Sign up for a Developer Edition account at: http://developer.force.com/join Nicola Lalla nlalla@saleforce.com n_lalla nlalla26 Safe Harbor Safe harbor statement under
More informationFPGA area allocation for parallel C applications
1 FPGA area allocation for parallel C applications Vlad-Mihai Sima, Elena Moscu Panainte, Koen Bertels Computer Engineering Faculty of Electrical Engineering, Mathematics and Computer Science Delft University
More informationUsing Library Dependencies for Clustering
Using Library Dependencies for Clustering Jochen Quante Software Engineering Group, FB03 Informatik, Universität Bremen quante@informatik.uni-bremen.de Abstract: Software clustering is an established approach
More informationWhat Is Specific in Load Testing?
What Is Specific in Load Testing? Testing of multi-user applications under realistic and stress loads is really the only way to ensure appropriate performance and reliability in production. Load testing
More informationIntroduction to Synoptic
Introduction to Synoptic 1 Introduction Synoptic is a tool that summarizes log files. More exactly, Synoptic takes a set of log files, and some rules that tell it how to interpret lines in those logs,
More informationA Test Suite for Basic CWE Effectiveness. Paul E. Black. paul.black@nist.gov. http://samate.nist.gov/
A Test Suite for Basic CWE Effectiveness Paul E. Black paul.black@nist.gov http://samate.nist.gov/ Static Analysis Tool Exposition (SATE V) News l We choose test cases by end of May l Tool output uploaded
More informationOptimization and analysis of large scale data sorting algorithm based on Hadoop
Optimization and analysis of large scale sorting algorithm based on Hadoop Zhuo Wang, Longlong Tian, Dianjie Guo, Xiaoming Jiang Institute of Information Engineering, Chinese Academy of Sciences {wangzhuo,
More informationThe programming language C. sws1 1
The programming language C sws1 1 The programming language C invented by Dennis Ritchie in early 1970s who used it to write the first Hello World program C was used to write UNIX Standardised as K&C (Kernighan
More informationWriting Self-testing Java Classes with SelfTest
Writing Self-testing Java Classes with SelfTest Yoonsik Cheon TR #14-31 April 2014 Keywords: annotation; annotation processor; test case; unit test; Java; JUnit; SelfTest. 1998 CR Categories: D.2.3 [Software
More informationDetecting Critical Defects on the Developer s Desktop
Detecting Critical Defects on the Developer s Desktop Seth Hallem CEO Coverity, Inc. Copyright Coverity, Inc. 2006. All Rights Reserved. This publication, in whole or in part, may not be reproduced, stored
More informationWeb Caching With Dynamic Content Abstract When caching is a good idea
Web Caching With Dynamic Content (only first 5 pages included for abstract submission) George Copeland - copeland@austin.ibm.com - (512) 838-0267 Matt McClain - mmcclain@austin.ibm.com - (512) 838-3675
More informationProgramming and Software Development CTAG Alignments
Programming and Software Development CTAG Alignments This document contains information about four Career-Technical Articulation Numbers (CTANs) for Programming and Software Development Career-Technical
More informationJOURNAL OF OBJECT TECHNOLOGY
JOURNAL OF OBJECT TECHNOLOGY Online at http://www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2006 Vol. 5, No. 6, July - August 2006 On Assuring Software Quality and Curbing Software
More informationEUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH CERN ACCELERATORS AND TECHNOLOGY SECTOR
EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH CERN ACCELERATORS AND TECHNOLOGY SECTOR CERN-ATS-2011-213 THE SOFTWARE IMPROVEMENT PROCESS - TOOLS AND RULES TO ENCOURAGE QUALITY K. Sigerud, V. Baggiolini, CERN,
More informationAgile Business Suite: a 4GL environment for.net developers DEVELOPMENT, MAINTENANCE AND DEPLOYMENT OF LARGE, COMPLEX BACK-OFFICE APPLICATIONS
Agile Business Suite: a 4GL environment for.net developers DEVELOPMENT, MAINTENANCE AND DEPLOYMENT OF LARGE, COMPLEX BACK-OFFICE APPLICATIONS In order to ease the burden of application lifecycle management,
More informationAutomated Faultinjection Series - Risk Management and Implementation
HEALERS: A Toolkit for Enhancing the Robustness and Security of Existing Applications Christof Fetzer, Zhen Xiao AT&T Labs Research 180 Park Avenue Florham Park, N.J. 07932 christof, xiao @research.att.com
More informationConcepts and terminology in the Simula Programming Language
Concepts and terminology in the Simula Programming Language An introduction for new readers of Simula literature Stein Krogdahl Department of Informatics University of Oslo, Norway April 2010 Introduction
More informationWESTMORELAND COUNTY PUBLIC SCHOOLS 2011 2012 Integrated Instructional Pacing Guide and Checklist Computer Math
Textbook Correlation WESTMORELAND COUNTY PUBLIC SCHOOLS 2011 2012 Integrated Instructional Pacing Guide and Checklist Computer Math Following Directions Unit FIRST QUARTER AND SECOND QUARTER Logic Unit
More informationVisCG: Creating an Eclipse Call Graph Visualization Plug-in. Kenta Hasui, Undergraduate Student at Vassar College Class of 2015
VisCG: Creating an Eclipse Call Graph Visualization Plug-in Kenta Hasui, Undergraduate Student at Vassar College Class of 2015 Abstract Call graphs are a useful tool for understanding software; however,
More informationTask: ASC Ascending Paths
Task: ASC Ascending Paths You are visiting the Royal Botanical Gardens. In the gardens there are n intersections connected by m roads. The intersections are numbered from 1 to n. Each road connects two
More informationSo today we shall continue our discussion on the search engines and web crawlers. (Refer Slide Time: 01:02)
Internet Technology Prof. Indranil Sengupta Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture No #39 Search Engines and Web Crawler :: Part 2 So today we
More informationChecking Access to Protected Members in the Java Virtual Machine
Checking Access to Protected Members in the Java Virtual Machine Alessandro Coglio Kestrel Institute 3260 Hillview Avenue, Palo Alto, CA 94304, USA Ph. +1-650-493-6871 Fax +1-650-424-1807 http://www.kestrel.edu/
More informationDecision Logic: if, if else, switch, Boolean conditions and variables
CS 1044 roject 3 Fall 2009 Decision Logic: if, if else, switch, Boolean conditions and variables This programming assignment uses many of the ideas presented in sections 3 through 5 of the Dale/Weems text
More informationChapter 1: Key Concepts of Programming and Software Engineering
Chapter 1: Key Concepts of Programming and Software Engineering Software Engineering Coding without a solution design increases debugging time - known fact! A team of programmers for a large software development
More informationTaking Subversion to a Higher Level. Branching/Merging Support. Component Management Support. And More
Taking Subversion to a Higher Level Branching/Merging Support Component Management Support And More About Impact CM Impact CM is a Service AddOn that facilitates software configuration management (CM)
More informationOntology Model-based Static Analysis on Java Programs
Ontology Model-based Static Analysis on Java Programs Lian Yu 1, Jun Zhou, Yue Yi, Ping Li, Qianxiang Wang School of Software and Microelectronics, Peking University, Beijing, 102600, PRC Abstract 1 Typical
More informationAppendixA1A1. Java Language Coding Guidelines. A1.1 Introduction
AppendixA1A1 Java Language Coding Guidelines A1.1 Introduction This coding style guide is a simplified version of one that has been used with good success both in industrial practice and for college courses.
More informationA Tool to Support Knowledge Based Software Maintenance: The Software Service Bay
A Tool to Support Knowledge Based Software Maintenance: The Software Service Bay Jonathan I. Maletic Robert G. Reynolds Computer Science Department Wayne State University 431 State Hall Detroit, MI 48202
More informationMapReduce. MapReduce and SQL Injections. CS 3200 Final Lecture. Introduction. MapReduce. Programming Model. Example
MapReduce MapReduce and SQL Injections CS 3200 Final Lecture Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. OSDI'04: Sixth Symposium on Operating System Design
More informationReal-Time Systems Prof. Dr. Rajib Mall Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Real-Time Systems Prof. Dr. Rajib Mall Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture No. # 26 Real - Time POSIX. (Contd.) Ok Good morning, so let us get
More informationApp Distribution Guide
App Distribution Guide Contents About App Distribution 10 At a Glance 11 Enroll in an Apple Developer Program to Distribute Your App 11 Generate Certificates and Register Your Devices 11 Add Store Capabilities
More informationDevelopment Environment and Tools for Java. Brian Hughes IBM
Development Environment and Tools for Java Brian Hughes IBM 1 Acknowledgements and Disclaimers Availability. References in this presentation to IBM products, programs, or services do not imply that they
More informationSoftware Engineering for LabVIEW Applications. Elijah Kerry LabVIEW Product Manager
Software Engineering for LabVIEW Applications Elijah Kerry LabVIEW Product Manager 1 Ensuring Software Quality and Reliability Goals 1. Deliver a working product 2. Prove it works right 3. Mitigate risk
More informationEVALUATION. WA1844 WebSphere Process Server 7.0 Programming Using WebSphere Integration COPY. Developer
WA1844 WebSphere Process Server 7.0 Programming Using WebSphere Integration Developer Web Age Solutions Inc. USA: 1-877-517-6540 Canada: 1-866-206-4644 Web: http://www.webagesolutions.com Chapter 6 - Introduction
More informationEvaluation of AgitarOne
Carnegie Mellon University, School of Computer Science Master of Software Engineering Evaluation of AgitarOne Analysis of Software Artifacts Final Project Report April 24, 2007 Edited for public release
More informationCost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:
CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm
More informationCHAPTER 5 INTELLIGENT TECHNIQUES TO PREVENT SQL INJECTION ATTACKS
66 CHAPTER 5 INTELLIGENT TECHNIQUES TO PREVENT SQL INJECTION ATTACKS 5.1 INTRODUCTION In this research work, two new techniques have been proposed for addressing the problem of SQL injection attacks, one
More informationVariable Base Interface
Chapter 6 Variable Base Interface 6.1 Introduction Finite element codes has been changed a lot during the evolution of the Finite Element Method, In its early times, finite element applications were developed
More informationSoftware Reliability Estimation Based on Static Error Detection
7 th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2011 October 31 November 3, Moscow Software Reliability Estimation Based on Static M. Moiseev, M. Glukhikh, A. Karpenko,
More informationTOOL EVALUATION REPORT: FORTIFY
TOOL EVALUATION REPORT: FORTIFY Derek D Souza, Yoon Phil Kim, Tim Kral, Tejas Ranade, Somesh Sasalatti ABOUT THE TOOL Background The tool that we have evaluated is the Fortify Source Code Analyzer (Fortify
More informationContinuous Integration with Jenkins. Coaching of Programming Teams (EDA270) J. Hembrink and P-G. Stenberg [dt08jh8 dt08ps5]@student.lth.
1 Continuous Integration with Jenkins Coaching of Programming Teams (EDA270) J. Hembrink and P-G. Stenberg [dt08jh8 dt08ps5]@student.lth.se Faculty of Engineering, Lund Univeristy (LTH) March 5, 2013 Abstract
More informationComp 411 Principles of Programming Languages Lecture 34 Semantics of OO Languages. Corky Cartwright Swarat Chaudhuri November 30, 20111
Comp 411 Principles of Programming Languages Lecture 34 Semantics of OO Languages Corky Cartwright Swarat Chaudhuri November 30, 20111 Overview I In OO languages, data values (except for designated non-oo
More informationContent. Development Tools 2(63)
Development Tools Content Project management and build, Maven Version control, Git Code coverage, JaCoCo Profiling, NetBeans Static Analyzer, NetBeans Continuous integration, Hudson Development Tools 2(63)
More informationUML-based Test Generation and Execution
UML-based Test Generation and Execution Jean Hartmann, Marlon Vieira, Herb Foster, Axel Ruder Siemens Corporate Research, Inc. 755 College Road East Princeton NJ 08540, USA jeanhartmann@siemens.com ABSTRACT
More informationBusiness Process Discovery
Sandeep Jadhav Introduction Well defined, organized, implemented, and managed Business Processes are very critical to the success of any organization that wants to operate efficiently. Business Process
More informationUser Stories Applied
User Stories Applied for Agile Software Development Mike Cohn Boston San Francisco New York Toronto Montreal London Munich Paris Madrid Capetown Sydney Tokyo Singapore Mexico City Chapter 2 Writing Stories
More informationIn recent years there has been growing concern with the financial and tax compliance risks associated with
Development of Financial Products Business Rules Using Business Intelligence Technology David Macias and Jennifer Li, IRS: Large Business and International Division In recent years there has been growing
More informationModule 10. Coding and Testing. Version 2 CSE IIT, Kharagpur
Module 10 Coding and Testing Lesson 26 Debugging, Integration and System Testing Specific Instructional Objectives At the end of this lesson the student would be able to: Explain why debugging is needed.
More informationDesign by Contract beyond class modelling
Design by Contract beyond class modelling Introduction Design by Contract (DbC) or Programming by Contract is an approach to designing software. It says that designers should define precise and verifiable
More informationGraphs without proper subgraphs of minimum degree 3 and short cycles
Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract
More informationChapter 8 Software Testing
Chapter 8 Software Testing Summary 1 Topics covered Development testing Test-driven development Release testing User testing 2 Program testing Testing is intended to show that a program does what it is
More informationAPPROACHES TO SOFTWARE TESTING PROGRAM VERIFICATION AND VALIDATION
1 APPROACHES TO SOFTWARE TESTING PROGRAM VERIFICATION AND VALIDATION Validation: Are we building the right product? Does program meet expectations of user? Verification: Are we building the product right?
More informationGetting Started with the Internet Communications Engine
Getting Started with the Internet Communications Engine David Vriezen April 7, 2014 Contents 1 Introduction 2 2 About Ice 2 2.1 Proxies................................. 2 3 Setting Up ICE 2 4 Slices 2
More information[Refer Slide Time: 05:10]
Principles of Programming Languages Prof: S. Arun Kumar Department of Computer Science and Engineering Indian Institute of Technology Delhi Lecture no 7 Lecture Title: Syntactic Classes Welcome to lecture
More informationKnow or Go Practical Quest for Reliable Software
Know or Go Practical Quest for Reliable Software Dr.-Ing. Jörg Barrho Dr.-Ing. Ulrich Wünsche AVACS Project meeting 25.09.2014 2014 Rolls-Royce Power Systems AG The information in this document is the
More informationImproved Software Testing Using McCabe IQ Coverage Analysis
White Paper Table of Contents Introduction...1 What is Coverage Analysis?...2 The McCabe IQ Approach to Coverage Analysis...3 The Importance of Coverage Analysis...4 Where Coverage Analysis Fits into your
More informationLogistics. Software Testing. Logistics. Logistics. Plan for this week. Before we begin. Project. Final exam. Questions?
Logistics Project Part 3 (block) due Sunday, Oct 30 Feedback by Monday Logistics Project Part 4 (clock variant) due Sunday, Nov 13 th Individual submission Recommended: Submit by Nov 6 th Scoring Functionality
More informationGild: An Integrated Learning and Development plug-in for Eclipse
Gild: An Integrated Learning and Development plug-in for Eclipse Teaching students how to program can be a challenging task. Unfortunately, there is a lack of tools that provide pedagogical support for
More informationA QUICK OVERVIEW OF THE OMNeT++ IDE
Introduction A QUICK OVERVIEW OF THE OMNeT++ IDE The OMNeT++ 4.x Integrated Development Environment is based on the Eclipse platform, and extends it with new editors, views, wizards, and additional functionality.
More informationComparing Four Static Analysis Tools for Java Concurrency Bugs
Comparing Four Static Analysis Tools for Java Concurrency Bugs Md. Abdullah Al Mamun, Aklima Khanam, Håkan Grahn, and Robert Feldt School of Computing, Blekinge Institute of Technology SE-371 79 Karlskrona,
More informationIntroduction to Software Engineering. 8. Software Quality
Introduction to Software Engineering 8. Software Quality Roadmap > What is quality? > Quality Attributes > Quality Assurance: Planning and Reviewing > Quality System and Standards 2 Sources > Software
More informationSoftware testing. Objectives
Software testing cmsc435-1 Objectives To discuss the distinctions between validation testing and defect testing To describe the principles of system and component testing To describe strategies for generating
More informationEvaluating a new programming language
In G. Kadoda (Ed). Proc. PPIG 13 Pages 275-289 Evaluating a new programming language Steven Clarke Microsoft Corporation 1 Microsoft Way Redmond, WA 98052 USA +1 425 705 5978 stevencl@microsoft.com Keywords:
More informationOracle BI EE Implementation on Netezza. Prepared by SureShot Strategies, Inc.
Oracle BI EE Implementation on Netezza Prepared by SureShot Strategies, Inc. The goal of this paper is to give an insight to Netezza architecture and implementation experience to strategize Oracle BI EE
More informationDocumentum Developer Program
Program Enabling Logging in DFC Applications Using the com.documentum.fc.common.dflogger class April 2003 Program 1/5 The Documentum DFC class, DfLogger is available with DFC 5.1 or higher and can only
More informationPROCESS OF MOVING FROM WATERFALL TO AGILE PROJECT MANAGEMENT MODEL
PROCESS OF MOVING FROM WATERFALL TO AGILE PROJECT MANAGEMENT MODEL Sanja Vukićević 1, Dražen Drašković 2 1 Faculty of Organizational Sciences, University of Belgrade, vukicevicsanja@yahoo.com 2 Faculty
More information