About me. I m Eiji Adachi Barbosa

About me I m Eiji Adachi Barbosa PhD student MSc. in Informatics from PUC-Rio I work under the supervision of prof. Alessandro Garcia since 2010 My research topic: Exception Handling I took this course in 2011 and again in 2013 Eiji Adachi M. Barbosa 1

Categorizing Faults in Exception Handling: A Study of Open Source Projects Eiji Adachi Barbosa Alessandro Garcia Simone D. J. Barbosa LES DI PUC-Rio Brasil SBES-2104 OPUS Research Group

Exceptions Exceptions Unexpected conditions or events that occur at runtime and prevent programs to continue their normal execution flow Press any key to continue _ Eiji Adachi M. Barbosa 3

Exception Handling Mechanisms Intended to improve software robustness Ease the design and implementation of the exception handling in software systems Provided in most mainstream programming languages Try Catch Throw Eiji Adachi M. Barbosa 4

Exception Handling Mechanisms in Programming Languages public void foo(...) { if( /* exceptional condition */ ) throw new XPTO_Exception( ERROR message ); try { // Business catch (Exception e) { // Recovery actions Eiji Adachi M. Barbosa 5

Developers viewpoint Shah et al. (2010) conducted a series of interviews with developers: Novice: Tend to ignore exceptions when developing systems Use exceptions primarily for debugging the code Dislike the forced handling of exceptions imposed by Java compiler Experts: Do not distinguish between normal and exceptional functionality Have trained themselves to think about both usual and exceptional behavior simultaneously Use exception-handling to convey important information to the user Also do not appreciate that exception-handling in Java is imposed Eiji Adachi M. Barbosa 6

Developers viewpoint Ebert and Castor (2014) conducted a survey with developers: Only 27% of respondents affirmed that they pay attention to exception handling Only 21% of respondents affirmed that their organizations have specific exception handling policies 40% of respondents consider the quality of exception handling good or very good and 14% of respondents consider the quality of exception handling bad or very bad Eiji Adachi M. Barbosa 7

Problems with Exception Handling The lack of proper handling actions is very common in real software systems 40%-72% of handling actions are only-logging or empty handlers Exception handling code is more error prone and concentrates more defects than normal code Uncaught exceptions Developers introduce faults in exception handling code even when they try to improve it Eiji Adachi M. Barbosa 8

What are the causes of faults (bugs) related to exception handling? i.e., faults related to the definition, throwing, propagation, handling and documentation of exceptions, as well as faults related to termination actions in case of exceptions. Eiji Adachi M. Barbosa 9

Study design Goal: Gather deeper knowledge about the causes of faults related to exception handling reported in software systems To achieve our research goal, we performed a longitudinal evaluation in the context of two large software projects Eiji Adachi M. Barbosa 10

Target systems Open-source projects Publicly available bug reports and source code Possibility to map bug reports to corresponding fixing patches First attempt: search bug reports and then find the corresponding fixing patches Failed because many bug reports did not have the corresponding patch Other approach: Search the revisions that fixed faults and then find the corresponding bug report Bug-Report-Id-patterns on revision comments Selected candidates: Tomcat and Hadoop Eiji Adachi M. Barbosa 11

Data collection Step 1 Gather complete history of revisions Target system source code repository Eiji Adachi M. Barbosa 12

Data collection Step 1 Gather complete history of revisions Target system source code repository Complete history of revisions Eiji Adachi M. Barbosa 13

Data collection Step 2 Identify revisions potentially related to exceptional faults Complete history of revisions Fix https://issues.apache.org/bugzilla/s how_bug.cgi?id=52591 Skip attributes where getters throw UnsupportedOperationException Revision comment Eiji Adachi M. Barbosa 14

Data collection Step 2 Identify revisions potentially related to exceptional faults Complete history of revisions Comment contains Report_Id && keyword Exception? Fix https://issues.apache.org/bugzilla/s how_bug.cgi?id=52591 Skip attributes where getters throw UnsupportedOperationException Revision comment Eiji Adachi M. Barbosa 15

Data collection Step 2 Identify revisions potentially related to exceptional faults Complete history of revisions Comment contains Report_Id \ && keyword Exception? Revisions potentially related to exceptional faults Fix https://issues.apache.org/bugzilla/s how_bug.cgi?id=52591 Skip attributes where getters throw UnsupportedOperationException Revision comment Eiji Adachi M. Barbosa 16

Data collection Step 3 Manual triage Revision comment Revisions potentially related to exceptional faults Bug report Source code diff Eiji Adachi M. Barbosa 17

Data collection Step 3 Manual triage Revision comment Revisions potentially related to exceptional faults Bug report Revisions related to exceptional faults Source code diff Eiji Adachi M. Barbosa 18

Raw data Eiji Adachi M. Barbosa 19

Analysis method Step 1 Diff description V2 V1 Eiji Adachi M. Barbosa 20

Analysis method Step 1 Diff description V2 V1 Changed the argument of a catch from InvalidProtocolBufferExce ption to Throwable. Also changed parameters of exception instantiation within catch block. Diff description Eiji Adachi M. Barbosa 21

Analysis method Step 2 Extraction of text fragments Revision comment Bug report Eiji Adachi M. Barbosa 22

Analysis method Step 2 Extraction of text fragments Revision comment Bug report Fragments explaining the cause or the correction Eiji Adachi M. Barbosa 23

Analysis method Step 2 Extraction of text fragments So, two conclusions: 1. The failure occurs above response.encodeurl("j_security_check")" call. 2. I suspect that _jspx_page_context is null. In that case the Throwable in the catch block is silently swallowed. Example of fragment extracted from bug report Eiji Adachi M. Barbosa 24

Analysis method Step 3 Explaining the cause Diff description What caused the observed fault? Fragments explaining the cause or the correction Eiji Adachi M. Barbosa 25

Analysis method Step 3 Explaining the cause Diff description What caused the \ observed fault? Cause of observed fault Fragments explaining the cause or the correction Eiji Adachi M. Barbosa 26

Categories of exceptional faults Eiji Adachi M. Barbosa 27

Categories of exceptional faults Informa on swallowed sub-categories Uninforma ve or wrong error message 7 8 Swallowed excep on 2 1 Suppressed excep on 1 2 Missing log 2 Destruc ve wrapping 2 Uninforma ve generic type thrown 1 0 2 4 6 8 10 12 14 16 Eiji Tomcat Adachi M. Barbosa Hadoop 28

Category - Suppressed exception public void copybytes() throws IOException{ try{ //manipulate bytes finally { closestream( out ); Eiji Adachi M. Barbosa 29

Category - Suppressed exception public void copybytes() throws IOException{ try{ //manipulate bytes finally { closestream( out ); Original exception does not flow out of the method AND Exception raised on the finally block does not contain information about the original exception Eiji Adachi M. Barbosa 35

Category Destructive remapping public void foo(){ try{ // do something catch ( IOException e ){ String s = e.getmessage(); throw new BusinessException( s ); Eiji Adachi M. Barbosa 36

Category Destructive remapping public void foo(){ try{ // do something Extracted from bug report: (The error is) the fact that the catch block that logs the exception is swallowing the original exception and (its) stack trace. catch ( IOException e ){ String s = e.getmessage(); throw new BusinessException( s ); Eiji Adachi M. Barbosa 37

Other categories Premature termination Catch block terminates without retrying I now think the best solution is catch all IOExceptions and then retry once. Overly protective try-block Try-block is very long and inadvertently guards the occurrence of many different exceptions. I believe the scope for which the FileNotFoundException' block applies is too great. Excessive Throwing Condition Exception is thrown by a condition that is not actually an exceptional condition There is no need to throw an exception and then immediately catch it and log it. The 'else throw' can be removed. Eiji Adachi M. Barbosa 38

Final remarks Limitations of this study: Sample Small size Both systems implemented in Java Data source Bugs are also reported on mail lists Analysis method May have missed an unknown number of faults related to exception handling Eiji Adachi M. Barbosa 39

Final remarks RQ: What are the causes of faults related to exception handling? 10 different macro-categories of faults related to exception handling 18 different categories, if sub-categories are considered Most faults were caused by insufficient information provided with the exception New research questions: What categories are more severe? harder to correct? How they are corrected? Eiji Adachi M. Barbosa 40

Final remarks Future: Replicate this study with other systems, programming languages, etc We invite you to replicate our study and contribute with new fault categories Study how faults are corrected Explore how these categories can be used to: Train developers Improve current static analysis tools to detect these categories of faults Eiji Adachi M. Barbosa 41

Categorizing Faults in Exception Handling: A Study of Open Source Projects Eiji Adachi Barbosa Alessandro Garcia Simone D. J. Barbosa LES DI PUC-Rio Brasil SBES-2104 Eiji Adachi M. Barbosa 42 OPUS Research Group