A Formal Framework for Specifying and Analyzing Logs as Electronic Evidence Eduardo Mazza 1, Marie-Laure Potet 1, Daniel Le Métayer 2 LISE Project Funded by the Agence Nationale de la Recherce (ANR-07-SESU-00) (1) Verimag, Grenoble, France (2) INRIA, Grenoble Rhône-Alpes, France
Motivation Challenge: to precise legal liability for software Log as digital evidence More and more necessary PROBLEM Actual solutions that define liability are not focused in logs as digital evidence Works in log analysis show little concern in liability Proposal An integrated framework for precisely defining liability and log content as electronic evidence Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 2 / 26
Outline Introduction Logs & Claims Log Functions Log Analyzer Conclusion Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 3 / 26
Introduction LISE Project Contract based environment Legal aspects studied in previous works [ICSE 2010] Context: FAULTS CLAIMS LIABILITY Two or more agents signing a legal contract to precise liability for potential claims Contract agreement between the agents Requirements Description of application Claims taken into account covered by the contract Evidence agreement Log content and architecture Log Analyzer Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 4 / 26
LISE Approach Two phases Contractual requirements and evidence agreement Analysis - when claims appear Contractual Phase Analysis Phase Generic model Use of the B-method focus on data and behaviour Log Analyzer (attachment in contract) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 5 / 26
Assumptions & Key Concepts Distributed system distributed logs Information spread along multiple log files Communication between agents by message exchange Well adapted for B2B applications Logs are grouped by agents A single log file may contain the information of many agents Incremental Analysis would be an advantage Claims may be analyzed in a partial setting of the distributed system Not always possible to immediately obtain all logs Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 6 / 26
Outline Introduction Logs & Claims Log Functions Log Analyzer Conclusion Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 7 / 26
Logs How to represent logs? Generic model supporting distributed logs Hypothesis: preserved causality, no loss, no duplication System specification AGENT ACTION Interface : ACTION AGENT Logs and log distributions Event: (Send Rec, AGENT, AGENT, ACTION) Log file: F(AGENT ) iseq(events) Distribution: F(F(AGENT )) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 8 / 26
Example of Logs System Specification Possible log distributions {Client}, {Agency}, {Bank}, {Hotel} {Client, Agency}, {Bank}, {Hotel} Possible logs: ({Client, Agency}, [Request Send, Request Rec,... ]) ({Hotel}, [Book Rec, Cancel Rec,... ]) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 9 / 26
Claims How to represent the claims? Logs that are concerned by the claim (agents) A precise characterization when the claim is accepted (log property) A claim consists of: A plaintiff (the complaining agent) A defendant A log property If the property holds, then the agent defendant is responsible. Claim: (AGENT AGENT PROP) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 10 / 26
Properties Property: F(AGENT ) (LOG FILE BOOL) Distributed setting property for partial distribution 1 Agents concerned with this property Information needed to verify a property 2 Partial function (w.r.t. agents) that maps a log file to TRUE or FALSE IMPORTANT: agents of the property = agents of the log evaluated Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 11 / 26
Example of Claims (claim NoRoom) Client requests a reservation and is charged but there is no reservation: Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 12 / 26
Example of Claims (claim NoRoom) Client requests a reservation and is charged but there is no reservation: 1 NoRoom CLAIM NoRoom = (Client, Agency, prop NoRoom ) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 12 / 26
Example of Claims (claim NoRoom) Client requests a reservation and is charged but there is no reservation: 1 NoRoom CLAIM NoRoom = (Client, Agency, prop NoRoom ) 2 agents(prop NoRoom ) = {Client, Agency} Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 12 / 26
Example of Claims (claim NoRoom) Client requests a reservation and is charged but there is no reservation: 1 NoRoom CLAIM NoRoom = (Client, Agency, prop NoRoom ) 2 agents(prop NoRoom ) = {Client, Agency} 3 val(prop NoRoom ) = λ log.(agents(log) = {Client, Agency} Request Send events(log) Debit Send events(log) Book Send events(log) pos(request Send, log) < pos(debit Send, log)) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 12 / 26
Example of Claims (claim NoRoom) Client requests a reservation and is charged but there is no reservation: 1 NoRoom CLAIM NoRoom = (Client, Agency, prop NoRoom ) 2 agents(prop NoRoom ) = {Client, Agency} 3 val(prop NoRoom ) = λ log.(agents(log) = {Client, Agency} Request Send events(log) Debit Send events(log) Book Send events(log) pos(request Send, log) < pos(debit Send, log)) Client Agency Bank Hotel Request Justify Debit Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 12 / 26
Outline Introduction Logs & Claims Log Functions Log Analyzer Conclusion Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 13 / 26
Log Functions Motivation: Manipulate distributed logs w.r.t. concerned agents Log functions: extract - obtain events in a log concerning a given group of agents merge - provide the set of logs that respect the causal order of events Several possible scenarios Property: extract ags [merge[logs]] merge[extract ags [logs]] Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 14 / 26
Example of merge Client Request Agency Cancel log Client log Agency merge[log Client, log Agency ] = {log 1, log 2 } log 1 = ({Client, Agency}, [Request Send, Request Rec, Cancel Send ]) log 2 = ({Client, Agency}, [Request Send, Cancel Send, Request Rec ]) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 15 / 26
Outline Introduction Logs & Claims Log Functions Log Analyzer Conclusion Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 16 / 26
Analyzing a claim How to establish if a claims should be accepted or rejected? Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 17 / 26
Analyzing a claim How to establish if a claims should be accepted or rejected? 1 For a given claim (Plain, Def, Prop) select certain logs that have the information required by Prop (agents(prop) agents(logs)) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 17 / 26
Analyzing a claim How to establish if a claims should be accepted or rejected? 1 For a given claim (Plain, Def, Prop) select certain logs that have the information required by Prop (agents(prop) agents(logs)) 2 Merge the selected log files Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 17 / 26
Analyzing a claim How to establish if a claims should be accepted or rejected? 1 For a given claim (Plain, Def, Prop) select certain logs that have the information required by Prop (agents(prop) agents(logs)) 2 Merge the selected log files 3 Extract the information required by Prop (agents(prop)) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 17 / 26
Analyzing a claim How to establish if a claims should be accepted or rejected? 1 For a given claim (Plain, Def, Prop) select certain logs that have the information required by Prop (agents(prop) agents(logs)) 2 Merge the selected log files 3 Extract the information required by Prop (agents(prop)) 4 Compute the possible set of scenarios where Prop holds. Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 17 / 26
Analyzing a claim How to establish if a claims should be accepted or rejected? 1 For a given claim (Plain, Def, Prop) select certain logs that have the information required by Prop (agents(prop) agents(logs)) 2 Merge the selected log files 3 Extract the information required by Prop (agents(prop)) 4 Compute the possible set of scenarios where Prop holds. 5 Interpretation of the results by the judge Two results: Set of all scenarios Set of scenarios where property hold (I) Conclude the investigation accept or reject a claim (II) More data needed Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 17 / 26
Log Analyzer Log Analyzer: tool that computes the two results to be interpreted INPUT: logs: set of logs prop: property OUTPUT: scen: all possible scenarios ok: scenarios where the property holds scen, ok Analysis(logs, prop) PRE agents(prop) agents(logs) THEN scen := extract agents(prop) [merge[logs]]; ok := scen val(prop) 1 [{TRUE}] END Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 18 / 26
Interpreting the results Depending of the values for scen, ok: Inconclusive results results are not enough to provide the intuition for accepting or rejecting a claim A fine study may be necessary. (incremental analysis) Some definitive conclusive results situations: if scen = ok then claim is accepted if ok = then claim is rejected Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 19 / 26
Example of analysis - claim NoRoom (paper Example 8) Client Agency Bank Hotel Request Justify Debit Agency wants to verify if the claim is valid without using Bank s log Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 20 / 26
Example of analysis - claim NoRoom (paper Example 8) Client Agency Bank Hotel Request Justify Debit Agency wants to verify if the claim is valid without using Bank s log 3 scenarios: Request Send, Request Rec, Debit Send, Justify Rec Request Send, Request Rec, Justify Rec, Debit Send Request Send, Justify Rec, Request Rec, Debit Send scen = ok claim accepted Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 20 / 26
Incremental analysis Inconclusive results may demand more logs to be analyzed Previous results may help in the computation of the new analysis scen, ok Analysis(logs logs, prop) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 21 / 26
Incremental analysis Inconclusive results may demand more logs to be analyzed Previous results may help in the computation of the new analysis scen, ok Analysis(logs logs, prop) Incremental calculus Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 21 / 26
Incremental analysis Inconclusive results may demand more logs to be analyzed Previous results may help in the computation of the new analysis scen, ok Analysis(logs logs, prop) Incremental calculus 1 Compute scen, ok Analysis(logs, prop) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 21 / 26
Incremental analysis Inconclusive results may demand more logs to be analyzed Previous results may help in the computation of the new analysis scen, ok Analysis(logs logs, prop) Incremental calculus 1 Compute scen, ok Analysis(logs, prop) 2 iscen, iok IncrAnalysis(logs, prop, scen, ok) iscen := extract[merge[logs scen]] iok := extract[merge[logs ok]] ADVANTAGE: No need to verify the property again ok iok ok scen iscen scen Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 21 / 26
Example of incremental Analysis (paper Example 9) (claim LateCancel) Client complain that was charged for a reservation that had been canceled prop LateCancel : agents(prop LateCancel ) = {Client, Agency} Debit Send events(log) Cancel Send events(log) pos(cancel Send, log) < pos(debit Send, log) Client Agency Bank Hotel Request Confirm Justify Debit Cancel Book CancelDebit Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 22 / 26
First analysis Client Agency Bank Hotel Request Confirm Justify Debit Cancel Book CancelDebit scen with 20 scenarios ok with 10 scenarios Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 23 / 26
Second analysis (incremental) Client Agency Bank Hotel Request Justify Confirm Debit Cancel Book CancelDebit scen with 3 scenarios ok = claim rejected (without property verification!) Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 24 / 26
Outline Introduction Logs & Claims Log Functions Log Analyzer Conclusion Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 25 / 26
Conclusion Contributions: General framework to precisely decribe claims in terms of logs Specification of a Log Analyzer tool Study of incremental aspects over the acceptability of claims Future works: Parametrized claims and properties Integration with previous works Analysis of log architecture [SEFM 2010] Help adding logs for incremental analysis Formal definition of liability When should a claim be accepted Claim with multiples responsible agents Mazza, Potet, Le Métayer (LISE Project) SBMF 2010 26 / 26