Role of Expert Judgement in Analysing Large Complex Reliability Data Sets

Transcription

1 Role of Expert Judgement in Analysing Large Complex Reliability Data Sets 9th June 2014

2 1 Background to the problem 2 Empirical analysis of observational condition test data 3 Engineering judgement reasoning of the problem example 4 Building a predictive model for inference about failure 5 Summary and next steps

3 Background to the problem Motivating engineering problem Predict the time to failure of vehicles given information about engine condition Our problem of interest today Mining observational condition monitoring data to understand patterns and correlation structures Compare empirical insights with engineering judgement of causal relationships Use relevant to inform predictive model development Condition monitoring observational data Relate to fleet of like vehicles with irregular usage profile Engine oil samples taken at regular intervals

4 Background to the problem

5 Background to the problem Data initially in 3 spreadsheets by vehicle type. Dimensions are (437 41), (286 41) and (70 41). The covariates are the same in all sheets but there are many missing values and entire missing covariates in some sheets. The data were messy and required initial cleansing. Specific variables: Distances travelled since oil change for each vehicle at each observation: km and motor hours. Number of indicator particles in oil: cutting, sliding, fatigue, total, large. Content of specific elements in oil: chromium, lead, copper, nickel, etc. Other measures: kinematic viscosity, flash point, total alkilinity number. The most complete data set is analysed in this talk.

6 Empirical analysis of observational condition test data km.tot km.oil mh.tot mh.oil viscosity flash alkalinity fuel water glycol oxid nitrate sulphate antioxid soot ferrum chromium lead copper tin aluminium nickel silver silicon borium natrium magnes calcium barium phosphor zinc molybdenum titanium vanadium part.tot cut.part slid.part fat.part large.part

7 Empirical analysis of observational condition test data count Correlation Variable 1 Variable 2 Correlation Total km Total motor hours 0.90 Fuel content Flash point Zinc content Calcium content 0.88 Chromium content Ferrum content 0.86 Sulphate content Nitrate content 0.71

8 Empirical analysis of observational condition test data large.part fat.part slid.part cut.part part.tot vanadium molybdenum zinc phosphor barium calcium magnes natrium borium silicon silver nickel aluminium copper lead chromium ferrum soot antioxid sulphate nitrate oxid fuel alkalinity flash viscosity mh.oil mh.tot km.oil km.tot large.part fat.part slid.part cut.part part.tot vanadium molybdenum zinc phosphor barium calcium magnes natrium borium silicon silver nickel aluminium copper lead chromium ferrum soot antioxid sulphate nitrate oxid fuel alkalinity flash viscosity mh.oil mh.tot km.oil km.tot

9 Empirical analysis of observational condition test data Two algorithms to determine Bayesian network structure from data: Growth-Shrink algorithm and Hill-Climbing algorithm. The Growth-Shrink algorithm uses Markov Blankets to detect the structure of the network. Definition A Markov Blanket BL(X ) for some variable X U is a set of variables which, for any further variables Y U BL(X ) X, has the property that X, Y are independent given BL(X ). The Hill-Climbing algorithm works as follows: 1 Start from an empty, full or random network. 2 Attempting every possible single-edge addition, removal, or reversal. 3 Choose the move which increases the score by the most. 4 The process stops when there is no single-edge change that increases the score. The score in question can either be log-likelihood, AIC, or BIC.

10 Empirical analysis of observational condition test data

11 Empirical analysis of observational condition test data natrium soot nitrate phosphor sulphate mh.tot nr ferrum km.tot oxid vanadium magnes calcium large.part nickel chromium alkalinity zinc antioxid lead borium silver slid.part molybdenum cut.part viscosity aluminium fat.part silicon fuel part.tot copper flash barium magnes vanadium oxid silver borium alkalinity sulphate nitrate slid.part viscosity fat.part nr large.part flash antioxid ferrum copper zinc calcium natrium lead chromium soot nickel molybdenum cut.part mh.tot fuel silicon phosphor barium km.tot part.tot aluminium Grow shrink Hill climbing

12 Empirical analysis of observational condition test data Scree Plot Grow shrink Eigenvalue Dimension Factor descriptions: Factor 1: chemical elements that reflect the primary materials of key dynamic (as in moving) engine parts. Factor 2: operational exposure over both degradation paths of oil and engine condition. Factor 3: operational exposure over degradation in confounded degradation path of oil and engine condition. Factor 4: measures of oil quality. Factor 5: chemical elements arising from other static. f2 Factor 6: measures of oil morphology. f6 f1 f4 f3 f5

13 Empirical analysis of observational condition test data Grow shrink algorithm barium large.part copper slid.part nitrate fat.part km.oil silicon part.tot phosphor cut.part mh.oil sulphate mh.tot km.tot molybdenum aluminium borium vanadium calcium nickel zinc chromium soot ferrum natrium alkalinity lead magnes oxid flash viscosity silver fuel antioxid

14 Engineering judgement reasoning of the problem example Expert Highly qualified (PhD) mechanical engineer with operational experience of vehicles Process Motivate and condition to provide engineering reasoning from cause to effect to establish relationships between engine condition and observables Structure around key questions to build causal maps of reasoning using why and how laddering E.g. if condition of engine is X then observation on test is Y (HOW manifest) E.g. if observe Y on test then state of engine condition is X (WHY might see) Consistency checking the causal reasoning Real-time reasoning verification by dual questions and preliminary quantification Multiple (3) stages of cross-questioning as facilitator learns Limitations Only one expert with knowledge domain of the operation, test and maintenance of vehicles

15 Engineering judgement reasoning of the problem example

16 Building a predictive model for inference about failure We can model the degredation of the oil morphology through the numbers of Cut, Slide and Fatigue particles. Y t,j,k = t X i,j,k, i=1 for vehicle j, measure k and exposure time point t. Define ( ) Yt,j,k log Y t 1,j,k Z t,j,k = d j,k (t) d j,k (t), where d j,k (t) is the exposure of vehicle j at time point t. Assume parametric forms Z t,j,k G(1, µ j,k ), µ j,k IG(α k, β k ). Then Y t,j,k µ j,k follows a geometric gamma process.

17 Building a predictive model for inference about failure

18 Summary and next steps Our problem is a typical general asset management problem although presented for specific data Our observational data are real and typical of condition monitoring data structures although our frequency of observation is less than some other assets with real-time applications (e.g. wind farms) Our judgemental data process challenges the engineer to think through causal reasoning from true condition state to observable measures of condition Our analysis reveals insights into life and condition of vehicles that align with engineering expectations about first order dependencies highlights more subtle dependency structures that should provide information about engine condition exposes the usual limitations of data mining without contextual knowledge e.g. sample number in blind analysis! Our next step is To use the insights gained to develop a predictive model grounded in the observations and the judgements

19 Summary and next steps