Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis Philippe LERAY, Olivier François, Ahmad Faour contact: Philippe.Leray@insa-rouen.fr PSI (Perception, Systems and Information) Laboratory FRE CNRS 2645 Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 1/12
Structural learning complete data The DAG space h a super-exponential size heuristics! Constraint bed methods (IC, PC, BN-PC...) Score bed methods complete search in Tree space (MWST) greedy search in DAG space, with node ordering (K2) or without (GS) greedy search and Markov equivalence (GES) Conferences : François & Leray RJCIA 03 (french), RFIA 04 (french) Journal : JEDAI 04 (french) MWST = good performances vs. computation time MWST for GS initialisation = robust initialisation Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 2/12
Structural learning incomplete data Few methods deal with incomplete data Usual principle = applying EM to score bed methods greedy search in DAG space (SEM = GS+EM) Conference : François & Leray EGC05 (french) [subm. to ECSQARU 05] : MWST+EM = MWST + score estimation with EM MWST+EM for SEM initialisation = robust initialisation Perspectives : greedy search and Markov equivalence = GES+EM constraint bed methods and incomplete data Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 3/12
Structural learning latent variables Combinatorial explosion Where are the latent variables in the DAG? Cardinality? new operators in SEM space restriction : hierarchical latent cls model (HLC) Conference : Leray & al. ECML03 Workshop (PGM for clsification) Tree augmented HLC Perspectives : SEM+EM = dealing with incomplete data and latent variable discovery Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 4/12
Structural learning a priori knowledge Using a priori knowledge to simplify the search space Perspectives : Dynamic bayesian networks (2TBN) = 2 structures : intra-slice (t) and inter-slice (t t + 1) Oriented object bayesian networks (OOBN), Multi-agent bayesian networks,... Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 5/12
Complex system modelling and diagnosis Discovering handwriting strategies of primary school children I. Zaarour PhD thesis (completed in feb. 2004) Collaboration with a psychology lab (PSY.CO Rouen) Conferences : ECML03 Worshop - IGS03 - RFIA04 (french) Journal : IJPRAI 04 Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 6/12
Complex system modelling and diagnosis Intrusion detection in computer networks A. Faour PhD thesis (begin sept. 2004) Collaboration with a network security expert Conferences : EGC05 Worshop (french) Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 7/12
Complex system modelling and diagnosis Bayesian networks for clsification O. François PhD thesis (end envisaged in dec. 2005) Journal : RIA 2004 (french) Dysfunction detection and localisation in a chemical reactor Collaboration with a chemical process engineering lab (LRCP Rouen) Conference : SFGP 2005 (french) Micro-wave transistor thermical modelling Project with Thales Air Defense and a aero-thermochemistry lab (CORIA Rouen) financed by Haute-Normandie Region G. Mallet MSc thesis (feb-july 2005) followed by a PhD Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 8/12
Scientific animation French workshop on bayesian networks : June 2001 first workshop, Paris (co-organisation) March 2003 second workshop, Rouen. Jan. 2005 French PGM workshop during EGC 2005 conference, Paris. Software : BNT Toolbox for Matlab code contributions responsable for structure learning package french BNT website and documentation http ://bnt.insa-rouen.fr Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 9/12
International activities Members of PASCAL (Pattern Analysis, Statistical Modelling and Computational Learning) european network of excellence Collaborations Causal networks and structural learning S. Meganck & B. Manderick, Computational Modeling Lab, Vrije Universiteit Brussel (VUB), Belgium. Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 10/12
Selected bibliography (in english) http ://i.insa-rouen.fr/ pleray/publisrb.php International journals : Zaarour, I. et al. (2004). Clustering and bayesian network approaches for discovering handwriting strategies of primary school children. International Journal of Pattern Recognition and Artificial Intelligence, 18(7) :1233-1251. International conferences : Leray, P.et al. (2003). A bayesian model for discovering handwriting strategies of primary school children. In Working Notes of the Workshop on Probabilistic Graphical Models for Clsification, ECML/PKDD-2003, 49-57. Zaarour, I.et al. (2003). A bayesian network model for discovering handwriting strategies of primary school children. In 11th Conference of the International Graphonomics society (IGS 2003), 178-181. Misc : Leray, P. and Francois, O. (2004). BNT structure learning package : Documentation and experiments. Technical report, Laboratoire PSI. Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 11/12
Selected bibliography (in french) Books : Naïm, P., Wuillemin, P.-H., Leray, P., Pourret, O., and Becker, A. (2004). Réseaux bayésiens. Eyrolles, Paris. French journals : Leray, P. and Francois, O. (2004). Réseaux bayésiens pour la clsification - méthodologie et illustration dans le cadre du diagnostic médical. Revue d Intelligence Artificielle, 18/2004 :169-193. François, O. and Leray, P. (2004). Etude comparative d algorithmes d apprentissage de structure dans les réseaux bayésiens. Journal électronique d intelligence artificielle, 5(39) :1-19. French conferences : Francois, O. and Leray, P. (2005). Apprentissage de structure dans les réseaux bayésiens et données incomplètes. In Proceedings of EGC 2005 (to appear), 1-6. Faour, A. and Leray, P. (2005). Réseaux bayésiens pour le filtrage d alarmes dans les systèmes de détection d intrusion. In Proceedings of EGC 2005 Atelier Modèles graphiques probabilistes (to appear), 1-8. Francois, O. and Leray, P. (2004). Evaluation d algorithmes d apprentissage de structure pour les réseaux bayésiens. In Proceedings of 14ème Congrès Francophone Reconnaissance des Formes et Intelligence Artificielle, RFIA 2004, 1453-1460. Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis p. 12/12