Machine Learning Algorithms

MINDD Models in decision making & data @nalysis Prof. Francesco Archetti archetti@disco.unimib.it Prof. Enza Messina messina@disco.unimib.it

Main Activities Research Areas: o Machine Learning Algorithms o Probabilistic and Relational Models o Optimization Under Uncertainty Applicative Domains: omultimedia o Life Sciences o Ambient Intelligence oworld Wide Web o Risk Management o Supply Chain

Machine Learning Algorithms Design, analysis and implementation of algorithms for pattern analysis, classification, clustering and prediction Our methodological skills Predictive and descriptive models Probabilistic Models Feature selection Time series analysis Applications of interest World Wide Web Document clustering Recommendation Systems Life Sciences Docking prediction Pharmacogenomic prediction Supply Chain Forecasting models Ambient Intelligence Anomaly Detection

World Wide Web User s query Hierarchical Document Clustering Group documents on the same topic into the same cluster, producing a taxonomy A B C μ D E F G H I 1 μ Taxonomy 2 Recommendation systems: o Hidden Markov Models for dynamic user behaviour using click streams o Combined content and click stream analysis for profiling web users - Machine Learning Algorithms-

Life Sciences Analysis of molecular docking Evaluation of target-drug chemical interactions based on chemical descriptors. molecular descriptors docking energy Analysis of Drug Response Cell lines derived from a variety of cancer tissues. gene expression level drug response - Machine Learning Algorithms-

Ambient Intelligence Probabilistic Models Anomaly Detection using Markov chain based models Tracking using Kalman Filter and Genetic Progamming: - Counting vehicles - Tracking objects - Machine Learning Algorithms-

Relational Models Development of algorithms which take into account (complex) relationships among a collection of different objects Our methodological skills Applications of interest Relational Clustering Relational Dynamic Bayesian Networks Social Network Analysis Probabilistic Relational Models Multimedia Video summarization Emotion Recognition Ambient Intelligence Unattended goods detection World Wide Web Document clustering Document ranking Life Sciences Pharmacogenomic prediction

Multimedia Fusion of Multimedia Sources for Video summarization Emotion Recognition - Relational Models -

Ambient Intelligence Unattended Goods Relational Dynamic Bayesian Network are able to track and make data association in complex systems L.color(t) L.color(t+1) P.hairColor(t) P.hairColor(t+1) Sensing and tracking the relations between different objects can be more effective than working directly on them. L.shape(t) P.shirtColor(t) distance(l,p,t) L.shape(t+1) P.shirtColor(t+1) distance(l,p,t+1) t - Relational Models -

World Wide Web Document representation What is the relation degree between the document entity and an image-block entity? - Relational Models -

Relational Clustering Life Sciences Relationship among gene expression profile and drug activity patterns for cancer treatment World Wide Web Clustering of Web Search Results: o Network of documents instead of single web pages o Degree of relationship among documents o Noisy content cleaning - Relational Models -

Optimization Under Uncertainty Development of model and algorithms for decision making under uncertainty Our methodological skills Applications of interest Stochastic Linear Programming Stochastic Integer Programming Scenario Generation Simulation Risk Management Scenario analysis Resource allocation models Pharmaceutical Marketing Call Planning Marketing mix optimization Life Sciences Metabolic network analysis Gene regulatory network

Risk Management Dynamic time series modelling o Switching models o Forecasting o Scenario generation Stochastic Programming models o Portfolio planning o Asset and Liability Management o Environmental Real Option Pricing o Capacity planning - Optimization Under Uncertainty -

Pharmaceutical Marketing Physicians o Optimization models for call planning based on aggregate response functions o System Dynamics for optimizing marketing mix o Social networks for targeting Response functions Targeting Relational models Prescriptions Call Planning Marketing Mix - Optimization Under Uncertainty -

Life Sciences Mathematical Programming Models for optimizing Biological Networks P 2 m 1 P 1 m 2 P 3 m 3 P 4 D 1 D 2 D 3 D 4 Cellular membrane Timed Stochastic Petri Nets to simulate Biological Networks - Optimization Under Uncertainty -

People Francesco Archetti Enza Messina Guglielmo Lulli Cristina Elena Manfredotti Elisabetta Fersini Ilaria Giordani Past PhD Students Patrick Valente, now IT Manager at Barclays Capital, UK Nico Di Domenica, now Senior Risk Analyst at Royal Bank of Scotland, UK Valentina Bosetti, now Senior Researcher at FEEM, Italy Toscani Daniele, now Researcher at CMR, Italy

A cooperation network for research projects and student mobility University of Toronto Brunel University CARISMA Research Center Norwegian University of Science and Technology Aachen University Massachusset Institute of Technology Hungarian Academy of Sciences Centre of Research and Technology Hellas -TXT e-solutions -Siemens -Project Automation -Aegate Ltd -OptiRisk -Astra Zeneca -DELOS -Comerson

Projects o JUMAS Judicial Management by Digital Libraries Semantics, EU FP7 o INSYEME - Integrated System for Emergency Management, MUR o TRADE - Tracking RFID-based Agents in Distributed Environments, Regione Lombardia o An integrated system for 3D image processing, (Fondo per l innovazione tecnologica Ministero per lo sviluppo economico, in collaborazione con Comerson s.r.l.) o Call Plan, Astra Zeneca o Bayesian Fusion of Stochastic Models, TXT o E-RelationHub, TXT o OSP - Optimization Service Provision EU FP5

Journal Publications (1) F. Archetti, E. Fersini, E. Messina Enhancing Web Page Classification using Visual Block Analysis. To appear in Information Processing and Management. F. Archetti, S. Lanzeni, E. Messina, L. Vanneschi, Genetic Programming for Computational Pharmacokinetics in Drug Discovery and Development. To appear in Genetic Programming and Evolvable Machines. E. Messina, D. Toscani Hidden Markov Models for Scenario Generation. To appear in IMA Journal of Management Mathematics. F. Archetti, S. Lanzeni, E.Messina, "Graph Models and Mathematical Programming in Biochemical Networks Analysis and Metabolic Engineering Design. To appear in Computers & Mathematics with Applications. G. Lulli, M. Romauch: A Mathematical Program to Refine Gene Regulatory Networks, to appear in Discrete Applied Mathematics. G. Andreatta, G. Lulli, A Multi-period TSP with Stochastic Regular and Urgent Demands, European Journal of Operations Research, 185, 1, pp. 122-132, 2008. P. Dell Olmo, A. Iovanella, G. Lulli, B. Scoppola, Exploiting incomplete information to manage multiprocessor, to appear in Computers and Operations Research, 35, 5, pp. 1589-1600, 2008. G. Lulli, A.R. Odoni, The European Air Traffic Flow Management Problem, Transportation Science, 41, 4, pp. 1-13. (short version accepted for the 11th IFAC Symposium on Control in Transportations Systems, August 2006, Delft - The Netherlands), 2007. G. Lulli, S. Sen, A Heuristic Algorithm for Stochastic Integer Program with Complete Recourse, European Journal of Operations Research, 171, 3, pp. 879-890, 2006. E. Messina, V. Bosetti, Integrating stochastic programming and decision tree techniques in land conversion problems, Annals of Operations Research, 142, pp. 243-258, 2006. M.O. Ball, G. Lulli, Ground Delay Programs: Optimizing over the Included Flight Set Based on Distance, Air Traffic Control Quarterly, 12, pp. 1-25, 2004. G. Lulli, S. Sen, A Branch-and-Price Algorithm for Multi-stage Stochastic Integer Programming with Application to Stochastic Batch-Sizing Problems, Management Science, 50, 6,pp. 786-796, 2004. P. Dell'Olmo, G. Lulli, Planning Activities in a Network of Logistic Platforms with Limited Capacity, Annals of Operations Research, 129, Issue 1-4, pp. 155-169, 2004.

Journal Publications (2) V. Bosetti, J.M.Conrad, E. Messina, The Value of Flexibility: Preservation, Remediation, or Development for Ginostra? Environmental and Resource Economics, 29, 2, pp. 219-229, 2004. P. Dell'Olmo, G. Lulli, A new hierarchical architecture for Air Traffic Management: Optimisation of airway's capacity in a free flight scenario, European Journal of Operations Research, 144, 1, pp. 179-193, 2003. P. Dell'Olmo, G. Lulli, A Dynamic Programming Approach for the Airport Capacity Allocation Problem, IMA Journal of Management Mathematics, 14, pp. 235-249, 2003. E. Messina, V. Bosetti, Uncertainty and Option Value in Land Allocation Problems, Annals of Operations Research 124, pp. 165-182, 2003. V. Bosetti, E. Messina, P. Valente, "Optimisation Technologies and Environmental Applications". IMA Journal of Management Mathematics, 13, pp. 167-185, 2002. S.A. Mir Hassani, C. Lucas, E. Messina, G. Mitra, Computational Solution of Capacity Planning Models under Uncertainty, Parallel Computing, 26, pp. 511-538, 2000. E. Messina, G. Mitra, "Modelling and analysis of multistage stochastic programming problems: a software environment European Journal of Operational Research, 101, pp. 343-359, 1997. P. Baricelli, C. Lucas, E. Messina, G. Mitra A model for strategic planning under uncertainty, TOP: O.R. in Practice, 4, 2, pp.361-384, 1996. A. Gaivoronski, E. Messina, A. Sciomachen, A statistical generalized programming algorithm for stochastic optimization problems, Annals of Operations Research, 58, pp. 297-321, 1995. A. Gaivoronski, E. Messina, A. Sciomachen, A stochastic optimization approach for robot scheduling, Annals of Operations Research, 56, pp. 109-133, 1995. F. Archetti, E. Messina, A. Sciomachen, "A graph theoretical approach to the performance analysis of highly concurrent systems", Journal of Combinatorial, Information and System Science, 19, 1-2, pp.87-95, 1994. E. Messina, A. Sciomachen, "Evaluation of resource allocation policies in a production line using Petri nets", Robotics & Computer- Integrated Manufacturing, 10, 6, pp.413-422, 1993.

International Conference Proceedings and Book Chapters (1) D. Bertsimas, G. Lulli, A. Odoni: The Air Traffic Flow Management Problem: An Integer Optimization Approach. To appear in Proceedings of 13th International Conference IPCO 2008 - Bertinoro, to appear in LNCS. K.F. Doerner, W. J. Gutjahr, R.F. Hartl, G. Lulli, Stochastic Local Search Procedures for the Probabilistic Two-Day Vehicle Routing Problem, to appear in A. Fink and F. Rothlauf eds., Advances in Computational Intelligence in Transportation and Logistics, Springer Series on Studies in Computational Intelligence. E. Fersini, C. Manfredotti, E. Messina, F. Archetti. Relational Clustering for Gene Expression Profiles and Drug Activity Pattern Analysis. SysBioHealth Symposium (ISBN: 978-88-903154-0-4), 2007. I. Giordani, L. Vanneschi, E. Fersini. Modelling the Relationship between the Microarray Data of the NCI-60 Anticancer Dataset with Therapeutic Responses by Genetic Programming. SysBioHealth Symposium (ISBN: 978-88-903154-0-4), 2007. S. Lanzeni, E. Messina, F. Archetti, Towards metabolic networks phylogeny using Petri Net-based expansional analysis, BMC Systems Biology 2007, 1(Suppl 1). F. Archetti, S. Lanzeni, E. Messina, L. Vanneschi "Genetic Programming and other Machine Learning approaches to predict Median Oral Lethal Dose (LD50) and Plasma Protein Binding levels (%PPB) of drugs" Lecture Notes in Computer Sciences, Evolutionary Computation,Machine Learning and Data Mining in Bioinformatics, 5th European Conference, EvoBIO 2007. F. Archetti, C. Manfredotti, M. Matteucci, E. Messina and D. G. Sorrenti, Multiple Hypotesis Markov Chains For On-Line Anomaly Detection in Traffic Video Surveillance, Proceedings ICDP 2006: Imaging for Crime Detection and Prevention, 13-14 June 2006. F.Archetti, C.E. Manfredotti, E. Messina, and D. G. Sorrenti Foreground-to-ghost Discrimination in Single-difference Preprocessing, Lecture Notes in Computer Science: Advanced Concepts for Intelligent Vision Systems, ACIVS 06, 263-274, 2006. F. Archetti, S. Lanzeni, E. Messina, L. Vanneschi, Genetic Programming for Human Oral Availability of Drugs, Lecture notes in Computer Science: Genetic and Evolutionary Computation (GECCO 06), 2006. F. Archetti, E. Messina, D. Toscani, L. Vanneschi, "Classifying and Counting Vehicles in Traffic Control Applications" Lecture Notes in Computer Science: Applications of Evolutionary Computing, 2006.

International Conference Proceedings and Book Chapters (2) F. Archetti, E. Fersini, P. Campanelli, E. Messina, "A Hierarchical Document Clustering Environment Based on the Induced Bisecting k-means" Lecture Notes in Computer Science: Flexible Query Answering Systems, 2006. F. Archetti, E. Messina, D. Toscani, "UP-DRES User Profiling for a Dynamic REcommendation System", Lecture Notes in Computer Science: Advances in Data Mining. Applications in Medicine, Web Mining, Marketing, Image and Signal Mining,, 2006. G. Andreatta, G. Lulli: The 2-period Probabilistic TSP: a Markov Decision Model, Proceedings of the EWGT2006 Joint Conferences: The 11th Meeting of the EURO Working Group on Trasportation Advances in Traffic and Transportation Systems Analysis and Extra EURO Conference Handling Uncertainty in Transportation, Technical University of Bari, September 27-29, pp. 672-676 (ISBN: 88-901798-2-1), 2006. F. De Paoli, G. Lulli, A. Maurino. Design of Quality-based Composite Services. In Proceedings of 4th International Conference on Service Oriented Computing - ICSOC 2006 - Chicago, USA LNCS vol. 4294, pp. 153-164, 2006. G. Lulli, G. Andreatta, Congestion Pricing and Queue Theory in Proceedings of the 1st International Conference on Research in Air Transportation, Žilina, ISBN 80-8070-196-2, 2004. P. Dell Olmo, G. Lulli, Models and Algorithms for the Airport Capacity Allocation Problem, in T. Ciriani et al. ed., OR is Space and Air, Kluwer Academic Publisher, pp. 351 368, 2003. G. Lulli, S. Sen, Stochastic Batch-Sizing Problems: Models and Algorithms, in D.L. Woodruff ed., Stochastic Integer Programming and Network Interdiction Models, Kluwer Academic Publisher, pp. 85-104, 2002. P. Dell Olmo, G. Lulli, A Mathematical Programming Approach to Optimize Airport Capacity in: Proceedings of the Fifth International Conference on Mathematical Programming and Applications, Varenna, 2002. G. Lulli, S. Sen, Scenario Updating Method for Stochastic Mixed-integer Programming Problems in: U. Leopold-Wildburger, F. Rendl and G. Wäscher (eds), The OR 2002 proceedings, Springer Verlag, Klagenfurt, 2002. L. Bianco, P. Dell Olmo, S. Giordani, G. Lulli, Models and Algorithms for Multi-airport Traffic Flow Coordination in: Galati and Zellweger (ed.), Proceedings of the International ATM 02 Workshop, Capri, 2002. V. Bosetti, C. Conrad and E. Messina, The value of flexibility, CRENoS Working Paper, Cagliari, 2001.

International Conference Proceedings and Book Chapters (3) F. Archetti, E. Messina, F. Stella, Modellazione e simulazione del traffico urbano: un approccio basato su agenti autonomi, Proceedings Convegno PFT2, Taormina, 1997. F. Archetti, E. Messina, B. Mishra, F. Stella, (1997) CATS: a Complex Adaptive Traffic Simulator, Proceedings of the IFAC\IFIP\IFORS Symposium on Transportation Systems, Chania, Greece, Vol. 3, pp. 1340-1344, 1997. F. Fantauzzi, A. Gaivoronski, E. Messina, Decomposition methods for network optimization problems in the presence of uncertainty, Lecture Notes on Economics and Mathematical Systems: Network Optimization, P. M. Pardalos, D.W.Hearn, W.W. Hager (Eds.), Gainesville, Florida, Vol. 450, pp. 234-248, 1996. A. Gaivoronski, E. Messina, Optimization of stationary behavior of general stochastic discrete event dynamic systems, Proceedings WODES96-International Workshop on Discrete Event Systems, IEEE, pp. 238-243, Edimburg, U.K., 1996. C. Lucas, E. Messina, G. Mitra, Risk and return analysis of multi-period strategic planning problems, Lecture Notes in Economics and Mathematical Systems: Stochastic Modelling in Innovation Manufacturing, A.H. Christer, S. Osaki, L.C. Thomas (Eds.), Springer- Verlag, Cambridge, pp. 81-96, 1995. F. Archetti, E. Messina, Process flexibility through stochastic optimization: a computational approach, Optimization in Industry, A. Sciomachen editor, John Wiley, Vol. 3, pp. 109-125, 1995. E. Messina, M. Colombo, "Application of Linear Programming reduction procedures to manufacturing models", Intelligent Automation and Soft Computing: Trends in Research, Development and Applications, M. Jamshidi, J. Yuh, C. C. Nguyen and R. Lumia eds., V.2, pp. 35-38, 1994. A. Gaivoronski, E. Messina, Stochastic optimization algorithms for regenerative DEDS, Lecture Notes in Control and Information Sciences: System Modelling and Optimization, J. Henry and J-P. Yvon eds., Springer Verlag, Vol. 197, pp. 320-331, 1993. E. Messina, A. Sciomachen Optimal sequencing via structural simulation, Proceedings of the 36th annual Conference Automation, Genova (Italy), Vol 3, pp 197-210, 1992.