DATA MINING IN FINANCE

Transcription

1 DATA MINING IN FINANCE Advances in Relational and Hybrid Methods by BORIS KOVALERCHUK Central Washington University, USA and EVGENII VITYAEV Institute of Mathematics Russian Academy of Sciences, Russia KLUWER ACADEMIC PUBLISHERS Boston/ Dordrecht/London

2 TABLE OF CONTENTS Foreword by Gregory Piatetsky-Shapiro Preface Acknowledgements xi xiii xv 1. The Scope and Methods of the Study 1.1 Introduction Problem definition Data mining methodologies Parameters Problem ID and profile Comparison of intelligent decision support methods Modern methodologies in financial knowledge discovery Deterministic dynamic system approach Efficient market theory Fundamental and technical analyses Data mining and database management Data mining: definitions and practice Learning paradigms for data mining Intellectual challenges in data mining Numerical Data Mining Models with Financial Applications 2.1. Statistical, autoregression models ARIMA models Steps in developing ARIMA model Seasonal ARIMA Exponential smoothing and trading day regression Comparison with other methods Financial applications of autoregression models Instance based learning and financial applications Neural networks Introduction Steps Recurrent networks Dynamically modifying network structure Neural networks and hybrid systems in finance Recurrent neural networks in finance Modular networks and genetic algorithms Mixture of neural networks Genetic algorithms for modular neural networks Testing results and the complete round robin method Introduction Approach and method Multithreaded implementation Experiments with SP500 and neural networks Expert mining Interactive learning of monotone Boolean functions Basic definitions and results 66

3 Algorithm for restoring a monotone Boolean function Construction of Hansel chains Rule-Based and Hybrid Financial Data Mining 3.1. Decision tree and DNF learning Advantages Limitation: size of the tree Constructing decision trees Ensembles and hybrid methods for decision trees Discussion Decision tree and DNF learning in finance Decision-tree methods in finance Extracting decision tree and sets of rules for SP Sets of decision trees and DNF learning in finance Extracting decision trees from neural networks Approach Trepan algorithm Extracting decision trees from neural networks in finance Predicting the Dollar Mark exchange rate Comparison of performance Probabilistic rules and knowledge based stochastic modeling Probabilistic networks and probabilistic rules The naïve Bayes classifier The mixture of experts The hidden Markov model Uncertainty of the structure of stochastic models Knowledge based stochastic modeling in finance Markov chains in finance Hidden Markov models in finance Relational Data Mining (RDM) 4.1. Introduction Examples Relational data mining paradigm Challenges and obstacles in relational data mining Theory of RDM Data types in relational data mining Relational representation of examples First-order logic and rules Background knowledge Arguments constraints and skipping useless hypotheses Initial rules and improving search of hypotheses Relational data mining and relational databases Algorithms: FOIL and FOCL Introduction FOIL FOCL Algorithm MMDR Approach MMDR algorithm and existence theorem Fisher test 159

4 4.8.4 MMDR pseudocode Comparison of FOIL and MMDR Numerical relational data mining Data types Problem of data types Numerical data type Representative measurement theory Critical analysis of data types in ABL Empirical axiomatic theories: empirical contents of data Definitions Representation of data types in empirical axiomatic theories Discovering empirical regularities as universal formulas Financial Applications of Relational Data Mining 5.1. Introduction Transforming numeric data into relations Hypotheses and probabilistic "laws" Markov chains as probabilistic "laws" in finance Learning Method of forecasting Experiment Forecasting Performance for hypotheses H1-H Forecasting performance for a specific regularity Forecasting performance for Markovian expressions Experiment Interval stock forecast for portfolio selection Predicate invention for financial applications: calendar effects Conclusion Comparison of Performance of RDM and other methods in financial applications 6.1. Forecasting methods Approach: measures of performance Experiment 1: simulated trading performance Experiment 1: comparison with ARIMA Experiment 2: forecast and simulated gain Experiment 2: analysis of performance Conclusion Fuzzy logic approach and its financial applications 7.1. Knowledge discovery and fuzzy logic "Human logic" and mathematical principles of uncertainty Difference between fuzzy logic and probability theory Basic concepts of fuzzy logic Inference problems and solutions Constructing coordinated contextual linguistic variables Examples Context space Acquisition of fuzzy sets and membership function Obtaining linguistic variables Constructing coordinated fuzzy inference 266

5 Approach Example Advantages of "exact complete" context for fuzzy inference Fuzzy logic in finance Review of applications of fuzzy logic in finance Fuzzy logic and technical analysis 281 REFERENCES 285 Subject Index