Mathematics in Data Science. Hi Jun Choe. CMAC(Center for Mathematical Analysis and Computations) Yonsei University, Seoul, Korea

Transcription

1 Mathematics in Data Science Hi Jun Choe CMAC(Center for Mathematical Analysis and Computations) Yonsei University, Seoul, Korea 1 / 20

2 1. Introduction Many believe we are in the process of the 4th industrial revolution and society is experiencing fundamental changes now. Communication and electronics technology were main basis for the 3rd revolution during 1990 to As a mathematician, I felt some distance to communication and electronics although important theories depend on mathematics. This spring Korean society were very much impressed by Seoul AlphaGo which was developed by UK company DeepMind and it won world champion Sedol Lee in go game. The 4th revolution is going to be driven by software technology like artificial intelligent and big data where mathematics is essential. Indeed most software is a mathematical reasoning and algorithm. I will explain machine learning and mathematics related to data science in depth and provide some insight. 2 / 20

3 In data science, which is still ambiguous in definition, there are two subjects; Data mining-big data Machine learning-neural network, Support vector machine, Decision tree, Bayesian classifier The role of Machine learning; RawData Datawarehouse Knowledge 3 / 20

4 The applications, which are treated in data, science include manufacturing, finance, robotics, internet(web), medicine, commercial, aviation, etc. Indeed, all aspects of our life are objects of data science. Therefore we may conclude that Data science is interdisciplinary. Single person can not have all necessary knowledges and energy, and collaboration is essential. For example, DeepMind, which developed AlphaGo, has 20 members from various fields. 4 / 20

5 2. Machine learning(ml) Because this talk is about mathematics in data science, I like to concentrate only on ML. But working in data mining(big data) also needs good understanding of various fields in mathematics. Essentially, ML has two subjects; classification and prediction. Historically there were two schools 1. Statistical method(statistics), 2.Structural recognition(computer science). 5 / 20

6 ML has three main ingredients- representation algorithm, evaluation algorithm, optimization. First, representation algorithm expresses cognitive activity and there are the methods; 1.Decision tree 2.Sets of rules 3.Bayesian/Markov test 4. Neural network 5. Support vector machine 6.clustering. Representation algorithm includes a large number of parameters which are to decided by learning from data. 6 / 20

7 Second we may find the following evaluation algorithms; 1.Posterior probability, 2.Likelihood, 3.Square root error, 4. Margin, 5.Accuracy, 6.Decision and recall, 7.Cost/Utility 8.entropy, etc In case of neural network we use sigmoid function of Reluc function to introduce nonlinearity of human nature. In case of decision tree, we use entropy of probability to find best tree structure. 7 / 20

8 Optimization algorithm provide the most probable or suitable parameters for given data or environment. For the parameters appearing in representation algorithm, we infer the most efficient parameters by optimization or learning. In fact most computation time in ML is spent in optimization and the word learning is from this process. 8 / 20

9 We emphasize most algorithms are iteration and time consuming. The followings are in practice; 1.Convex optimization(gradient descent), 2.Constraint optimization(linear programming), 3.Combinatorial optimization. It usually requires a large memory and fast processes. Recent progress of ML begins with a discovery of backpropagation which enables a large parameter optimization. 9 / 20

10 The feature of ML has one of the following form according to objects or data input method; 1. Supervised learning, 2. Semisupervised learning 3. Reinforcement learning, 4. Unsupervised learning The most successful ML related to image processing adopted supervised learning, but recent trend moves to unsupervised or reinforcement learning. For example Euro-AlphaGo adopted supervised learning, but Seoul-AlphaGo adopted reinforcement learning and it was powerful enough to beat World Go Champion Lee, Sedol. 10 / 20

11 There are almost 700 magazine for ML and the number of articles increases exponentially. I can not cover all and only list typical methods; 1.Neural network, 2.Support vector machine, 3.Decision tree, 4. Rule induction, 5.Bayesian learning, 6. Instance based learning, 7.Model ensemble, 8.etc By way of supervised learning, we conduct classification, prediction, regression, knowledge extraction and anomaly detection. 11 / 20

12 Examples of classification could be 1.Face recognition, 2.Speech recognition, 3.Medical diagnosis, 4.Pattern recognition, 5.Character recognition, 6.etc. So far the successful applications are 1.Speech recognition, 2.Computer vision 3.Medical analysis, 4.Robot control, 5.Computational biology, 6.Finance. 12 / 20

13 Unsupervised learning are closely related to Big Data. The methods are 1.Clustering, 2.Dimensionality reduction, 3.etc. 13 / 20

14 3. Successful implementation of ML I would like to suggest several steps for successful implementation of ML. Step 1. First we need to understand the objects well. Since most problems are ambiguous and unstructured, one needs to have very clear goal. In this step, one needs to grasp data format and acquisition method. Various time consuming trial errors occur from this step. Step 2. Data acquisition, data mining, data cleansing, preprocess and datawarehouse are next step. One might needs a good understanding of computer architecture, coding languages, software packages. Sometimes we call this step data mining. 14 / 20

15 Step 3. One has to find suitable learning model. One needs to choose ML model appropriate for data from supervised, reinforcement or unsupervised learning. This step requires good understanding mathematics and analytics. Step 4. Analyze the results like overfitting to adjust ML model. Various statistical method and computer graphics are necessary. Step 5. Complete ML model and apply real world. Monitor ML model. Save and track necessary data. Step 6. Continuously optimize model from the saved data. 15 / 20

16 4. Mathematics Theorem by Cybencko. Theorem The class of neural networks is dense in the continuous function class. The following areas in mathematics are essential; 1. Real analysis, 2. Linear algebra, 3. Statistics and probability, 4. Matrix and convex analysis 16 / 20

17 Neural network One may write neural network in minimization problem; Set W = R n the parameter space whose components are partially ordered. We have an input and out set D = {(x i, y i ), i = 1, 2,...} which is learning data and cost function is given as L((x, y), w). We want to minimize min w W L((x i, y i ), w). Gradient descent is realized by backpropagation. i 17 / 20

18 Support vector machine Optima separating plane is decided by constraint minimization.(cortes-vapnik, 1995) Minimize Φ(w, η) = η σ i subject to y i (w x i + b) 1 η i, η i > / 20

19 Clustering For given data matrix D, minimize the rank of A subject to A part = D part. 19 / 20

20 Thank you for your attention! 20 / 20