Predictive Modeling of Titanic Survivors: a Learning Competition

Transcription

1 SAS Analytics Day Predictive Modeling of Titanic Survivors: a Learning Competition Linda Schumacher

2 Problem Introduction On April 15, 1912, the RMS Titanic sank resulting in the loss of 1502 out of 2224 passengers and crew. Predicting the survivors based on demographic variables is a predictive modeling classification problem. kaggle.com hosts a Titanic Getting Starting public competition with model scoring based on the accuracy fit statistic. Predictive Modeling was performed using SAS Enterprise Miner 12.1

3 Data Exploration Target variable is binary: survived. Input variables: name, age, fare, sex, embarked, cabin, ticket, pseudo class, #siblings/spouse, #parents/children. Sex, social standing related variables (fare, pclass, cabin) and age have highest worth.

4 Date Preparation Impute Node Age has 20% missing values and is imputed with a tree method using a computed variable, title, #siblings/spouse, and #parents/children. Title is extracted from the name. Transform Node Fare is right skewed with a large range. The log transformation improved the skew and kurtosis. Range Skew Kurtosis fare log(fare+1)

5 Modeling in SAS Enterprise Miner Data Partitioning Node Because of the limited number of cases, 890 passengers, data was partitioned into 70% training and 30% validation. A separate test set of 418 passengers was used for scoring. Modeling Nodes Decision Trees, Gradient Boosting Machine, Logistic Regression, Neural Network, Rule Induction. Modeling objective is highest accuracy on scored test data.

6 Modeling Variable Importance Title is the most important variable identified by trees, GBM, and logistic regression. Models next selected variables pclass, #siblings/spouse, ticket, fare, and age. Splits on #parents/children, embarked, and cabin rarely occurred.

7 Modeling Diagram Ensemble A Autonomous Decision trees were built using split criteria entropy or Gini. Trees were optimized using the Average Square Error assessment on validation data. An interactive tree was also created. A Gradient Boosting Machine with a maximum depth of 10 and a maximum of 2 surrogate rules was created.

8 Model Comparison Fit Statistics The SAS EM nodes all performed well. The ensembles have the best accuracy fit statistics on scored test data. Model Valid ROC Valid ASE Valid MISC Test Accuracy Score Ensemble A Ensemble B DT entropy GBM Regression Neural Net Rule Induct DT interact Ensemble B

9 Model Comparison ROC Chart Validation ROC index: Test accuracy: An autonomous tree using the entropy split criterion had the highest ROC index. Ensemble A had the highest test accuracy.

10 Modeling Results Title was initially created to impute age but became the most important variable for predictions. Title encapsulates age, gender, marital status, and some professions. In general, models predicted survivors with titles: Miss, Mrs., Master Females traveling in 1 st & 2 nd class and pockets within 3 rd class. Males 12 and younger with less than 3 siblings. Mr., Rev., and Others were not predicted to survive. Males 13 and older. Note that although 72 men were survivors in training & validation data, the models did not identify any groups of men as survivors. No Reverends in training or validation data survived

11 Conclusion Decision Trees are good predictive models for the Titanic disaster survival because: Variables are related or redundant. Most variables are ordinal or nominal. Trees make no assumptions on the distributions of variables. Interactions between variables are present. Interpretation of tree models is straightforward. Ensemble nodes improve on the predictions of component models by forming consensus predictions.

12 For more details contact presenter at: Linda Schumacher Phone: Faculty Advisor: Dr. Goutam Chakraborty Competition Website: Acknowledgements: The author thanks Austen Head, Rick Pack, and Brian Fannin for their valuable comments and suggestions.