View source for MMS: Data Mining (2018-I)

== Course contents ==

=== Chapter 1. Introduction to PYTHON ===

==== Session 1 ====
* Downloading and installing PYTHON
* Installing and loading packages
* Reading and writing data
* Converting types on character variables

# Presentation [[File:MMS DataMining1.pdf| slides]]
## First script [http://hpclab.ucentral.edu.co/~hfranco/session1_first_script.ipynb]

=== Chapter 2. Data Exploration with Real Live Data ===
==== Session 2 ====
* Using PYTHON to manipulate data
* Reading a dataset from a CSV file
* Applying basic statistics
* Working with univariate descriptive statistics in PYTHON
* Performing correlations.
* Operating a probability distribution in PYTHON
* Fitting a linear regression model with Python
* Summarizing linear model fits

'''Files for Download:'''
* Session 2 (iPython Notebook) [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/2ndSessionPython-13-02-2018.ipynb]
* Data examples:
** Text File [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/animals.txt]
** Excel File [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/93CAR.xlsx]
** CSV File [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/Yayacon.csv]
* Interactive Regression - Machine Learning (iPython Notebook)  [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/InteractiveRegressionML.ipynb]
* Interactive Logistic Regression (iPython Notebook) [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/InteractiveLogisticRegression.ipynb]
* Curve Fitting (iPython Notebook)  [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/Ajuste_Curvas_1.ipynb]
'''Homework - Workshop 1:'''
* Workshop iPython Notebook: [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/TallerNo1.ipynb]
* Dataset 1 (mobile phone robbering data sample -Colombia, 2017): [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/muestra_hurto.xlsx]
* Dataset 2 (customer database -marketing): [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/customer_dbase.xlsx]

==== Session 3 ====
* Detecting missing values
* Imputing missing values
* Understanding data sampling in PYTHON
* Parametric and non-parametric statistical inference
* Conducting an exact binomial test

'''Files to download'''

* Sampling and imputation [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/3rdSession-FaltantesMuestreo.ipynb]
* Normal Distribution [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/3rdSession-NormalDistribution.ipynb]
* Example data [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/INMISIONES.xlsx]
'''Homework - Workshop 2:'''
* Workshop iPython Notebook: [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/Taller2_Data_Mining_MMS.ipynb]

==== Session 4 ====
* Performing the Kolmogorov-Smirnov test
* Understanding the Wilcoxon Rank Sum and Signed Rank test
* Working with Pearson’s Chi-squared test
* Conducting a one-way ANOVA
* Performing a two-way ANOVA
'''Files'''
* Nonparametric tests [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/4thSession-NoParametricas.ipynb]
* Example data [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/bank-additional-full.xlsx]

==== Special Session: Project definition ====
* Propose the course final project based on a real-life challenging data analysis problem
* Form: [[Data Analysis Problem definition]]
 

==== Session 5 ====
* Introduction to Machine Learning
* Supervised and unsupervised learning
* Getting a suitable dataset
* Predicting passenger survival with a decision tree
* Validating the power of prediction with a confusion matrix
* Assessing performance with the ROC curve

'''Files:'''
* Session presentation [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/SessionV/VSession.pdf]
* Session notebook [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/SessionV/V_Session_10-04-2018.ipynb]
* Balance scale data file [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/SessionV/balance-scale.data]
* Titanic test [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/SessionV/titanic_test.xlsx]
* Titanic test [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/SessionV/titanic_train.xlsx]
* Graphviz [http://hpclab.ucentral.edu.co/~hfranco/data_analysis/SessionV/graphviz-2.38.zip]

=== Chapter 2. Classification (I) – Tree, Lazy, and Probabilistic ===

==== Session 6 ====
* Preparing the training and testing datasets
* Building a classification model with recursive partitioning trees
* Visualizing a recursive partitioning tree
* Measuring the prediction performance of a recursive partitioning tree
* Pruning a recursive partitioning tree

==== Session 7 ====

* Building a classification model with a conditional inference tree
* Visualizing a conditional inference tree
* Measuring the prediction performance of a conditional inference tree
* Classifying data with logistic regression

==== Applied Material (to be updated) ====

=== Chapter 3.  Working with Spatial Data ===

==== Session 8 ====

* Motivation: What's so great about spatial data? 
* Spatial data structures.
* Making maps. 
** Static maps with Python. 
** Projections.
** Geocoding, routes, and distances. 
** Dynamic maps with leaflet. 
* Extended example: Congressional districts. 
** Election results. 
** Congressional districts. 
** Putting it all together. 
** Using leaflet.
* Effective maps: How (not) to lie. 
* Extended example: Historical airline route maps. 
* Projecting polygons.