Part 2 – Evaluating your supervised learning models

a) Select three supervised learning modelling algorithms to test against one another by running the following code. Make sure you enter your student ID into the command set.seed(.). Your 3 modelling approaches are given by myModels.

b) For each of your supervised learning approaches you will need to:

i. Run the algorithm in R on the training set.

ii. optimise the parameter(s) of the model (except for binary logistic regression modelling).

iii. Evaluate the predictive performance of the model on the test set, and provide the confusion matrix for the estimates/predictions, along with the sensitivity, specificity and accuracy of the model.

iv. Perform recursive feature elimination (RFE) on the logistic regression model (only) to ensure the model is not overfitted. See Workshop 5 for an example, except in this instance, specify the argument function=lrFuncs in the rfeControl(.) command instead

c) For the logistic regression model, report on the RFE process and the final logistic regression model, including information on which k-fold CV was used, and the number of repeated CV if using repeatedcv.

d) For the other two models, report how the models are tuned, including information on search range(s) for the tuning parameter(s), which k-fold CV was used, and the number of repeated CVs (if applicable), and the final optimal tuning parameter values and relevant CV statistics (where appropriate).

e) Report on the predictive performances of the three models and how they compare to each other. 

emailsamples

malwaresamples