Assignment 10: Tuning Model Parameters
Contents
10. Assignment 10: Tuning Model Parameters#
10.1. Quick Facts#
Due: 2020-11-17 11:59pm
10.3. Asseessment#
task |
skill |
---|---|
test the model on test data and interpret in context |
classification, clustering, OR regression (2) |
choose and justify appropriate parameters parameter grid for the context |
classification, clustering, OR regression (2) AND process (2) |
evaluate fit of the model while varying the cross validation parameters and interpret |
evaluate (2) |
optimize model parameter (s) and interpret |
optimize (2) |
ask releveant questions of the data domain and interpret results in context |
process (2) |
use EDA techniques to examine the experimental results |
summarize (2), visualize (2) |
Note you can only earn one of classification, clustering, OR regression, but to earn one of those you must both interpret the model and the parameters.
For process you must situate your overall analysis in context (which you should do even if you already have the process achievement; you should always do this) and explain how you’re picking parameters to evaluate. Your reasons only have to be reasonable, they don’t have to be correct. It’s okay if what you try doesn’t improve the model, but then you have to interpret that.
10.4. Instructions#
summary Extend the work you did in assignment 7,8, or 9, by optimizing the model parameters.
Choose your dataset, task, and a model. It can be any model we have used in class so far.
Fit the model and show some exploration to choose reasonable model parameter values for your parameter grid
Use grid search to find the best performing model parameters
Examine and interpret the cv results. How do they vary in terms of time? Is the performance meaningfully different or just a little?
Try varying the cross validation parameters. Does this change your conclusions?
Tip
this is best for regression or classificaiton, but if you use clustering
use the scoring
parameter to pass better metrics than the default
of the score method.
Hint
Assignment 11 will be to optimize two models and then compare two models on the same task
Thinking Ahead
What other tradeoffs might you want to make in choosing a model? How could you present these results using your EDA skills?