10. Assignment 10: Tuning Model Parameters#
10.1. Quick Facts#
Due: 2023-11-21
Important
If you have earned at least 2 out of regression, classification, and clustering already or are confident that after the portfolio you will have 2, then I recommend submitting only assignment 11, not 10.
If you still need 2 of (regression, classification, and clustering) then complete A10 for one of those and A11 for another.
10.3. Assessment#
Eligible skills: (links to checklists)
10.4. Instructions#
summary Extend the work you did in assignment 7,8, or 9, by optimizing the model parameters.
Choose your dataset, task, and a model. It can be any model we have used in class so far. Include a breif description of the task(this can be copied from 7,8, or 9 if applicable or revised if you did not earn level 2 on those assignments but please include it for completeness).
Fit the model with default parameters and check the score. Interpret the score in context. (again, can be reused)
Choose reasonable model parameter values for your parameter grid by assessing how well the model fit with the default values.
Use grid search to find the best performing model parameters.
Examine the best fit model, how different is it from the default? Score the best fit model on a held out test set.
Examine and interpret the cross validation results. How do they vary in terms of time? Is the performance meaningfully different or just a little?
Try varying the cross validation parameters (eg the number of folds and/or type of cross validation). Does this change your conclusions?
Tip
this is best for regression or classification, but if you use clustering
use the scoring parameter to pass better metrics than the default
of the score method.
Thinking Ahead
What other tradeoffs might you want to make in choosing a model? How could you present these results using your EDA skills?