Ref: https://www.udemy.com/course/aws-ai-practitioner-certified/learn/lecture/45796455
ML - Hyperparameter Tuning
- 🔧 Hyperparameters are settings that define the model structure, learning algorithm and process
- Set before training begins
- Can be tuned across iterations to optimize model performance
- Tuning hyperparameters for better values improves model accuracy
- reduces overfitting, enhances generalization
ML - Important Hyperparameters
Learning Rate
- Size of learning steps when updating model's weights during training
- High → faster convergence, but risks overshooting the optimal solution
- Low → more precise, but slower convergence
Batch Size
- Number of training examples used to update the model weights in one iteration
- Smaller batches → more stable learning, but more time to compute
- Larger batches → faster, but may lead to less stable updates
Number of Epochs
- Number of iterations the model will be trained with entire dataset
- ‼️ Too few epochs can lead to underfitting, while too many may cause overfitting!