Hyperparameter Tuning
- 🔧 Hyperparameters are settings that define the model structure, learning algorithm and process
- Set before training begins
- Can be tuned across iterations to optimize model performance
- Tuning hyperparameters for better values improves model accuracy
- reduces overfitting, enhances generalization
Important Hyperparameters
Learning rate
- Size of learning steps when updating model's weights during training
- High → faster convergence, but risks overshooting the optimal solution
- Low → more precise, but slower convergence
Batch size
- Number of training examples used to update the model weights in one iteration
- Smaller batches → more stable learning, but more time to compute
- Larger batches → faster, but may lead to less stable updates
Number of Epochs
- Number of iterations the model will be trained with entire dataset
- ‼️ Too few epochs can lead to underfitting, while too many may cause overfitting!