Hyperparameter Search Strategies

• Hyperparameter Tuning Methods - Grid, Random or Bayesian Search? | Towards Data Science
• 3.2. Tuning the hyper-parameters of an estimator — scikit-learn 1.7.2 documentation Parameters to optimize:
• Layer count of deep neural network
• Neuron count per layer
• Learning rate
• etc. Automated Machine Learning - Successive Halving and Hyperband - YouTube Major Methods:

1. Grid Search: catesian product of all candidate parameters, and search each one
2. Random Search: sample each hyperparameter from a uniform/some chosen distribution
3. Halving Search
   1. Halving Random: sample all from distribution, then successively eliminate bad ones
   2. Halving Grid: take all cartesian products, then successively eliminate bad ones