Training Optimal Models
0 likes40 views
The document discusses hyperparameter tuning in machine learning, specifically using Azure Machine Learning to optimize model performance through multiple training runs with varying hyperparameters. It explains defining search spaces for discrete and continuous hyperparameters, and emphasizes the importance of configuring sampling and early termination policies to improve efficiency during the tuning process. It also provides links and references for further information.
![Defining a search space
The set of hyperparameter values tried during hyperparameter tuning is known as
the search space. The definition of the range of possible values that can be chosen
depends on the type of hyperparameter.
Discrete hyperparameters
Some hyperparameters require discrete values - in other words, you must select the value
from a particular set of possibilities. You can define a search space for a discrete
parameter using a choice from a list of explicit values, which you can define as a
Python list (choice([10,20,30])), a range (choice(range(1,10))), or an arbitrary set of comma-
separated values (choice(30,50,100))](https://image.slidesharecdn.com/c8-trainingoptimalmodels-221105045844-c22a082e/85/Training-Optimal-Models-4-320.jpg)
Training Optimal Models
- 2. Eng Teong Cheah Microsoft MVP
- 3. Hyperparameter Tuning Hyperparameter tuning is accomplished by training the multiple models, using the same algorithm and training data but different hyperparameter values. The resulting model from each training run is then evaluated to determine the performance metric for which you want to optimize (for example, accuracy), and the best-performing model is selected. In Azure Machine Learning, you achieve this through an experiment that consists of a hyperdrive run, which initiates a child run for each hyperparameter combination to be tested. Each child run uses a training script with parameterized hyperparameter values to train a model, and logs the target performance metric achieved by the trained model.
- 4. Defining a search space The set of hyperparameter values tried during hyperparameter tuning is known as the search space. The definition of the range of possible values that can be chosen depends on the type of hyperparameter. Discrete hyperparameters Some hyperparameters require discrete values - in other words, you must select the value from a particular set of possibilities. You can define a search space for a discrete parameter using a choice from a list of explicit values, which you can define as a Python list (choice([10,20,30])), a range (choice(range(1,10))), or an arbitrary set of comma- separated values (choice(30,50,100))
- 5. Defining a search space Continuous hyperparameters Some hyperparameters are continuous - in other words you can use any value along a scale. To define a search space for these kinds of value, you can use any of the following distribution types: •normal •uniform •lognormal •loguniform
- 7. Configuring early termination With a sufficiently large hyperparameter search space, it could take many iterations (child runs) to try every possible combination. Typically, you set a maximum number of iterations, but this could still result in a large number of runs that don't result in a better model than a combination that has already been tried. To help prevent wasting time, you can set an early termination policy that abandons runs that are unlikely to produce a better result than previously completed runs. The policy is evaluated at an evaluation_interval you specify, based on each time the target performance metric is logged. You can also set a delay_evaluation parameter to avoid evaluating the policy until a minimum number of iterations have been completed.