Hyperparameter tuning looks for the combination that gives the best cross-validated performance: grid search, random search, or more economical methods. It is done on the training data only, the test set being reserved for the final evaluation of a single chosen model.