Skip to content

Hyperparameter Tuning

Once data has been suitably cleaned and transformed, it is regarded as prepared and ready for building Machine Learning models. The AI & Analytics Engine supports three main types of problems: Classification, Regression or Forecasting a Time Series. We now describe what is a Machine Learning model and how to create effective models.

Machine Learning model

A Machine Learning model can be thought of as a function or algorithm that maps the given input data to a specific output. Input data can be a row in a cleaned tabular data, such as a database record that describes a Customer with attributes of the form "Age" and "Income". To predict "credit-worthiness", we may create a regression model that outputs a numerical score. To decide if the customer is likely to churn or otherwise, we may create a classification model that outputs labels of the form "will churn" or "will not churn". If the rows of your tabular data correspond to customer transactions with attributes such as "Date of transaction" and "Amount of purchase", you can create a forecasting model that predicts the likely shopping behavior of your customer over the next few days. In each of these scenarios, a model is created by first selecting a model template and then training it with prepared data.

Model Template

A model template is essentially a Machine Learning algorithm such as a "Decision Tree" or a "Neural Network". Our platform offers a large selection of model templates for each of the problem types, i.e. Classification, Regression and Forecasting. Selection of an appropriate model template is influenced by two main metrics, namely "training time" and "predictive performance" which are described below. Note that these metrics are a function of the model template and the given dataset.

Metrics

The cost of model creation is mainly driven by the time taken to create a model, as higher training times correspond to higher rental costs on cloud resources. Quantifying the benefits of a model is not straightforward, as often there are multiple performance metrics used in the Machine Learning community, such as "Accuracy" and "F1" for Classification and "Root Mean Squared Error" and "Mean Absolute Error" for Regression. On our platform, we introduce a consistent and simplified metric known as "Predictive Performance". This is a normalized score in the range (0-100), that is used across the problem types with a common notion that a model with higher Predictive-Performance is better.

Hyperparameters

Any model templates has two types of parameters known as "Hyperparameters" and "Learnable parameters". The latter, as the name suggests, is learned directly from the training data and as such no action is required by the user. For example, a decision tree can learn an optimal split-points for partitioning a customer dataset into different groups based on income levels. However, the "maximum-depth" of the decision tree is a hyperparameter and is external to the training data. Determining the best hyperparameter setting is known as "hyperparameter tuning" and may be done with brute force search through the parameter space or via random search. On our platform, we facilitate the creation of effective models with our Machine Learning based "Model Recommender" that predicts the costs and benefits of choosing different model templates for your dataset.

Summary

A critical step post data preparation is the selection of appropriate model templates and hyperparameter tuning. A brute force approach to finding the best model and its hyperparameters involves extensive experimentation that large monetary and time costs. The AI & Analytics Engine offers a proprietary "Model Recommender" that helps foresee the costs and benefits of choosing different model templates and helps selecting the best model template for your dataset.