Skip to content

Training models

This section describes how to train models in the AI & Analytics Engine using the GUI, and using API access through SDK.

When models are trained in the same app, all of them use the same train-test split generated when the app was created. This ensures the models can be compared fairly and scientifically.

Note: The flow of training a model in the UI is in the opposite order to the flow in the API via code. While in the API via code you first utilize the model recommendation and then create the models for the app. In the UI, you first start with creating a model and then you arrive to the model recommendation.

Using the GUI

Create a Model

From the dashboard after selecting an app, click "New Model" to create a model.

Alternatively, navigate to the "Models" tab in an app and click on the floating actions button at the bottom right, followed by "Train New Model".

Get Model Performance Predictions

In the "New Model" dialog, first select a feature set on which you wish to train a model. Click here for how to create a feature set. Click next and you will be shown a list of model templates (machine learning algorithms) to choose from. In this view, the output of the model recommender is shown to you, to aid you in choosing the model template that suits your needs. The recommender has estimated the performance of each of these templates on your data in terms of:

  1. The "predictive performance", a percentage value that measures how close your model's predictions will be to the true values, on new data that was not available when the model was trained. A 100% predictive performance indicates that the model is able to predict without any errors,
  2. The time taken to train the model, and
  3. The time taken by the model to predict on new data.

The "Choose a model" step of the dialog shows this information in a visual manner, as seen below:

On this screen you can:

  1. Sort the models with respect to each of the three metrics estimated by the recommender, from best to worst.
  2. See the details of the model template.
  3. See for every model the ratings out of five for each of the three estimated metrics. A higher rating implies:
    1. Better predictions for the "predictive performance" metric,
    2. Faster training for the "train time" metric, and
    3. Faster predictions, for the "prediction time" metric.
  4. View a bubble plot showing estimated training time on the x-axis, estimated prediction time on the y-axis, and the estimated predictive performance as text inside the bubbles.
  5. Read a description of the selected model

If you have previously trained models for this application, model templates will be flagged as "trained". Click on a model to view its predicted performance and its description. You can also select different models via the arrows on either side of the predicted performance plot:

To choose a model for training, click the checkbox next to the model's name. Models selected for training will appear in the predicted performance plot. Once you are satisfied with your choices, click "Next" to configure the models' hyperparameters.

Configure Model Hyperparameters

Here, you can (1) choose to use a default configuration of hyperparameters for your models, or (2) choose custom hyperparameters. Clicking on (2) will list each model along with a text box containing a dict of hyperparameters for the model. Modify the values as you see fit, then click "Next" to proceed.

The final tab allows you to review the models you have chosen along with their hyperparameters before finally training them. Click "Back" to go back and change things if needed. If satisfied, click "Train Models" to begin the training job.

You will then be taken to the models listing page of your App. On this screen, you can see the model templates' names and progress bars for the training jobs:

Using API access through SDK

To access the API functions,you need to first authenticate into the Engine by

from aiaengine import api

client = api.Client()

Import modules

import json

from aiaengine import util
from aiaengine.api import featureset, model

Train a model

To create a model, you need to specify the app id, the id of the feature set on which you want to train the model, and the template id. Here template id, as well as other information on model recommendation, can be found in featureset. To get model recommendation, simply run the following

get_feature_set_response = client.featuresets.GetFeatureSet(
    featureset.GetFeatureSetRequest(
        id='id_of_feature_set'
    )
)

model_recommendation = json.loads(get_feature_set_response.recommendations)['recommendation']

From model_recommendation, you can find a list of models with the corresponding template id and performance over a range of evaluation metrics. Now you can select those models that fit your situation by specifying parameter values as follows.

client.models.CreateModel(
    model.CreateModelRequest(
        app_id='id_of_app',
        feature_set_id='id_of_feature_set',
        name='Model Name',
        template_id='id_of_template',
        hyperparameters='{}',   # default values
        evaluation=model.ModelEvaluation(
            metric='f1_macro',    # metric used for hyperparameter tuning
            threshold=0.9,
            min_feedback_count=0.0        
        )
    )
)

Once created, the model automatically starts to train.