Skip to content

Training models

This section describes how to train models in the AI & Analytics Engine using

  • Using the GUI
  • Using API access through SDK

Using the GUI

Creating a model in an app

Here we will focus on the model recommendation menu which wasn't described in the model creation section. As described there, after clicking the "Train new model" selection, the model recommender menu will open.

Note: The flow of training a model in the UI is in the opposite order to the flow in the API via code. While in the API via code you first utilize the model recommendation and then create the models for the app. In the UI, you first start with creating a model and then you arrive to the model recommendation. For model creation, please refer to the Creating a model in an app section.

As can be seen, it is a three-step menu. The model recommender allows you to train multiple models, but more importantly, select them in an intelligent way by using the "ratings" of the model templates. These ratings allow you to get metrics for estimated performances prior to the actual training. This has the potential to create major savings in both time and money.

In the above image we can see the first step. In this step the user selects the model required for training. For each selected model you will be shown the following ratings

  • Expected predictive performance
  • Expected training speed
  • Expected prediction speed

Thus, the flow of training is: Once the models are chosen (1) the user clicks "Next" (2) and confirms the chosen models in the next screen of (3) the dialog the user selects the "Train Model" (4) option.

After some models are chosen for training and the dialog closes, the user is taken to the "models" listing page of the application, where the models are shown to be in "Processing" stage:

Models are trained on the training set and evaluated on the test set. When models are trained in the same app, all of them use the same train-test split generated when the app was created. This ensures the models can be compared fairly and scientifically. When models have finished training on the training set and evaluated on the test set, their statuses on the dashboard and the listing page will be updated to "Ready":

Using API access through SDK

To access the API functions,you need to first authenticate into the Engine. For details, see

Import modules

import json

from aiaengine.api import app, model

Train a model

To create a model, you need to specify the id of the app where the model is included and the template id. Here template id, as well as other information on model recommendation, can be found in app. To get model information, simply run the following

get_app_response = client.apps.GetApp(
    app.GetAppRequest(id='id_of_app_model_is_created_for')
)

model_recommendation = json.loads(get_app_response.metadata)['recommendation']

From model_recommendation, you can find a list of models with the corresponding template id and performance over a range of evaluation metrics. Now you can select those models that fit your situation by specifying parameter values as follows.

client.models.CreateModel(
    model.CreateModelRequest(
        app_id='id_of_app_model_is_created_for',
        name='Model Name',
        template_id='id_of_template',
        hyperparameters='{}',   # default values
        evaluation=model.ModelEvaluation(
            metric='f1_macro',    # metric used to evaluate model
            threshold=0.9,
            min_feedback_count=0.0,
            auto_retrain=False
        )
    )
)

Once created, the model automatically starts to train.