Skip to content

Obtaining Evaluation Results

Import necessary modules

Then you need to import the following modules that are used in the next steps.

import json

from aiaengine.api import model

Obtain results

Now you can retrieve information of models. For those that are trained successfully, you can access the results of evaluation by the following

get_model_response = client.models.GetModel(
    model.GetModelRequest(
        id='id_of_model_to_explore'
    )
)

model_evaluation = json.loads(get_model_response.last_success_training.result)

The model_evaluation variable keeps the results over a range of evaluated metrics. The following is an example for classification models.

>>> model_evaluation['evaluation_scores']

{
  'accuracy': 0.8530734632683659,
  'average_precision': 0.6682887654106783,
  'f1_micro': 0.8530734632683659,
  'f1_macro': 0.739573077876545,
  'f1_weighted': 0.8413883365361269,
  'precision_micro': 0.8530734632683659,
  'precision_macro': 0.7926047443268806,
  'precision_weighted': 0.841784862340374,
  'recall_micro': 0.8530734632683659,
  'recall_macro': 0.7117236254200362,
  'recall_weighted': 0.8530734632683659,
  'roc_auc': 0.8463913814113215,
  'prediction_time': 2.3829942938686906e-10,
  'prediction_credits': 2.647771437631878e-12,
  'train_time': 16.588364839553833,
  'train_credits': 0.18431516488393146,
  'predictive_performance': 73.9573077876545,
  'f1_macro_rating': 3.9969590329172573,
  'train_time_rating': 4.228687385582896,
  'prediction_time_rating': 4.678588193150658,
  'predictive_performance_rating': 3.9969590329172573
}