Introduction to Sessions

Introduction to Sessions

Model Builders on the integrate.ai platform use sessions to train models, generate predictions, and perform other data science work.

The platform supports multiple session types. Regardless of the type, there are several building blocks required to create and run a session in a notebook.

Running a session

To run a session, follow the examples below:

  1. Import your token and instantiate the client (task runner).

from integrate_ai_sdk.api import connect
import os
import json
import pandas as pd

IAI_TOKEN = ""
client = connect(token=IAI_TOKEN)

  1. Specify the model_config and data_config. These parameters in these two configuration files are specific to the type of session you are running. For details, see the example of the HFL or VFL session type.

  2. Create and start (load) the session.

#Set up the task builder that contains the tasks

from integrate_ai_sdk.taskgroup.taskbuilder.integrate_ai import IntegrateAiTaskBuilder
from integrate_ai_sdk.taskgroup.base import SessionTaskGroup

iai_tb_aws = IntegrateAiTaskBuilder(client=client,task_runner_id="")

#Create the session

hfl_session = client.create_fl_session(         #This is an HFL session example. Specify the name of the session (eg hfl_session)
    name="Testing notebook - HFL",              #Type a name to appear in the workspace UI to help you locate your session
    description="I am testing HFL FFNet session creation with a task runner through a notebook",        #This is a long description that can contain details about your session
    min_num_clients=1,                          #
    num_rounds=2,                               #The number of rounds over which to train the model
    package_name="iai_ffnet",                   #The model package name
    model_config=model_config,                  #The model configuration specified in step 2
    data_config=data_schema                     #The data schema/config specified in step 2
).start()

hfl_session.id                  # Prints the training session ID for reference

  1. Create a task group and specify the dataset names. Each dataset, or train/test dataset pair, is used in a single task.

task_group = (
    SessionTaskGroup(hfl_session)
    .add_task(iai_tb_aws.hfl(
        train_dataset_name="dataset1",  //required 
        test_dataset_name="test_set1",  //required
        batch_size=batch_size,          //optional - specify the batch size
        vcpus=vcpus,                    //optional - specify to override the default vcpus for the server for this task
        memory=memory,))\               //optional - specify to override the default memory for the server for this task
    .add_task(iai_tb_aws.hfl(
        train_dataset_name="dataset2",  //required
        test_dataset_name="test_set2",  //required
        batch_size=batch_size,          //optional - specify the batch size
        vcpus=vcpus,                    //optional - specify to override the default vcpus for the server for this task
        memory=memory,))\               //optional - specify to override the default memory for the server for this task
)

task_group_context = task_group.start()             #Starting the task group starts the training session on the task runner

Each session type has a specific client task configuration. The example here shows only the minimum requirements for an HFL session. Review the details for the session type you are running when you configure your task group.

  1. Wait for the session to run and monitor its progress either through the UI or by using task_group_context.monitor_task_logs() in the notebook.

Reviewing session results

At times, you may want to review the results of a previously run session, for example to inspect the metrics.

To load a previously run session:

client.session(session.id)

where session.id is the ID of the previous session.

For more information about session types, see the following sections:

Contents