Feature Server Groups

Private Preview

This feature is currently in Private Preview.

This feature has the following limitations:

• Must be enabled by Tecton support.

If you would like to participate in the preview, please file a support ticket.

Introduction

The Feature Server is a Tecton-managed compute node that returns the feature vectors of a set of Feature Services via an HTTP API. A Feature Server Group is a group of Feature Server nodes that are capable of autoscaling and are associated with a particular workspace.

A Feature Server Group is “isolated” at the workspace level, meaning that it can only serve traffic from a single workspace’s Feature Services.

Isolation of Feature Server Groups provides the following benefits:

• Eliminates cross-workspace disruption caused by shared serving infrastructure, preventing resource contention between one team's test cases and another's production traffic. Feature Server Groups isolation ensures that each team's operations remain independent and performant.
• Facilitates granular resource and cost management through workspace-level server provisioning controls.
• Provides the ability to do better cost attribution of online serving costs.

Backwards compatibility with the Feature Server infrastructure:

• Feature Server Groups have an independent infrastructure from the non-isolated Feature Server infrastructure.
• It is safe to have both Feature Server Groups and the Feature Server running at the same time.
• The serving API v1 remains intact and can be continued to serve traffic; Feature Server Groups use serving API v2.

Limitations

• Currently, workspaces are limited to one Feature Server Group each. Support for multiple Feature Server Groups per workspace is coming soon.
• There is a maximum of 100 Feature Server Groups allowed per cluster per region.
• Each Feature Server Group has a default quota of 50 nodes. If you require a higher quota, please reach out to support for assistance!

Creating, Updating, and Deleting Feature Server Groups

This section describes how to create, update, and delete Feature Server Groups in a given workspace.

Create

1. Select the desired live workspace:

% tecton workspace select <WORKSPACE_NAME>

2. Add a Feature Server Group declaration:

with Autoscaling enabled:

from tecton import FeatureServerGroup
from tecton.framework import configs

fraud_group = FeatureServerGroup(
    name="fraud-server-group",
    scaling_config=configs.AutoscalingConfig(min_nodes=1, max_nodes=2),
    owner="fraud-detection",
    description="Fraud detection team Feature Server Group",
)

with Autoscaling disabled:

from tecton import FeatureServerGroup
from tecton.framework import configs

fraud_group = FeatureServerGroup(
    name="fraud-server-group",
    scaling_config=configs.ProvisionedScalingConfig(desired_nodes=1),
    owner="recsys",
    description="Recommendation Systems team Feature Server Group",
)

info

Single Feature Server Group in a workspace: If multiple Feature Server Groups are defined in a workspace, the tecton apply command will fail, and neither group will be created. This is a temporary limitation and we plan to remove this limitation soon.

info

Default to legacy Feature Servers: If no Feature Server Group is defined in the workspace, you can still use the legacy Feature Server endpoints (https://<cluster>.tecton.ai/api/v1/get-features) to serve your production traffic.

3. Reference the group in the feature service and apply the declarative config to the workspace:

fraud_detection_feature_service = FeatureService(
    name="fraud_detection_feature_service",
    online_serving_enabled=True,
    feature_server_group=fraud_group,
    features=[fuzzy_similarity],
)

% tecton apply

4. Wait for the Feature Server Group to be in READY status. It typically takes 5 to 10 minutes for the infrastructure to become fully operational and capable of serving traffic. Check the status of the server group with the tecton CLI command:

% tecton server-group list

               Id                         Name                  Type            Status    Environment                  Description                        Created At               Owner        Last Modified By
================================================================================================================================================================================================================
a23747baadc9a4cdf0c7378d3c461c07   fraud-server-group   FEATURE_SERVER_GROUP   PENDING        N/A       Fraud detection team Feature Server Group   2024-10-19 00:10:51 UTC   fraud-detection    jon@tecton.ai

5. Once the group reaches the READY status, refer to the Querying Feature Server Groups section for instructions on calling the available APIs.

Update

1. Select the desired live workspace:

tecton workspace select <WORKSPACE_NAME>

2. Update the Feature Server Group declaration in the declarative config to set the needed group scaling params:

with Autoscaling enabled:

from tecton import FeatureServerGroup
from tecton.framework import configs

fraud_group = FeatureServerGroup(
    name="fraud-server-group",
    scaling_config=configs.AutoscalingConfig(min_nodes=3, max_nodes=4),
    owner="fraud-detection",
    description="Fraud detection team Feature Server Group",
)

with Autoscaling disabled:

from tecton import FeatureServerGroup
from tecton.framework import configs

fraud_group = FeatureServerGroup(
    name="recsys-server-group",
    scaling_config=configs.ProvisionedScalingConfig(desired_nodes=3),
    owner="recsys",
    description="Recommendation Systems team Feature Server Group",
)

3. Apply the declarative config to the workspace:

% tecton apply

4. Since this update modifies the scaling parameters, there is no need to wait for the update to finish before calling the serving API. However, if a user wants to ensure that they have enough capacity before calling the API (or changing the pattern of how they call the API) they need to wait for the Feature Server Group's “Current Configuration” to show the same values as the “Target Configuration” (i.e. for the change to be applied). This can be done with the tecton server-group describe CLI command:

% tecton server-group describe -n fraud-server-group
...
======================
SCALING CONFIGURATIONS
======================
+---------------------+-------------------------+-------------------------+
|        Field        |  Target Configuration   |  Active Configuration   |
+=====================+=========================+=========================+
| Min Nodes           | 3                       | 3                       |
+---------------------+-------------------------+-------------------------+
| Max Nodes           | 4                       | 4                       |
+---------------------+-------------------------+-------------------------+
| Desired Nodes       | N/A                     | N/A                     |
+---------------------+-------------------------+-------------------------+
| Autoscaling Enabled | True                    | True                    |
+---------------------+-------------------------+-------------------------+
| Last Updated At     | 2024-09-09 03:13:47 UTC | 2024-09-09 03:16:38 UTC |
+---------------------+-------------------------+-------------------------+

Delete

1. Select the desired live workspace:

% tecton workspace select <WORKSPACE_NAME>

2. Remove (or comment out) the declaration of the Feature Server Group in the declarative config:

# from tecton import FeatureServerGroup
# from tecton.framework import configs
#
# fraud_group = FeatureServerGroup(
#     name="fraud-server-group",
#     scaling_config=configs.AutoscalingConfig(min_nodes=3, max_nodes=4),
#     owner="fraud-detection",
#     description="Fraud detection team Feature Server Group",
# )

3. Apply the declarative config to the workspace:

% tecton apply

4. No further actions are needed, as soon as the Feature Server Group is deleted it won’t be shown in the list of the groups provided by the tecton server-group list CLI command. For a couple of minutes the group will be present in the list in the DELETING status:

% tecton server-group list


               Id                         Name                  Type            Status    Environment                  Description                        Created At               Owner        Last Modified By
================================================================================================================================================================================================================
a23747baadc9a4cdf0c7378d3c461c07   fraud-server-group   FEATURE_SERVER_GROUP   DELETING        N/A       Fraud detection team Feature Server Group   2024-10-19 00:10:51 UTC   fraud-detection    jon@tecton.ai

Querying Feature Server Groups

Feature Services

Single key

This API retrieves feature values for a Feature Service from Tecton's online store.

% curl -X POST -d '{...}' https://serving.<region>.<cluster-name>.tecton.ai/api/v2/workspaces/{workspace_name}/feature-services/{feature_service_name}/get-features

Request:

{
  "params": {
    "joinKeyMap": {
      "user_id": "A123",
      "ad_id": "5417"
    },
    "requestContextMap": {
      "amount": 500
    },
    "metadataOptions": {
      "includeSloInfo": true,
      "includeEffectiveTimes": true,
      "includeNames": true,
      "includeDataTypes": true,
      "includeServingStatus": true
    }
  }
}

Response:

{
  "result": {
    "features": [
      true,
      28941.299999999977,
      842.8599999999999,
      90946.93999999996,
      [
        "892054b9598370dce846bb6e4b5805a1",
        "cc9f13814a736160984bc9896222e4d9",
        "43a9799c961de6ebb22c122d8c7eb340"
      ]
    ]
  },
  "metadata": {
    "features": [
      {
        "name": "transaction_amount_is_higher_than_average.transaction_amount_is_higher_than_average",
        "dataType": {
          "type": "boolean"
        },
        "status": "PRESENT"
      },
      {
        "name": "user_transaction_amount_metrics.amt_sum_1d_10m",
        "effectiveTime": "2023-05-04T15:50:00Z",
        "dataType": {
          "type": "float64"
        },
        "status": "PRESENT"
      },
      {
        "name": "user_transaction_amount_metrics.amt_sum_1h_10m",
        "effectiveTime": "2023-05-04T15:50:00Z",
        "dataType": {
          "type": "float64"
        },
        "status": "PRESENT"
      },
      {
        "name": "user_transaction_amount_metrics.amt_sum_3d_10m",
        "effectiveTime": "2023-05-04T15:50:00Z",
        "dataType": {
          "type": "float64"
        },
        "status": "PRESENT"
      },
      {
        "name": "user_transaction_counts.transaction_id_last_3_1d_1d",
        "effectiveTime": "2023-05-03T00:00:00Z",
        "dataType": {
          "type": "array",
          "elementType": {
            "type": "string"
          }
        },
        "status": "PRESENT"
      }
    ],
    "sloInfo": {
      "sloEligible": true,
      "sloServerTimeSeconds": 0.015835683,
      "dynamodbResponseSizeBytes": 23722,
      "serverTimeSeconds": 0.016889888,
      "storeMaxLatency": 0.02687345,
      "storeResponseSizeBytes": 23722
    }
  }
}

Many keys

This API retrieves feature values in batch for a Feature Service from Tecton's online store. This endpoint is available only for Feature Services with REDIS Feature Views.

% curl -X POST -d '{...}' https://serving.<region>.<cluster-name>.tecton.ai/api/v2/workspaces/{workspace_name}/feature-services/{feature_service_name}/get-features-batch

Request:

{
  "params": {
    "requestData": [
      {
        "joinKeyMap": {
          "user_uuid": "6c423390-9a64-52c8-9bb3-bbb108c74198"
        },
        "requestContextMap": {
          "amount": 2000
        }
      },
      {
        "joinKeyMap": {
          "user_uuid": "8c423390-9a64-52c8-9bb3-bbb108c74197"
        },
        "requestContextMap": {
          "amount": 500
        }
      }
    ],
    "metadataOptions": {
      "includeSloInfo": true,
      "includeEffectiveTimes": true,
      "includeNames": true,
      "includeDataTypes": true,
      "includeServingStatus": true
    }
  }
}

Response:

{
  "result": [
    {
      "features": ["37", "3", "11"]
    },
    {
      "features": ["43", "128", "254"]
    }
  ],
  "metadata": {
    "features": [
      {
        "name": "user_click_counts.clicked_count_1d_1h",
        "effectiveTime": "2023-05-26T18:00:00Z",
        "dataType": {
          "type": "int64"
        },
        "status": ["PRESENT", "PRESENT"]
      },
      {
        "name": "user_click_counts.clicked_count_3d_1h",
        "effectiveTime": "2023-05-26T18:00:00Z",
        "dataType": {
          "type": "int64"
        },
        "status": ["PRESENT", "PRESENT"]
      },
      {
        "name": "user_distinct_ad_count_7d.distinct_ad_count",
        "effectiveTime": "2023-05-26T00:00:00Z",
        "dataType": {
          "type": "int64"
        },
        "status": ["PRESENT", "PRESENT"]
      }
    ],
    "sloInfo": [
      {
        "sloEligible": true,
        "sloServerTimeSeconds": 0.001231726,
        "dynamodbResponseSizeBytes": 1873,
        "serverTimeSeconds": 0.001231726,
        "storeMaxLatency": 0.000877174,
        "storeResponseSizeBytes": 1873
      },
      {
        "sloEligible": true,
        "sloServerTimeSeconds": 0.001329688,
        "dynamodbResponseSizeBytes": 1937,
        "serverTimeSeconds": 0.001329688,
        "storeMaxLatency": 0.000791104,
        "storeResponseSizeBytes": 1937
      }
    ],
    "batchSloInfo": {
      "sloEligible": true,
      "sloServerTimeSeconds": 0.001329688,
      "serverTimeSeconds": 0.00181146,
      "storeMaxLatency": 0.000877174
    }
  }
}

Metadata

This API retrieves the metadata of a Feature Service, including the schema for join keys and request context.

curl -X GET https://serving.<region>.<cluster-name>.tecton.ai/api/v2/workspaces/{workspace_name}/feature-services/{feature_service_name}/metadata

Response:

{
  "featureServiceType": "DEFAULT",
  "inputJoinKeys": [
    {
      "name": "user_uuid",
      "dataType": {
        "type": "string"
      }
    }
  ],
  "inputRequestContextKeys": [
    {
      "name": "amt",
      "dataType": {
        "type": "float64"
      }
    }
  ],
  "featureValues": [
    {
      "name": "transaction_amount_is_higher_than_average.transaction_amount_is_higher_than_average",
      "dataType": {
        "type": "boolean"
      }
    },
    {
      "name": "user_transaction_amount_metrics.amt_mean_1d_10m",
      "dataType": {
        "type": "float64"
      }
    },
    {
      "name": "user_transaction_amount_metrics.amt_mean_3d_10m",
      "dataType": {
        "type": "float64"
      }
    },
    {
      "name": "user_transaction_counts.transaction_id_last_3_1d_1d",
      "dataType": {
        "type": "array",
        "elementType": {
          "type": "string"
        }
      }
    }
  ]
}