Version: Beta 🚧

Model Generated Features

Private Preview

This feature is currently in Private Preview.

This feature has the following limitations:

Must be enabled by Tecton Support.
Available for Rift-based Feature Views.

If you would like to participate in the preview, please file a support ticket.

Model-generated features are a powerful technique for creating high-quality context for boosting the performance of predictive or generative AI systems. A few examples of model-generated features are:

Custom embeddings: Transforming product descriptions and categories into dense vector representations, enabling more accurate recommendation systems in e-commerce.
Text classification: Performing sentiment analysis on user posts.
Image analysis: Extracting signals such as product color from images.
Named entity recognition: Identifying and categorizing named entities (e.g., person names, organizations, locations) in unstructured text data.

Tecton provides a seamless and efficient way to use custom models for context generation. This document details the process of registering a model with Tecton and using it for inference in a Batch Feature View.

Development Overview

Developing and registering a model with Tecton involves defining the model's functionality in a model file such as model.py , specifying the model metadata and artifacts in a config file such as config.py, iterating in local development mode and creating the model using Tecton's CLI.

Here are recommended steps for developing and registering a model with Tecton:

Create a new local directory for your model repo: mkdir ${MODEL_REPO_DIR}
Create a Model File, model.py in this model repo. See Model File section for details.
Create or choose an environment that your model is compatible to run with. See Environment section for details.
Create a config.py that specifies the model metadata that includes name, schemas, model/artifacts files, and environments. See Model Config section for details.
Iterate and test in local development mode. See Local Development & Testing for details.
Run tecton model create config.py 🎉

New Model CLIs

Tecton provides a suite of command line utilities to help you manage the lifecycle of a model.

tecton model -h

See CLI Docs for details.

Model File

The Model File is a Python file that contains some required and optional functions that define how the model works. This file serves as the entry point for Tecton to execute a model. Here’s an example model.py that contains a sentiment analysis model:

Sentiment Analysis Model
Custom Embedding Model
XGBoost

from transformers import AutoModelForSequenceClassification
from transformers import AutoTokenizer
import numpy as np
import torch


def preprocessor(input, context):
    tokenizer = context["tokenizer"]
    text_list = input["text"].tolist()
    encoded_input = tokenizer(text_list, return_tensors="pt", padding=True)
    return encoded_input


def postprocessor(input, context):
    scores = torch.nn.Softmax()(input[0]).cpu().numpy()
    ranking = np.argsort(scores)
    ranking = ranking[:, ::-1]
    labels = ["negative", "neutral", "positive"]
    fn = np.vectorize(lambda i: labels[i])
    result = fn(ranking[:, 0])
    return result


def load_context(data_dir, context):
    model_name = "cardiffnlp/twitter-roberta-base-sentiment"
    context["tokenizer"] = AutoTokenizer.from_pretrained(model_name)
    context["model"] = AutoModelForSequenceClassification.from_pretrained(model_name)

from torch import nn
import torch


class SimpleModule(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super().__init__()
        self.layer1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.layer2 = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        x = self.layer1(x)
        x = self.relu(x)
        x = self.layer2(x)
        return x


def load_context(data_dir, context):
    context["model"] = SimpleModule(1, 2, 3)


def preprocessor(input, context):
    x = input["info"]
    return {"x": torch.tensor(x, dtype=torch.float32).unsqueeze(1)}

import xgboost as xgb
import pandas as pd


def load_context(data_dir, context):
    model = xgb.XGBClassifier()
    model.load_model(data_dir.joinpath("iris_xgb.json").as_posix())
    context["model"] = model


def predict(inputs, context):
    df = pd.DataFrame(inputs).values
    result = context["model"].predict(df)
    return result

Here are the details of each function of model.py:

Pytorch Models

Pytorch models automatically run on GPUs and are optimized to dynamically adjust batch sizes based on available memory. The nn.Module just needs to have the forward method implemented and the table below lists the functions you could use to extend Pytorch model behavior.

function Required/Optional description

def load_context(data_dir: pathlib.Path, context: MutableMapping[str, Any]) -> None

Required

Tecton runs this function and loads the necessary context to run model inference.

data_dir is the directory that contains all artifacts you specified in the model config. You can access them by joining the data_dir and the path to the artifact relative to the root directory.
Tecton reserves "model" keyword in context for you to put your initialized Model.

def preprocessor(input: Mapping[str, numpy.ndarray], context: MutableMapping[str, Any] -> Mapping[str, torch.Tensor]

Optional

This function is the entry point to pre-process the data passed in for model inference.

The input maps the column name to a Numpy array. Column names are the same as the input_schema specified in the model config
This function is responsible for transforming the input into the correct format that the model can accept in kwargs format.

Note:

If the pre-processor is not used, input will be converted to Mapping[str, torch.Tensor] and passed as kwargs to the model inference function.
Do not change the order of input data

def postprocessor(input: Any, context:MutableMapping[str, Any]) -> numpy.ndarray

Optional

This function is the entry point for post-processing the data returned from the model inference.

The input can be any type, depending on the model.
This function is responsible for transforming the input to a Numpy array.

Note:

You need a post-processor unless your model output is an exact torch.Tensor that can be converted to the Numpy array.
Do not change the order of input data

Python Models

You can define a Python model with any arbitrary code and leverage xgboost, sklearn, lightgbm, or other libraries by creating an environment and using the functions listed below.

function	Required/Optional	description
`def load_context(data_dir: pathlib.Path, context: MutableMapping[str, Any]) -> None`	Required	Tecton runs this function and loads the necessary context to run model inference. `data_dir` is the directory that contains all artifacts you specified in the model config. You can access them by joining the `data_dir` and the path to the artifact relative to the root directory. Tecton reserves `"model"` keyword in context for you to put your initialized Model.
`def predict(input: Mapping[str, numpy.ndarray], context: MutableMapping[str, Any]) -> numpy.ndarray`	Required	This function is the entry point function to execute your prediction. Should return an array matching the input size. Do not change the order of input data

Python models do not use the GPU, unless explicitly specified by the user in the load_context function.

caution

Do not change the order of input data in the pre or post processor, which will cause incorrect results.

Environment

You need to choose or create an environment that your model can run with at materialization time. The environment you choose should contain all third party packages your model replies on.

Tecton provides a managed environment, tecton-rift-ml-1.0.0, using SDK 1.0. You can leverage this environment directly if its dependencies meet the requirements of your model.

Resolved Dependencies for tecton-rift-ml-1.0.0

tecton==1.0.0
transformers==4.42.0
attrs==24.2.0
boto3==1.35.20
click==8.1.7
colorama==0.4.6
deltalake==0.18.2
duckdb==1.0.0
google-cloud-bigquery-storage==2.26.0
google-cloud-bigquery==3.25.0
google-cloud-storage==2.18.2
jinja2==3.1.4
numpy==1.24.4
pandas==2.0.3
pathspec==0.12.1
pendulum==2.1.2
pex==2.19.1
pip==24.2
protobuf==4.25.3
pyarrow==15.0.2
pydantic==1.10.18
pygments==2.18.0
pypika==0.48.9
pytest==8.3.3
pytimeparse==1.1.8
pyyaml==6.0.2
requests==2.32.3
semantic-version==2.10.0
setuptools==69.5.1
snowflake-connector-python==3.12.2
snowflake-snowpark-python==1.22.1
sqlparse==0.5.1
texttable==1.7.0
torch==2.4.1
tqdm==4.66.5
typeguard==2.13.3
typing-extensions==4.12.2
urllib3==1.26.20
yaspin==2.5.0
filelock==3.16.1
huggingface-hub==0.25.0
packaging==24.1
regex==2024.9.11
safetensors==0.4.5
tokenizers==0.19.1
botocore==1.35.20
jmespath==1.0.1
s3transfer==0.10.2
pyarrow-hotfix==0.6
google-api-core==2.19.2
google-auth==2.34.0
proto-plus==1.24.0
db-dtypes==1.3.0
google-cloud-core==2.4.1
google-resumable-media==2.7.2
python-dateutil==2.9.0.post0
google-crc32c==1.5.0
markupsafe==2.1.5
pytz==2024.2
tzdata==2024.1
pytzdata==2020.1
exceptiongroup==1.2.2
iniconfig==2.0.0
pluggy==1.5.0
tomli==2.0.1
certifi==2024.8.30
charset-normalizer==3.3.2
idna==3.10
asn1crypto==1.5.1
cffi==1.17.1
cryptography==43.0.1
platformdirs==4.3.5
pyopenssl==24.2.1
pyjwt==2.9.0
sortedcontainers==2.4.0
tomlkit==0.13.2
cloudpickle==2.2.1
wheel==0.44.0
fsspec==2024.9.0
networkx==3.1
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.20.5
nvidia-nvtx-cu12==12.1.105
sympy==1.13.2
triton==3.0.0
termcolor==2.4.0
googleapis-common-protos==1.65.0
grpcio-status==1.62.3
grpcio==1.66.1
cachetools==5.5.0
pyasn1-modules==0.4.1
rsa==4.9
six==1.16.0
pycparser==2.22
nvidia-nvjitlink-cu12==12.6.68
mpmath==1.3.0
pyasn1==0.6.1

If it doesn't meet the requirement, a custom environment can be created by using:

$ tecton environment create --name "my-custom-env-0.1" --description "My Custom Env 0.1" --requirements /path/to/requirements.txt

The requirements.txt should list the necessary pip packages for your model and tecton[rift-materialization]. See the environment page for more details.

Model Config

Model Config defines all the necessary metadata for a model and is used to register the model through tecton model create path/to/config.py. See SDK Doc for details.

The supported data types for both input_schema and output_schema include Int32, Int64, Float32, Float64, String, and arrays of these types. See the data type page for more details.

Example

Example #1: Sentiment Analysis
Example #2: Custom Embeddings
Example #3: XGBoost

from tecton import ModelConfig
from tecton.types import Field, String

model = ModelConfig(
    name="your_model_name",  # Replace "your_model_name" with the actual model name
    model_type="pytorch",
    model_file="model.py",
    input_schema=[Field("text", String)],
    output_schema=Field("sentiment", String),
    environments=["tecton-rift-ml-1.0.0"],
)

from tecton import ModelConfig
from tecton.types import Field, Int64, Array, Float32

model = ModelConfig(
    name="your_model_name",  # Replace "your_model_name" with the actual model name
    model_type="pytorch",
    model_file="model.py",
    input_schema=[Field("info", Int64)],
    output_schema=Field("embeddings", Array(Float32)),
    environments=["tecton-rift-ml-1.0.0"],
)

from tecton import ModelConfig
from tecton.types import Field, Int32, Float32

model = ModelConfig(
    name="your_model_name",  # Replace "your_model_name" with the actual model name
    model_type="python",
    model_file="model.py",
    artifact_files=["iris_xgb.json"],
    input_schema=[Field("slen", Float32), Field("swid", Float32), Field("plen", Float32), Field("pwid", Float32)],
    output_schema=Field("iris", Int32),
    environments=["custom-xgboost-env"],
)

`Inference` Feature in a Batch Feature View

The Inference feature type is used to invoke Tecton-registered models to compute feature values.

See Inference SDK Doc for details about this feature.

Here is an example:

from tecton import pandas_batch_config, Entity, BatchSource, batch_feature_view, Attribute, Inference, RiftBatchConfig
from tecton.types import Field, Int64, String
from datetime import timedelta, datetime

# Define the entity
entity = Entity(name="user_id", join_keys=[Field("user_id", String)])

# Define a data source
@pandas_batch_config(supports_time_filtering=False)
def chat_history():
    import pandas

    df = pandas.DataFrame(
        columns=["user_id", "timestamp", "text"],
        data=[
            ["user_1", "2024-05-14T00:00:00", "thank you so much!"],
            ["user_1", "2024-05-15T00:00:00", "I am very disappointed."],
            ["user_1", "2024-05-16T00:00:00", "okay"],
        ],
    ).astype({"timestamp": "datetime64[us]"})
    return df


chat_history_ds = BatchSource(name="chat_history", batch_config=chat_history)


@batch_feature_view(
    name="sentiment_bfv",
    mode="pandas",
    sources=[chat_history_ds],
    entities=[entity],
    batch_schedule=timedelta(days=1),
    feature_start_time=datetime(2024, 5, 1),
    timestamp_field="timestamp",
    features=[
        Attribute("text", String),
        Inference(
            input_columns=[
                Field("text", String),
            ],
            model="roberta-sentiment-v0",
            name="user_sentiment",
        ),
    ],
    # Turning off `online` and `offline` parameter can skip the `environment` check.
    online=True,
    offline=True,
    run_transformation_validation=False,
    environment="tecton-rift-ml-1.0.0",
    batch_compute=RiftBatchConfig(
        # NOTE: we recommend using L4 GPU instances for Embeddings inference
        instance_type="g6.xlarge",
    ),
)
def bfv(input_table):
    return input_table

Local Development & Testing

To locally test and iterate on your model, you can define a ModelConfig object in a notebook and use model.run() to verify the output.

Use the Model Config and Model File examples below to set up your model locally in a notebook.

Model Config
Model File

from tecton import ModelConfig
from tecton.types import Field, String

model = ModelConfig(
    name="roberta-sentiment-v0",
    model_type="pytorch",
    model_file="model.py",
    input_schema=[Field("text", String)],
    output_schema=Field("sentiment", String),
    environments=[],  # `environments` is not required in local development.
)

# Inspect model inference results.
df = model.run(
    {
        "text": ["I am excited", "I am disappointed"],
    }
)

from transformers import AutoModelForSequenceClassification
from transformers import AutoTokenizer
import numpy as np
from scipy.special import softmax


def preprocessor(input, context):
    tokenizer = context["tokenizer"]
    text_list = input["text"].tolist()
    encoded_input = tokenizer(text_list, return_tensors="pt", padding=True)
    return encoded_input


def postprocessor(input, context):
    scores = torch.nn.Softmax()(input[0]).cpu().numpy()
    ranking = np.argsort(scores)
    ranking = ranking[:, ::-1]
    labels = ["negative", "neutral", "positive"]
    fn = np.vectorize(lambda i: labels[i])
    result = fn(ranking[:, 0])
    return result


def load_context(data_dir, context):
    model_name = "cardiffnlp/twitter-roberta-base-sentiment"
    context["tokenizer"] = AutoTokenizer.from_pretrained(model_name)
    context["model"] = AutoModelForSequenceClassification.from_pretrained(model_name)

After verifying the model, you have two options to refer to the model in your local Feature View.

Register the model via the command line (tecton model create config.py) and then refer to it in a local Batch Feature View.
Call model.register() to temporarily register the model in your local notebook session.

from tecton import ModelConfig
from tecton.types import Field, String

model = ModelConfig(
    name="roberta-sentiment-v0",
    model_type="pytorch",
    model_file="model.py",
    input_schema=[Field("text", String)],
    output_schema=Field("sentiment", String),
    environments=[],
)

model.register()

caution

If a local model name conflicts with a remote model, the local model will be used, and you will see a warning when registering it locally.

After registering the model with either way, the Batch Feature View can be developed locally as normal. The Batch Feature View example defined above can be used directly with the local model. Turning off online and offline parameter can skip the environment check in local development.

Limitations

This capability is currently limited to Batch Feature Views. Support for Realtime and Stream Feature Views is coming soon.

Development Overview​

New Model CLIs​

Model File​

Pytorch Models​

Python Models​

Environment​

Model Config​

Example​

Inference Feature in a Batch Feature View​

Local Development & Testing​

Limitations​

Was this page helpful?