Skip to main content
Version: Beta ๐Ÿšง

๐Ÿš€ Tecton Quickstart

Open In Colab

Tecton helps you build and productionize AI applications by making it easy to define, test, and deploy features for model training and serving.

Let's take a quick look at how you can build a low-latency streaming feature for a fraud detection use case using nothing but Python.

For an end-to-end tutorial that takes you from raw data to a real-time model, check out Building a Production AI Application with Tecton.

note

This tutorial is designed to work with our Interactive Demo, which you can sign up for at explore.tecton.ai.

โš™๏ธ Install Tectonโ€‹

pip install 'tecton[rift]==0.9.0'

โœ… Log in to Tectonโ€‹

Make sure to hit enter after pasting in your authentication token.

import tecton

tecton.login("explore.tecton.ai")
tecton.set_validation_mode("auto")

๐Ÿงช Define and test a streaming featureโ€‹

Using Tecton's Feature Engineering Framework we will define 3 new features for our fraud detection model:

  • A user's total transaction amount in the last 1 minute
  • A user's total transaction amount in the last 1 hour
  • A user's total transaction amount in the last 30 days

To do so, we will first define a Stream Source which tells Tecton where to retrieve events online and offline. For the online path, we've configured Tecton to accept real-time events via an HTTP ingestion API which we will try sending records to in the next step. For the offline path, we've pointed Tecton to an S3 path that contains a historical record of our stream. Tecton uses this path for offline development and backfills.

Next we define a Stream Feature View which can create one or more features via transformations against the Stream Source. In a Stream Feature View, we can run Python transformations on incoming records, and optionally apply time-windowed aggregations via the aggregations parameter. These transformations run identically online and offline which is critical for preventing skew.

Lastly, we can immediately test our features directly in our notebook via the get_features_in_range method. Try it out!

from tecton import *
from tecton.types import *
from datetime import datetime, timedelta


transactions_stream = StreamSource(
name="transactions_stream",
stream_config=PushConfig(),
batch_config=FileConfig(
uri="s3://tecton.ai.public/tutorials/transactions.pq",
file_format="parquet",
timestamp_field="timestamp",
),
schema=[Field("user_id", String), Field("timestamp", Timestamp), Field("amount", Float64)],
)


user = Entity(name="user", join_keys=["user_id"])


@stream_feature_view(
source=transactions_stream,
entities=[user],
mode="pandas",
aggregations=[
Aggregation(function="sum", column="amount", time_window=timedelta(minutes=1)),
Aggregation(function="sum", column="amount", time_window=timedelta(hours=1)),
Aggregation(function="sum", column="amount", time_window=timedelta(days=30)),
],
schema=[Field("user_id", String), Field("timestamp", Timestamp), Field("amount", Float64)],
)
def user_transaction_amount_totals(transactions_stream):
return transactions_stream[["user_id", "timestamp", "amount"]]


# Test the feature locally using historical data
df = (
user_transaction_amount_totals.get_features_in_range(start_time=datetime(2022, 1, 1), end_time=datetime(2022, 2, 1))
.to_pandas()
.fillna(0)
)
df.head(5)
user_idamount_sum_1m_continuousamount_sum_1h_continuousamount_sum_30d_continuous_valid_to_valid_from
0user_221088738404.553429.122022-01-22 00:00:002022-01-21 00:00:00
1user_241716460001.972048.182022-01-10 00:00:002022-01-09 00:00:00
2user_9757807451098.37123652022-01-07 00:00:002022-01-06 00:00:00
3user_1939957235003616.42022-01-20 00:00:002022-01-19 00:00:00
4user_1990251765007721.292022-01-24 00:00:002022-01-23 00:00:00

โšก๏ธ Ingest data and retrieve updated feature values in real-timeโ€‹

Tecton objects get registered via a declarative workflow. Features are defined as code in a repo and applied to a workspace (like a project) in a Tecton account using the Tecton CLI. This declarative workflow enables productionization best practices such as "features as code," CI/CD, and unit testing.

We've gone ahead and registered these features so you can try ingesting new events and querying for online features. After features get registered with a workspace, Tecton handles the backfilling and ongoing materialization of data to the offline and online store for training and serving.

This step requires generating an API key, which you can do here. Copy the generated key and paste it in below.

import random, string

tecton.set_credentials(tecton_api_key="your-api-key") # replace with your API key

# Fetch registered Tecton objects
ws = tecton.get_workspace("prod")
ds = ws.get_data_source("transactions_stream")
fv = ws.get_feature_view("user_transaction_amount_totals")

# Generate a random user_id for the next step
user_id = "user_" + "".join(random.choices(string.digits, k=7))
print("Generated random user id: " + user_id)

โœ… Successfully set credentials. Generated random user id: user_7370526

๐Ÿ”ฅ Run this repeatedly and watch the features update!โ€‹

Note: It may take a few minutes for your API key permissions to update.

# Ingest a new transaction with any amount you want
record = ds.ingest({"user_id": user_id, "timestamp": datetime.utcnow(), "amount": 100})
print(f"Ingested record for '{user_id}':")
print(record)

# Read updated features via Tecton's HTTP API
features = fv.get_online_features(join_keys={"user_id": user_id}).to_dict()
print(f"\nUpdated features for '{user_id}':")
print(features)

Ingested record for 'user_7370526': {'workspaceName': 'prod', 'ingestMetrics': {'featureViewIngestMetrics': [{'featureViewName': 'user_transaction_amount_totals', 'onlineRecordIngestCount': '1', 'featureViewId': 'c3c076e11740b141097d5f864bfceb1f'}]}}

Updated features for 'user_7370526': {'amount_sum_1h_continuous': 100.0, 'amount_sum_1m_continuous': 100.0, 'amount_sum_30d_continuous': 100.0}

โญ๏ธ Conclusionโ€‹

Now that you've seen how easy it is to build real-time features with Tecton, check out Building a Production AI Application to see how you can productionize an end-to-end online AI Application in just 30 minutes.

If you want to productionize your own features with your own data, you can sign up for a free trial at tecton.ai/free-trial.

Was this page helpful?

๐Ÿง  Hi! Ask me anything about Tecton!

Floating button icon