Skip to content

Materialization

Feature Views define features as a view on top of of your underlying data sources. Materializing your Feature Views with Tecton will save computed feature values in Tecton's online and offline stores, such that you can have fast access to the feature values during training and inference.

Overview

Materialization runs are processing jobs that precompute the queries defined by a Feature View, then store the result. This enables efficient retrieval of Feature values at lookup time. Tecton stores Materialized data in two Tecton-managed locations: the Online Feature Store and the Offline Feature Store.

The Online Feature Store is a key-value store that contains the most recent version of feature data. This data is used to serve features to prediction consumers at low-latency.

The Offline Feature Store contains past and present feature values of all feature data. This data will be used for batch processes, such as creating training data sets from historical data.

Note that only Feature Views based on batch or stream data sources can be materialized. On-Demand Feature Views cannot be materialized, since they are calculated only at request-time.

Materialization Operations

Tecton seamlessly handles backfill and steady-state materialization for all batch and stream features based on your feature view configuration.

Steady-state

Steady-state Materialization refers to materialization being performed on new data arriving in real-time. Steady State Materialization continuously occurs on all FeatureViews where Materialization is enabled.

When a FeatureView has materialization enabled, Tecton will schedule steady-state materialization jobs on an ongoing basis in order to maintain fresh feature values. The frequency of steady-state materialization is controlled by the batch_schedule parameter.

Backfills

Backfill refers to any materialization operations performed on data in the past. There are two Backfill operations.

Bootstrap

The initial materialization of a Feature View is referred to as a bootstrap. During a botstrap materialization, existing raw data is processed into feature values.

When materialization is initially enabled for a Feature View, Tecton performs a bootstrap materialization. The amount of data materialized during a bootstrap is controlled by the feature_start_time parameter.

Overwrite

You can recalculate Materialized data for a range of timestamps that are later found to have an error in the data source. This is available at either at the Feature View level or the Data Source level using an Overwrite operation.

Overwrite backfills must be run manually by Tecton support. To perform an overwrite backfill, contact support@tecton.ai, which will manually process the request.

Enabling Materialization for a Feature View

When creating a new feature, we typically start without materialization until we're ready to consume the feature in a model.

In order to turn on materialization to the online or offline store, set online=True and offline=True in the Feature View annotation parameters. These options are available for the following types of Feature Views:

Monitoring

Tecton provides tools to monitor and debug production Feature Views. The web UI provides Materialization overviews, the SDK contains specifics about the Feature View class, and the CLI makes cluster materialization overviews available. More information on monitoring is available in Monitoring Materializations.