Skip to content

Materialization

Materialization is the process of computing values from FeaturePackage definitions and making the resulting data available for queries to the feature store.

Overview

Materialization runs are processing jobs that precompute the queries defined by a Feature Package, then store the result. This enables efficient retrieval of Feature values at lookup time. Tecton stores Materialized data in two Tecton-managed locations: the Online Feature Store and the Offline Feature Store.

The Online Feature Store is a key-value store that contains the most recent version of feature data. This data is used to serve features to prediction consumers at low-latency.

The Offline Feature Store contains past and present feature values of all feature data. This data is served to batch lookup onsumers who make

The following diagram illustrates this arrangement.

Materialization Data Flow

To enable materialization on Feature Packages, pass a MaterializationConfig argument to the FeaturePackage. For more details, see materialization configuration.

Materialization Operations

Tecton materializes data by automatically performing various materialization operations on FeaturePackages.

Steady-state

Steady-state Materialization refers to materialization being performed on new data arriving in real-time. Steady State Materialization continuously occurs on all FeaturePackages where Materialization is enabled.

When a FeaturePackage has materialization enabled, Tecton will schedule steady-state materialization jobs on an ongoing basis in order to maintain fresh feature values. The frequency of steady-state materialization is controlled by the schedule_interval parameter in the Feature Package's materialization options.

Backfills

Backfill refers to any materialization operations performed on data in the past. There are two Backfill operations.

Bootstrap

The initial materialization of a Feature Package is referred to as a bootstrap. During a botstrap materialization, existing raw data is processed into feature values.

When materialization is initially enabled for a Feature Package, Tecton performs a bootstrap materialization. The amount of data materialized during a bootstrap is controlled by the feature_start_time parameter in the Feature Package's materialization options.

Overwrite

You can recalculate Materialized data for a range of timestamps that are later found to have an error in the data source. This is available at either at the Feature Package level or the Data Source level using an Overwrite operation.

Overwrite backfills are currently in private preview and they must be run manually by Tecton support. To perform an overwrite backfill, contact support@tecton.ai, which will manually process the request.

Monitoring

Tecton provides tools to monitor and debug production Feature Packages. The web UI provides Materialization overviews, the SDK contains specifics about the Feature Package class, and the CLI makes cluster materialization overviews available. More information on monitoring is available in Monitoring Materializations.