Materialization is the process of computing values from FeaturePackage definitions and making the resulting data available for queries to the feature store.
Materialization runs are processing jobs that precompute the queries defined by a Feature Package, then store the result. This enables efficient retrieval of Feature values at lookup time. Tecton stores Materialized data in two Tecton-managed locations: the Online Feature Store and the Offline Feature Store.
The Online Feature Store is a key-value store that contains the most recent version of feature data. This data is used to serve features to prediction consumers at low-latency.
The Offline Feature Store contains past and present feature values of all feature data. This data is served to batch lookup onsumers who make
The following diagram illustrates this arrangement.
To enable materialization on Feature Packages, pass a
MaterializationConfig argument to the FeaturePackage. For more details, see materialization configuration.
Tecton materializes data by automatically performing various materialization operations on FeaturePackages.
Steady-state Materialization refers to materialization being performed on new data arriving in real-time. Steady State Materialization continuously occurs on all FeaturePackages where Materialization is enabled.
When a FeaturePackage has materialization enabled, Tecton will schedule steady-state materialization jobs on an ongoing basis in order to maintain fresh feature values. The frequency of steady-state materialization is controlled by the
schedule_interval parameter in the Feature Package's materialization options.
Backfill refers to any materialization operations performed on data in the past. There are two Backfill operations.
The initial materialization of a Feature Package is referred to as a bootstrap. During a botstrap materialization, existing raw data is processed into feature values.
When materialization is initially enabled for a Feature Package, Tecton performs a bootstrap materialization. The amount of data materialized during a bootstrap is controlled by the
feature_start_time parameter in the Feature Package's materialization options.
You can recalculate Materialized data for a range of timestamps that are later found to have an error in the data source. This is available at either at the Feature Package level or the Data Source level using an Overwrite operation.
Overwrite backfills are currently in private preview and they must be run manually by Tecton support. To perform an overwrite backfill, contact firstname.lastname@example.org, which will manually process the request.
Tecton provides tools to monitor and debug production Feature Packages. The web UI provides Materialization overviews, the SDK contains specifics about the Feature Package class, and the CLI makes cluster materialization overviews available. More information on monitoring is available in Monitoring Materializations.