The ttl (time-to-live) Parameter

ttl is a Batch and Stream Feature View parameter, as well as a Feature Table parameter.

The value of ttl affects the availability of feature data in the online store, the generation of training feature data, and the deletion of feature values from the online store.

ttl specifies the amount of time, prior to the current time, for which feature data should be available in the online store. At materialization time, feature data with timestamps earlier than the current time minus the ttl value are not written to the online store. Tecton never writes feature data with timestamps earlier than feature_start_time to the online store. So Tecton writes features as far back as the max(feature_start_time, current time - ttl) to the online store

For feature views without aggregations, the default value of ttl is 'None', which means that no feature data will expire from the online store. When generating offline training datasets, the window to "look back" relative to the training example's timestamp will begin at the feature start time.

note

For a Feature View that contains one or more Aggregations, the Feature View's ttl value is implicitly set to the aggregation_interval value.

Effect of ttl on the materialization of feature data into the online store

Increasing the ttl value can increase the amount and throughput of data written to the online store. and may affect availability for reads from the online store.

Effect of ttl on the availability of feature data in the online store

ttl specifies the amount of time, prior to the current time, that feature data is available in the online store. Feature data with timestamps earlier than the current time minus the ttl value will expire.

Effect of ttl on the generation of training feature data

ttl specifies the maximum amount of time prior to the timestamp of a training event, that data in a Feature View's data source is available for generating feature data for the training event.

note

Lower ttl values will allow get_historical_features() to run more efficiently in some cases, because the amount of training data generated will be reduced.

Effect of ttl on the deletion of a feature value from the online store

A feature value is deleted from the online store when all of the following conditions are met:

• The feature value has expired from the online store (because the feature value's timestamp is earlier than the current time minus the ttl)
• The online store is running on Redis and the Feature View was created after August 3, 2022.
• For a non-aggregate feature value: current time - feature row timestamp > ttl + 7 days.
• For an aggregate feature value: current time - timestamp of the feature value > aggregation_interval + longest time_window + 7 days.

note

Lower ttl values will reduce feature data storage costs.

If there is more than a 7 day gap between the current time and the last time a Feature View's values were written to the online store, some of the Feature View's values not exceeding the ttl period may be automatically deleted from the online store. In this case, these values will be null. For assistance with this situation, contact Tecton Support for assistance.

note

The ttl parameter has no effect on the deletion of feature values from the offline store. To remove values from the offline store, consider the following options:

1. .delete_keys(): Feature Views and Feature Tables have a .delete_keys() method to delete entries matching specified join key(s) from the online and offline store.

2. S3 Lifecycle Management: Set up an S3 Lifecycle configuration to automatically delete S3 objects after expiration.

Details of ttl for Batch Feature Views

Batch Feature Views do not use the wall clock time to determine when to stop serving a feature value. Instead, batch feature values are expired out only when the next incremental batch materialization jobs completes. This is in order to prevent expected or unexpected delays in the batch pipeline from degrading online serving.

For example, when a batch feature view has a batch_schedule=1d then feature values are materialized on a daily cadence, e.g. events from Jan 2 are actually materialized on the next daily run on Jan 3. If that Batch Feature View has a ttl=1d, then Tecton will serve those feature values until the next daily run on Jan 4 has completed. This prevents scheduling delays, job execution times, or batch outages from degrading online serving.