Types of Materialization Alerts
Tecton provides multiple types of alerts for Feature Views. When alerts are triggered, an email is sent to the address specified in the
alert_email parameter of the
There are different types of alerts for each type of materialization issue:
- Freshness Alerts
- Repeated Failures Alerts
- Too Many Failures Alerts
Feature data becomes more stale over time when new features are not being materialized. Tecton triggers a
FeatureViewNotFresh when feature data becomes too stale, which is a threshold set by the
expected_feature_freshness parameter. If this parameter is not specified, the alert threshold defaults to twice the
aggregation_slide_period for Stream Window Aggregate Feature Views or twice the
batch_schedule for other Feature Views.
The most common causes of this type of alert are:
- Missing upstream data
- Errors in feature definitions that cause materialization jobs to either fail or produce zero feature values
- An outage causes materialization jobs to fail
You can increase
expected_feature_freshness to a longer time interval based on the expected freshness of your features. See Monitoring Materializations for more information.
Repeated Failures Alerts
Tecton automatically schedules retries for failing materialization jobs. If these retries fail consistently, Tecton will trigger an alert. There are two types of repeated failure alerts:
FeatureViewBatchMaterializationFailures. Batch materialization jobs have failed 2 or more times.
FeatureViewStreamingMaterializationFailures. Streaming materialization jobs have failed 2 or more times.
Materialization jobs that need to be retried due to spot instance availability are not considered failures. Most often, this error is caused by errors in feature definitions.
Too Many Failures Alerts
When materialization retries fail too many times, Tecton will move the Feature View to a "Too Many Failures" state and will not continue to retry materialization.
At this point, the
FeatureViewTooManyFailures alert will be fired. This alert is most commonly caused by incorrect Transformation code.