Types of Materialization Alerts
Types of Alerts
Tecton provides multiple types of alerts for Feature Packages. When alerts are triggered, an email is sent to the address specified in the
alert_email parameter of the
List of Alerts
There are different types of alerts for each type of materialization issue:
- Freshness Alerts
- Repeated Failures Alerts
- Too Many Failures Alerts
Feature data becomes more stale over time when new features are not being materialized. Tecton triggers a
FeaturePackageNotFresh when feature data becomes too stale, which a threshold set by the
expected_feature_freshness parameter. If this parameter is not specified, the alert threshold defaults to twice the Feature Package materialization interval (
schedule_interval, or for streaming Feature Packages,
The most common causes of this type of alert are:
- Missing upstream data
- Errors in feature definitions that cause materialization jobs to either fail or produce zero feature values
- An outage causes materialization jobs to fail
You can increase
expected_feature_freshness to a longer time interval based on the expected freshness of your features. See Monitoring Materializations for more information.
Repeated Failures Alerts
Tecton automatically schedules retries for failing materialization jobs. If these retries fail consistently, Tecton will trigger alert. There are two types of repeated failure alerts:
FeaturePackageBatchMaterializationFailures. Batch materialization jobs have failed 2 or more times.
FeaturePackageStreamingMaterializationFailures. Streaming materialization jobs have failed 2 or more times.
Materialization jobs that need to be retried due to spot instance availability are not considered failures. Most often, this error is caused by errors in feature definitions.
Too Many Failures Alerts
When materialization retries fail too many times, Tecton will move the Feature Package to a "Too Many Failures" state and will not continue to retry materialization.
At this point, the
FeaturePackageTooManyFailures alert will be fired. This alert is most commonly caused by incorrect Transformation code.