Monitor Materialization
Materialization jobs orchestrated by Tecton can be monitored and debugged using the Tecton Web UI, SDK, and CLI in case of failures.
Monitoring Failures​
Tecton provides tools to monitor and debug materialized Feature Views from Tecton's Web UI, SDK, and CLI.
Web UI: Materialization Tab​
The easiest way to check the health of a materialized Feature View is using the Web UI. Navigate to the Feature View in the Web UI and select the Materialization tab to see Feature View materialization information at a glance.
SDK: materialization_status()​
Tecton's SDK provides a
materialization_status()
method on Feature View objects that returns details about materialization
attempts.
import tecton
fv = tecton.get_workspace("my_space").get_feature_view("my_fv")
fv.materialization_status()
>>> All the displayed times are in UTC time zone
TYPE WINDOW_START_TIME WINDOW_END_TIME STATUS ATTEMPT_NUMBER JOB_CREATED_AT JOB_LOGS
================================================================================================================
BATCH 2021-12-15 00:00:00 2021-12-22 00:00:00 SUCCESS 1 2021-12-22 00:00:27 https://...
BATCH 2021-12-14 00:00:00 2021-12-21 00:00:00 SUCCESS 1 2021-12-21 00:00:14 https://...
BATCH 2021-12-13 00:00:00 2021-12-20 00:00:00 SUCCESS 1 2021-12-20 00:00:13 https://...
BATCH 2021-12-12 00:00:00 2021-12-19 00:00:00 SUCCESS 1 2021-12-19 00:00:10 https://...
BATCH 2021-12-11 00:00:00 2021-12-18 00:00:00 SUCCESS 1 2021-12-18 00:00:06 https://...
CLI: materialization-status​
You can also use the tecton materialization-status $FV_NAME to see the
materialization status of a specific Feature View.
$ tecton materialization-status my_feature_view
All the displayed times are in UTC time zone
TYPE WINDOW_START_TIME WINDOW_END_TIME STATUS ATTEMPT_NUMBER JOB_CREATED_AT JOB_LOGS
================================================================================================================
BATCH 2021-12-15 00:00:00 2021-12-22 00:00:00 SUCCESS 1 2021-12-22 00:00:27 https://...
BATCH 2021-12-14 00:00:00 2021-12-21 00:00:00 SUCCESS 1 2021-12-21 00:00:14 https://...
BATCH 2021-12-13 00:00:00 2021-12-20 00:00:00 SUCCESS 1 2021-12-20 00:00:13 https://...
BATCH 2021-12-12 00:00:00 2021-12-19 00:00:00 SUCCESS 1 2021-12-19 00:00:10 https://...
BATCH 2021-12-11 00:00:00 2021-12-18 00:00:00 SUCCESS 1 2021-12-18 00:00:06 https://...
Monitoring Feature Freshness​
Feature Views might serve stale data due to various reasons, including failed materialization jobs, late-arriving data, or under-provisioned streams. The Feature Freshness metric, accessible via the web-UI, is designed to monitor these issues. For Batch Feature Views, Feature Freshness gauges the age of the data from the most recent successful materialization. In contrast, for Stream Feature Views, the metric assesses the age based on the last ingested event.
Web UI: Monitoring tab​
The Monitoring tab for a Feature view contains freshness monitoring charts for any Feature View with materialization enabled.
Web UI: Materialization tab​
The Materialization tab for a Feature View contains information about expected and actual freshness for a Feature View along with a materialization timeline.
CLI: tecton freshness for all Feature Views​
Tecton's CLI can return the status of all Feature Views using the
tecton freshness command.
$ tecton freshness
Feature View Stale? Freshness Expected Freshness Created At
=================================================================================================
partner_ctr_performance:14d Y 2wk 1d 2d 12/02/21 10:52
ad_group_ctr_performance N 1h 1m 2h 11/28/21 19:50
user_ad_impression_counts N 1m 35s 2h 10/01/21 2:16
content_keyword_ctr_performance:v2 N 1m 36s 2h 09/04/21 22:22
content_keyword_ctr_performance N 1m 37s 2h 08/26/21 12:52
user_total_ad_frequency_counts N 1m 38s 2h 08/26/21 12:52
Expected Feature Freshness​
A Feature Views's freshness is expected to be less than twice its
materialization schedule interval. This interval is determined using the
aggregation_interval for Window Aggregate Feature Views or the
batch_schedule for other Feature Views.
By default, alerts are triggered once the specified threshold is exceeded, with a slight grace period added. For Stream Feature Views, the freshness setting can be set to as low as 30 minutes. It's important to note that for Batch Feature Views, no alerts will be activated if a materialization is successful but yields no data.
The grace period's duration depends on the FeatureView's materialization schedule:
| Schedule | Grace Period |
|---|---|
| <= 10 minutes | 30 minutes |
| <= 30 minutes | 90 minutes |
| <= 1 hour | 2 hours |
| <= 4 hours | 4 hours |
| <= 24 hours | 12 hours |
| > 24 hours | 24 hours |
The table below has examples of materialization schedules mapped to default alert thresholds:
| Schedule | Default Alert Threshold |
|---|---|
| 5 minutes | 40 minutes |
| 30 minutes | 2.5 hours |
| 1 hour | 4 hours |
| 4 hours | 12 hours |
| 24 hours | 60 hours |
