Skip to main content
Version: 0.6

Monitoring Materialization

Materialization jobs orchestrated by Tecton can be monitored and debugged using the Tecton Web UI, SDK, and CLI in case of failures.

Monitoring Failures

Tecton provides tools to monitor and debug materialized Feature Views from Tecton's Web UI, SDK, and CLI.

Web UI: Materialization Tab

The easiest way to check the health of a materialized Feature View is using the Web UI. Navigate to the Feature View in the Web UI and select the Materialization tab to see Feature View materialization information at a glance.

Monitoring Materialization 1

SDK: materialization_status()

Tecton's SDK provides a materialization_status() method on Feature View objects that returns details about materialization attempts.

import tecton

fv = tecton.get_workspace("my_space").get_feature_view("my_fv")
fv.materialization_status()
>>> All the displayed times are in UTC time zone
TYPE WINDOW_START_TIME WINDOW_END_TIME STATUS ATTEMPT_NUMBER JOB_CREATED_AT JOB_LOGS
================================================================================================================
BATCH 2021-12-15 00:00:00 2021-12-22 00:00:00 SUCCESS 1 2021-12-22 00:00:27 https://...
BATCH 2021-12-14 00:00:00 2021-12-21 00:00:00 SUCCESS 1 2021-12-21 00:00:14 https://...
BATCH 2021-12-13 00:00:00 2021-12-20 00:00:00 SUCCESS 1 2021-12-20 00:00:13 https://...
BATCH 2021-12-12 00:00:00 2021-12-19 00:00:00 SUCCESS 1 2021-12-19 00:00:10 https://...
BATCH 2021-12-11 00:00:00 2021-12-18 00:00:00 SUCCESS 1 2021-12-18 00:00:06 https://...

CLI: materialization-status

You can also use the tecton materialization-status $FV_NAME to see the materialization status of a specific Feature View.

$ tecton materialization-status my_feature_view
All the displayed times are in UTC time zone
TYPE WINDOW_START_TIME WINDOW_END_TIME STATUS ATTEMPT_NUMBER JOB_CREATED_AT JOB_LOGS
================================================================================================================
BATCH 2021-12-15 00:00:00 2021-12-22 00:00:00 SUCCESS 1 2021-12-22 00:00:27 https://...
BATCH 2021-12-14 00:00:00 2021-12-21 00:00:00 SUCCESS 1 2021-12-21 00:00:14 https://...
BATCH 2021-12-13 00:00:00 2021-12-20 00:00:00 SUCCESS 1 2021-12-20 00:00:13 https://...
BATCH 2021-12-12 00:00:00 2021-12-19 00:00:00 SUCCESS 1 2021-12-19 00:00:10 https://...
BATCH 2021-12-11 00:00:00 2021-12-18 00:00:00 SUCCESS 1 2021-12-18 00:00:06 https://...

Monitoring Feature Freshness

Feature Views can serve stale data for many reasons such as failed materialization jobs, late-arriving data, and under-provisioned streams. Feature freshness can be monitored using the Web UI.

Web UI: Monitoring tab

The Monitoring tab for a Feature view contains freshness monitoring charts for any Feature View with materialization enabled.

Monitoring Freshness 1

Web UI: Materialization tab

The Materialization tab for a Feature View contains information about expected and actual freshness for a Feature View along with a materialization timeline.

Monitoring Freshness 2

CLI: tecton freshness for all Feature Views

Tecton's CLI can return the status of all Feature Views using the tecton freshness command.

$ tecton freshness
           Feature View               Stale?   Freshness   Expected Freshness     Created At
=================================================================================================
partner_ctr_performance:14d Y 2wk 1d 2d 12/02/21 10:52
ad_group_ctr_performance N 1h 1m 2h 11/28/21 19:50
user_ad_impression_counts N 1m 35s 2h 10/01/21 2:16
content_keyword_ctr_performance:v2 N 1m 36s 2h 09/04/21 22:22
content_keyword_ctr_performance N 1m 37s 2h 08/26/21 12:52
user_total_ad_frequency_counts N 1m 38s 2h 08/26/21 12:52

Expected Feature Freshness

A Feature Views's freshness is expected to be less than twice its materialization schedule interval. This interval is determined using the aggregation_interval for Window Aggregate Feature Views or the batch_schedule for other Feature Views.

By default, alerts will fire once this threshold, plus a small grace period, is crossed. For streaming Feature Views, freshness can be configured as low as 30 minutes. The grace period's duration depends on the Feature View's materialization schedule:

ScheduleGrace Period
<= 10 minutes30 minutes
<= 30 minutes90 minutes
<= 1 hour2 hours
<= 4 hours4 hours
<= 24 hours12 hours
> 24 hours24 hours

The table below has examples of materialization schedules mapped to default alert thresholds:

ScheduleDefault Alert Threshold
5 minutes40 minutes
30 minutes2.5 hours
1 hour4 hours
4 hours12 hours
24 hours60 hours

Was this page helpful?

🧠 Hi! Ask me anything about Tecton!

Floating button icon