0.8 to 0.9 Upgrade Guide

Overview

With Tecton 0.9, we're excited to announce the Public Preview of Rift: Tecton’s AI-optimized, Python-native compute engine.

We've worked closely with Private Preview customers to enhance Rift's capabilities and make it ready for broader availability. Along the way, we’ve also made significant improvements to Rift’s performance and reliability. Key new capabilities include:

• Tecton Secrets for secure credential management and usage within Tecton
• Custom Python Environments for Batch & Stream Feature Views, which unlock the ability to use any Python dependency in Tecton Feature Views
• Support for Tecton's powerful Aggregation Engine, which optimizes the feature engineering experience and production performance of aggregation features in Tecton

But that's not all — 0.9 includes a number of other enhancements to Tecton:

• Data Compaction for Stream Feature Views (Private Preview): A powerful capability that makes online feature serving and storage more performant than ever before
• Publish Features to Warehouse (Public Preview): Streamline data access by publishing feature values directly to your data warehouse for analytics and exploration
• New methods for offline feature retrieval (General Availability): New methods that make the offline feature retrieval experience more intuitive and useful
• And various quality-of-life improvements to Tecton’s framework & CLI

Read the general Upgrade Process page in addition to the following documentation for guidance on how to safely upgrade from 0.8 to 0.9.

To ensure a safe upgrade, Tecton disallows any destructive changes (i.e. Recreates) to your Feature Repository while upgrading. For example, tecton apply will prevent changes to a Feature View's Offline Store format while upgrading as this would normally cause re-materialization.

Step-by-Step Upgrade Flow

Upgrading from 0.8 to 0.9 should be a lightweight process with minimal breaking changes. This should typically take ~15-30 minutes to make the required changes, in addition to time for testing and verification.

These are the most important changes in 0.9 that are relevant to the upgrade process:

• Spark-based Feature Views require a Materialization Runtime Version (tecton_materialization_runtime) of at least 0.9.0.
• Rift-based Feature Views require a Materialization Environment (environment) of at least tecton-rift-core-0.9.0.
• PushSource is deprecated and should be replaced by a StreamSource that uses a PushConfig.
• The default Databricks Runtime Version is updated to 11.3.x-scala2.12
• The default EMR Version is updated to 6.9.1.
• The Offline Retrieval methods get_historical_features and run are marked as deprecated.

Most customers should follow this guidance while upgrading:

Upgrade Your Feature Repository

1. Update Materialization Runtime & Environment Defaults:

   • Spark-based Feature Views: Update the tecton_materialization_runtime field in your repo.yaml file to 0.9.0. Note that this will restart currently-running jobs for StreamFeatureViews. With Tecton's zero-downtime stream restarts, this will cause minimal disruption.

   • Rift-based Feature Views: Update the environment field in your repo.yaml to tecton-rift-core-0.9.0.

     Here's an example repo.yaml:

   defaults:
     batch_feature_view:
       tecton_materialization_runtime: 0.9.0 # For Spark-based Feature Views
       environment: tecton-rift-core-0.9.0 # For Rift-based Feature Views
     stream_feature_view:
       tecton_materialization_runtime: 0.9.0 # For Spark-based Feature Views
       environment: tecton-rift-core-0.9.0 # For Rift-based Feature Views
     feature_table:
       tecton_materialization_runtime: 0.9.0 # For Spark-based Feature Views
       environment: tecton-rift-core-0.9.0 # For Rift-based Feature Views

2. Replace PushSource objects with a StreamSource that uses a PushConfig.

   PushSource has been deprecated and will be fully removed in an upcoming SDK release. Please update any PushSource to instead be a StreamSource with stream_config=PushConfig(). This change should not cause downstream feature views to be recreated.

   Before:

   user_click_push_source = PushSource(
       name="user_event_source",
       schema=user_schema,
   )

   After:

   user_click_push_source = StreamSource(
       name="user_event_source",
       stream_config=PushConfig(),
       schema=user_schema,
   )

3. New Default Spark Runtime Versions

   • The default EMR Version in Tecton 0.9 is 11.3.x-scala2.12.
   • The default Databricks Runtime Version in Tecton 0.9 is 6.9.1.

   Users can pin a specific EMR or Databricks Runtime Version as detailed here.

Upgrade Development Environments (e.g. Notebooks) and CI/CD

1. Upgrade to the new Offline Retrieval Methods

   The methods get_historical_features() and run() have been marked as deprecated in Tecton 0.9. This is not a breaking change -- these methods will continue working but will show deprecation messages when used. These methods will be fully removed in a future SDK release.

   Migrate to the new and improved methods get_features_for_events(), get_features_in_range(), get_partial_aggregates() and run_transformation(). See Offline Retrieval Methods for more information.

2. Replace calls to FeatureView.deletion_status()

   This method was deprecated in Tecton 0.8 and has now been fully removed. See the new recommended workflow in the following sections.

SDK Interfaces that are deprecated or removed in 0.9

Methods that are deprecated in 0.9 and will be removed in the future

Deprecated Parameter	Replacement
FeatureView.get_historical_features(start,end)	FeatureView.get_features_in_range(start,end)
FeatureView.get_historical_features(spine)	FeatureView.get_features_for_events(events)
FeatureService.get_historical_features(spine)	FeatureService.get_features_for_events(events)
FeatureView.run(aggregation_mode="none")	FeatureView.run_transformation(...)
FeatureView.run(aggregation_mode="partial")	FeatureView.get_partial_aggregates(start,end)
FeatureView.run(aggregation_mode="full")	FeatureView.get_features_in_range(start,end)

Methods, parameters, & attributes that were previously deprecated and are officially removed in 0.9

This is a breaking change if your Feature Repository still uses the removed methods, parameters, or attributes.

Removed method, parameter, or attribute	Replacement
FeatureView.deletion_status()	Retrieve the Deletion Job ID (via FeatureView.delete_keys() or FeatureView.list_materialization_jobs()) and call get_materialization_job(job_id).state