How to manage Tecton versions and safely upgrade

Tecton releases new features, product improvements, bug fixes, and security updates with each new version (see Versioning Scheme for more details). This page outlines upgrade guidance to minimize breaking changes or unexpected issues when upgrading to a new release.

1) Read the Release Notes & Upgrade Guide

For each new release, Tecton publishes Release Notes and an Upgrade Guide which contain details about new features, breaking / non-breaking changes, and changes to SDK functionality. Read these pages prior to upgrading to prevent any unexpected issues during the upgrade process.

2) Pin a Tecton minor version

Tecton recommends pinning minor versions, but allowing for automatic upgrades for patch versions (e.g. 1.0.*) This ensures that customers receive low-risk bug fixes automatically, but can adopt new minor versions safely. Generally, this means pinning the Tecton version (pip install tecton==1.0.*, pinning Tecton Spark UDFs) and pinning Python versions (e.g. to Python 3.8).

note

If you will be running unit tests, use one of the following commands, instead of pip install tecton==1.0.*, to pin the Tecton version:

• To install with Pyspark 3.1: pip install 'tecton[pyspark]==1.0'
• To install with Pyspark 3.2: pip install 'tecton[pyspark3.2]==1.0'
• To install with Pyspark 3.3: pip install tecton==1.0 pyspark==3.3

The table below describes the areas where Tecton is versioned and recommended action items (if any) for staying up-to-date with the latest release.

Type	Description	Action items
CLI	The main place to update is in CI/CD (e.g. to use tecton apply, to rotate API keys, run tests)	Pin the Tecton pip version and Python versions in the environment running the CLI.
Offline feature retrieval (including notebook environments)	This is how users connect to Tecton for feature development and offline feature retrieval. This can be for training data generation or batch scoring. With Tecton on Spark, users use a Spark-based notebook cluster which needs to be updated. In Tecton on Snowflake, users can retrieve features in any Python environment.	Tecton on Spark: As per initial setup for Databricks or EMR, users should pin versions of Tecton libraries + jars at the cluster level. Customers may also want to pin the Python version too. For Databricks, this can be achieved using a specific Databricks Runtime (which installs a specific Python version). For EMR, customers can specify a custom AMI. EMR notebook users can also select a specific version via a %%configure cell. Tecton on Spark (Athena-based feature retrieval): Pin any production inference or training jobs that leverage Tecton’s Athena-based feature retrieval. Tecton on Snowflake: Ensure that production inference or training jobs pin dependencies (e.g. pip install 'tecton[snowflake]==1.0.*' and Python version = 3.8). If you use a Jupyter notebook server like Sagemaker notebooks, make sure you pin Tecton versions in the servers.
[Tecton Managed] Materialization jobs	Tecton launches jobs to materialize feature data to the offline and online store.	No action. Tecton on Spark: Starting in Tecton 0.6, users can optionally pin EMR / Databricks versions in these jobs.
[Tecton Managed] Tecton internal Spark cluster	Tecton uses this internal cluster to validate Feature Views and Data Sources.	No action. Tecton automatically upgrades this cluster.

3) Upgrade a Staging Workspace

note

To ensure a safe upgrade, Tecton disallows any destructive changes (i.e. Recreates) to your Feature Repository while upgrading. For example, tecton apply will prevent changes to a Feature View's Offline Store format while upgrading as this would normally cause re-materialization.

Upgrade your Feature Repository first and run tecton apply before making any destructive changes or Recreates.

As users go to production, they typically maintain different workspaces for different stages of a launch. A good practice is to test changes in a staging workspace that mimics the live production workspace. Below, we walk through how to test a new Tecton version on such a staging workspace.

Note that it is OK for a Tecton cluster to have different Tecton versions on different workspaces.

There are two options for upgrading:

• Option 1: Use tecton plan to do an in-place upgrade on a staging workspace. This is sufficient for most releases.

• Option 2: A/B test using two different staging workspaces to validate that feature values are the same across Tecton versions. This is a slower and more expensive process but is the safer option.

info

Tecton SDK Versions are backwards compatible but not forwards compatible. This is to say that Tecton clients can interact with workspaces that were applied with older SDK Versions. They cannot interact with workspaces that were applied with newer versions.

This means these Tecton Clients need to be upgraded before the corresponding workspaces they use are upgraded. You might have to make changes to the clients if they use features that have been deprecated or are impacted by any breaking change.

a) In-place upgrade

1. Upgrade your local environment's version of Tecton.
2. Upgrade Tecton versions for all clients using the Tecton SDK for feature development and retrieval. This could be training jobs, batch inference jobs or a development notebook. See version specific upgrade guide for upgrade instructions.
3. Run tecton plan on an existing staging live workspace that is equivalent to the production workspace. Follow the version upgrade guide until tecton plan produces an expected diff (i.e. no re-materialization, no errors).
   1. This should usually indicate no need to re-materialize features (which can be costly). If it does, consult the version specific upgrade guide or contact support.
   2. If there are any deprecated fields or breaking changes, follow the version specific upgrade guide to resolve.
   3. Tecton may upgrade some definitions to be compatible with the latest version, as described in the plan output.
   4. (if necessary) run tecton plan --suppress-recreates when Tecton’s upgrade guide recommends it
4. (Optional) If there are a lot of changes in the above, consider using the slower A/B test approach to be safe.
5. On the staging workspace, run tecton apply of the final plan id from step 2. (i.e. tecton apply --plan-id <plan-id>) when you’re confident you’ve made all the right changes.
6. Validate that there are no potential production impacting issues, especially with batch inference jobs, training jobs, or (rare) re-materialization jobs for feature views.

b) A/B test using two workspaces

One way to do this is to have two different Tecton workspaces that have the same feature definitions. Upgrade the version on one workspace to the latest Tecton version. Also, set up a notebook environment that uses the new Tecton version. You can then A/B test the new Tecton SDK version and verify the outputs of get_historical_featuresare the same for a feature service.

Example test on Spark:

• Control environment (existing)
  • Staging workspace applied using Tecton 0.8.6
  • Notebook cluster using Tecton 0.8.6, which calls get_historical_features
• Treatment environment (upgrading)
  • Workspace applied using Tecton 1.0
  • Notebook cluster using Tecton 1.0, which calls get_features_for_events
• Expected outcome: The result of get_features_for_events and get_historical_features on the same events dataframe/spine should be identical. Ensure that the input events dataframe/spine is deterministically generated (e.g read from S3) otherwise the results may differ.

4) Upgrade Production Workspaces

After validating a successful upgrade on a staging workspace, you can repeat the above process on the corresponding production workspaces.