Configure Feature Repositories

Introduction

The Repo Config is a required file (named repo.yaml by default) used to configure how Tecton builds and applies feature definitions during tecton plan, tecton apply, and tecton test.

It can be used to set defaults for Tecton objects in your Feature Repository, resulting in simpler and more easy-to-understand feature definitions. Defaults defined in the Repo Config can be overridden on a per-object basis in your Feature Repository. This helps teams using Tecton focus on developing powerful features while abstracting away parameters shared across feature definitions.

The Repo Config is supported in Version 0.8 and above.

Defining a Repo Config

By default, the plan/apply/test commands will look for Repo Config file named repo.yaml in the root of your Feature Repository. To specify another file, use the --config flag (e.g. tecton plan --config my_config.yaml).

Tecton will automatically generate a starter Repo Config during tecton init. To generate a starter Repo Config for an existing Feature Repository, run tecton repo-config init. This command will generate a repo.yaml in your Feature Repository root containing some pre-filled defaults. For additional help with defining a Repo Config, run tecton repo-config show to display your Repo Config (e.g. to validate that it’s defined as expected).

Note that tecton restore will always restore your Repo Config to repo.yaml at the root of your Feature Repository, even after running tecton apply with the --config option. This is to ensure that the Repo Config file is easy to find and can be restored even when defined outside of your Feature Repository.

Supported Defaults

The Repo Config currently supports setting the following defaults for your Feature Repository:

Tecton Object	Repo Config Key	Supported Parameters
Batch Feature View	batch_feature_view	batch_compute offline_store & online_store tecton_materialization_runtime
Stream Feature View	stream_feature_view	batch_compute & stream_compute offline_store & online_store tecton_materialization_runtime
Feature Table	feature_table	batch_compute offline_store & online_store tecton_materialization_runtime
Feature Service	feature_service	on_demand_environment

The Repo Config will add support for other Tecton objects and parameters in the future.

If a default is specified in both the Repo Config and directly in the Feature Repository, the value defined in the Feature Repository's code takes precedence. This is especially relevant to object-type parameters (offline_store, online_store, batch_compute, and stream_compute); an object-level setting in the repository code will wholly override its counterpart in the Repo Config. In the future, the Repo Config will allow for key-specific configurations within these objects.

Example Repo Config

Below is an example Repo Config with some defaults configured.

# repo.yaml

defaults:
  batch_feature_view:
    tecton_materialization_runtime: 0.8.0
    online_store:
      kind: RedisConfig
    offline_store:
      staging_table_format:
        kind: ParquetConfig
   batch_compute:
     kind: DatabricksClusterConfig
     instance_type: m5.xlarge
     number_of_workers: 2
     extra_pip_dependencies:
       - haversine==2.8.0
  stream_feature_view:
    stream_compute:
      kind: DatabricksClusterConfig
      instance_availability: on_demand
      instance_type: m5.2xlarge
      number_of_workers: 4
    offline_store:
      staging_table_format:
        kind: ParquetConfig
  feature_table:
    tecton_materialization_runtime: 0.8.0
    batch_compute:
      kind: DatabricksClusterConfig
      instance_type: m5.xlarge
      number_of_workers: 2
    online_store:
      kind: RedisConfig
  feature_service:
    on_demand_environment: tecton-python-extended:0.4