Skip to content

S3 Buckets


To grant Tecton access to your S3 data sources, use AWS IAM role-based permissions. This requires setting bucket policies for the S3 buckets you want to use with Tecton.

To add S3 buckets:

  1. Add bucket policies
  2. Register the S3 data source
  3. Test the S3 data source


If your S3 data source is partitioned in a directory structure, we recommend that you register the data source with your AWS Glue Data Catalog and add the data as a Hive data source.

Adding a Bucket Policy

This AWS blog post explains how to configure bucket policies using IAM roles. An example bucket policy is shown below. The bucket policy gives permissions to the IAM role that Tecton uses to run Spark jobs.

The Principal agent in the policy below is used to add S3 data sources to Tecton's Free Trial. If you have an enterprise version of Tecton, obtain your company's IAM role from your Tecton deployment specialist.

    "Version": "2012-10-17",
    "Statement": [
            "Sid": "Grant Tecton Access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::{YOUR-TECTON-AWS-ACCOUNT}:root"
            "Action": [
            "Resource": [

On rare occasions, traffic between AWS accounts gets routed outside the AWS networks. If your security policy requires zero tolerance for data going across a public network, ask your Tecton deployment special to set up a VPC Endpoint for S3 to ensure that all traffic remains on AWS private networks.

Registering an S3 Data Source

Once Tecton has access, register the data sources with Tecton as part of the file in your Feature Repository.

Create a config object using FileDSConfig and place it in a VirtualDataSource object with metadata to discover the new data source. For example:

sample_data_config = FileDSConfig(

sample_data_vds = VirtualDataSource(
            'release': 'production'

After you have created these objects in your local Feature Repository, call tecton apply to submit them to the production Feature Store.

Testing an S3 Data Source

To test that the connection to the S3 data source has been made correctly, open the interactive notebook that you use for Tecton development and preview the data:

ds = tecton.get_virtual_data_source('sample_data')

If you get a 403 ERROR when calling the preview command, Tecton does not have permission to access the data. Check the bucket policy and the AWS setup. If you continue to get errors, contact Tecton support.