Spark vs Snowflake

What is the difference between Tecton on Spark and Tecton on Snowflake?

Tecton helps productionize real-time machine learning applications by providing a unified feature engineering framework and integrations with customers' existing data stack that help users define, test, serve, and manage features. Tecton on Spark and Tecton on Snowflake enable customers to use either Apache Spark or Snowflake as the underlying compute engine for the full feature lifecycle.

Tecton on Spark manages and orchestrates Spark clusters within customers' Spark environments to help with materialization, feature retrieval, and more. It allows customers to integrate with various data sources such as Amazon S3, Hive, and Redshift.

Tecton on Spark can integrate with Databricks (AWS or Google Cloud), EMR, and Dataproc as a materialization engine. Interactive feature development can be done in any notebook with access to Spark.

Tecton on Snowflake integrates with Snowflake's cloud-native data platform. It leverages customers' Snowflake instances for things like materialization and feature retrieval. Tecton on Snowflake also connects to customers' Snowflake tables as data sources and to store offline features.

For more details on how to use Tecton on Snowflake and differences in Tecton's product between Spark & Snowflake, see our documentation.

Can I access data in my Snowflake warehouse despite using Tecton on Spark?

Tecton on Spark can connect to Snowflake tables as a data source for batch features. See our documentation for more details.

Should I use Tecton on Spark or Tecton on Snowflake?

This largely depends on your existing data infrastructure. Tecton has helped onboard customers onto a diverse set of machine learning use cases for both Tecton on Spark and Tecton on Snowflake. Talk to our team to learn more about which option is a better fit for your current data stack.