Skip to main content
Version: 0.8

Configure Dataproc in Google Cloud

The following steps explain how to connect Tecton to Dataproc.

Confirm Tecton control plane service account​

Your Tecton control plane service account is provided by your Tecton deployment specialist. It will look like tecton-<deployment>-control-plane@<tecton-deployment>.iam.gserviceaccount.com.

Create a VPC Network or reuse an existing VPC Network​

This can be a new or existing network. Later, when developing features in Tecton, your Dataproc compute configuration will reference this network.

Connect Tecton to Dataproc​

Grant the Tecton control plane service account access to manage Dataproc resources.

Tecton requires the following permissions on Dataproc:

compute.zones.get
compute.projects.get
dataproc.clusters.create
dataproc.clusters.delete
dataproc.clusters.get
dataproc.clusters.use
dataproc.jobs.create
dataproc.jobs.get
dataproc.jobs.list
dataproc.operations.cancel
dataproc.operations.get
dataproc.operations.list
dataproc.workflowTemplates.instantiateInline
resourcemanager.projects.get

Additionally, the Tecton control plane service account will need to be granted "Service Account User" on the Spark job service account.

Was this page helpful?

🧠 Hi! Ask me anything about Tecton!

Floating button icon