Configure Dataproc in Google Cloud
The following steps explain how to connect Tecton to Dataproc.
Confirm Tecton control plane service account​
Your Tecton control plane service account is provided by your Tecton deployment
specialist. It will look like
tecton-<deployment>-control-plane@<tecton-deployment>.iam.gserviceaccount.com.
Create a VPC Network or reuse an existing VPC Network​
This can be a new or existing network. Later, when developing features in Tecton, your Dataproc compute configuration will reference this network.
Connect Tecton to Dataproc​
Grant the Tecton control plane service account access to manage Dataproc resources.
Tecton requires the following permissions on Dataproc:
compute.zones.get
compute.projects.get
dataproc.clusters.create
dataproc.clusters.delete
dataproc.clusters.get
dataproc.clusters.use
dataproc.jobs.create
dataproc.jobs.get
dataproc.jobs.list
dataproc.operations.cancel
dataproc.operations.get
dataproc.operations.list
dataproc.workflowTemplates.instantiateInline
resourcemanager.projects.get
Additionally, the Tecton control plane service account will need to be granted "Service Account User" on the Spark job service account.