Private Preview

This feature is currently in Private Preview.

Unity Catalog Single User Clusters with Fine-Grained Access Control

Fine-grained access control (FGAC) is a Databricks Private Preview capability that allows Single User Unity Catalog clusters access to data governed by Delta Live Tables, views & dynamic views, and tables with RLS/CM (row-level security and column masking).

To use Unity Catalog Single User Access Mode with FGAC:

• Please inform Tecton so we can appropriately configure your deployment
• Install SDK 0.9.4+
  • In addition, set your Feature View's tecton_materialization_runtime to 0.9.4 or above. You can alternatively set this as a default for all Feature Views in your workspace by updating your repo.yaml file.
• Modify your UnityConfig to include access_mode=UnityCatalogAccessMode.SINGLE_USER_WITH_FGAC
• Modify your DatabricksClusterConfig or DatabricksJsonClusterConfig:
  • Pin your Databricks Runtime Version to 14.3.x-scala2.12
  • Your spark_config must include "spark.databricks.remoteFiltering.enabled": "true" & "spark.databricks.remoteFiltering.type": "serverless"

The example changes are as follows:

DatabricksClusterConfig

unity_catalog_data_source = BatchSource(
    name="unity_catalog_data_source",
    batch_config=UnityConfig(
        catalog="main", schema="default", table="department", access_mode=UnityCatalogAccessMode.SINGLE_USER_WITH_FGAC
    ),
)


@batch_feature_view(
    sources=[unity_catalog_data_source],
    tecton_materialization_runtime="0.9.3",  # Or update the workspace default in `repo.yaml`
    batch_compute=DatabricksClusterConfig(
        dbr_version="14.3.x-scala2.12",
        spark_config={
            "spark.databricks.remoteFiltering.enabled": "true",
            "spark.databricks.remoteFiltering.type": "serverless",
        },
    ),
    ...,
)
def feature_view():
    return ...

DatabricksJsonClusterConfig

unity_catalog_data_source = BatchSource(
    name="unity_catalog_data_source",
    batch_config=UnityConfig(
        catalog="main", schema="default", table="department", access_mode=UnityCatalogAccessMode.SINGLE_USER_WITH_FGAC
    ),
)


@batch_feature_view(
    sources=[unity_catalog_data_source],
    tecton_materialization_runtime="0.9.3",  # Or update the workspace default in `repo.yaml`
    batch_compute=DatabricksJsonClusterConfig(
        """
      {
        "new_cluster": {
          "spark_version": "14.3.x-scala2.12",
          "spark_conf": {
              "spark.databricks.remoteFiltering.enabled": "true",
              "spark.databricks.remoteFiltering.type": "serverless"
          }
        }
      }
      """
    ),
    ...,
)
def feature_view():
    return ...