TectonDataFrame
A thin wrapper around Pandas, Spark, and Snowflake dataframes.
Attributes​
Name | Data Type | Description |
---|---|---|
columns | List[str] | The columns of the dataframe |
Methods​
Name | Description |
---|---|
explain(...) | Prints the query tree. |
get_time_range(...) | |
subtree(...) | Creates a TectonDataFrame from a subtree of prior querytree labeled by a node id in .explain(). |
to_pandas() | Returns data as a Pandas DataFrame. |
to_snowpark() | Returns data as a Snowpark DataFrame. |
to_spark() | Returns data as a Spark DataFrame. |
__init__(...)​
Parameters​
spark_df
(Default:None
)pandas_df
(Default:None
)snowflake_df
(Default:None
)querytree
(Default:None
)temp_table_registered
(Default:None
)
explain(...)​
Prints the query tree. Should only be used when this TectonDataFrame is backed by a query tree.
Parameters​
-
node_id
(bool
) – If True, the unique id associated with each node will be rendered. (Default:True
) -
name
(bool
) – If True, the class names of the nodes will be rendered. (Default:True
) -
description
(bool
) – If True, the actions of the nodes will be rendered. (Default:True
) -
columns
(bool
) – If True, the columns of each node will be rendered as an appendix after tree itself. (Default:False
)
get_time_range(...)​
Parameters​
timestamp_key
subtree(...)​
Creates a TectonDataFrame from a subtree of prior querytree labeled by a node id in .explain().
Parameters​
node_id
to_pandas()​
Returns data as a Pandas DataFrame.
Parameters​
pretty_sql
([bool
]) - Not applicable when using Tecton on Spark. If True, the sql generated by to_pandas() will be reformatted and executed as a more readable, multiline string which is useful for debugging. If False, the SQL will be executed as a one line string. Use pretty_sql=False for better performance. (Default:False
)
Returns​
A Pandas DataFrame.
to_snowpark()​
Returns data as a Snowpark DataFrame.
Parameters​
pretty_sql
([bool
]) - Not applicable when using Tecton on Spark. If True, the sql generated by to_snowpark() will be reformatted and executed as a more readable, multiline string which is useful for debugging. If False, the SQL will be executed as a one line string. Use pretty_sql=False for better performance. (Default:False
)
Returns​
A Snowpark DataFrame.
to_spark()​
Returns data as a Spark DataFrame.
Returns​
A Spark DataFrame.