Execute a SQL query on a Databricks cluster.
See Retrieve the connection details in the Databricks documentation to discover how to retrieve the needed configuration properties. We're using the Databricks JDBC driver to execute a Query, see https://docs.databricks.com/integrations/jdbc-odbc-bi.html#jdbc-driver-capabilities for its capabilities.
Due to current limitation of the JDBC driver with Java 21, Arrow is disabled, performance may be impacted, see here and here from Databricks status on Java 21 support.
type: "io.kestra.plugin.databricks.sql.Query"id: databricks_sql_query
namespace: company.team
tasks:
- id: sql_query
type: io.kestra.plugin.databricks.sql.Query
accessToken: <your-accessToken>
host: <your-host>
httpPath: <your-httpPath>
sql: SELECT 1
Databricks host.
Databricks cluster HTTP Path.
To retrieve the HTTP Path, go to your Databricks cluster, click on Advanced options then, click on JDBC/ODBC. See Retrieve the connection details for more details.
SQL query to be executed.
Databricks access token.
The time zone id to use for date/time manipulation. Default value is the worker default zone id.
The number of fetched rows.
The URI of the result file in Kestra's internal storage (.ion file i.e. Amazon Ion text format).