PluginsElasticsearchTasks

Esql​Esql

yaml
type: "io.kestra.plugin.elasticsearch.Esql"

Query Elasticsearch using ES|QL.

This task allows you to query Elasticsearch using ES|QL.

Examples

Load data in bulk to Elasticsearch and query it using ES|QL.

yaml
id: bulk_load_and_query
namespace: company.team

tasks:
  - id: extract
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/resolve/main/jsonl/books.jsonl

  - id: load
    type: io.kestra.plugin.elasticsearch.Bulk
    from: "{{ outputs.extract.uri }}"

  - id: sleep
    type: io.kestra.plugin.core.flow.Sleep
    duration: PT5S
    description: Pause needed after load before we can query

  - id: query
    type: io.kestra.plugin.elasticsearch.Esql
    fetchType: STORE
    query: |
      FROM books
        | KEEP author, name, page_count, release_date
        | SORT page_count DESC
        | LIMIT 5

pluginDefaults:
  - type: io.kestra.plugin.elasticsearch
    values:
      connection:
        headers:
          - "Authorization: ApiKey yourEncodedApiKey"
        hosts:
          - https://yourCluster.us-central1.gcp.cloud.es.io:443

Properties

connection

The connection properties.

fetchType

  • Type: string
  • Dynamic: ❌
  • Required: ✔️
  • Default: FETCH
  • Possible Values:
    • STORE
    • FETCH
    • FETCH_ONE
    • NONE

The way you want to store the data.

FETCH_ONE output the first row, FETCH output all the rows, STORE store all rows in a file, NONE do nothing.

query

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The ElasticSearch value.

Can be a JSON string. In this case, the contentType will be used or a raw Map.

filter

  • Type: object
  • Dynamic: ✔️
  • Required: ❌

Query filter.

Specify a DSL query in the filter parameter to filter the set of documents that an ES|QL query runs on.

routing

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Controls the shard routing of the request.

Using this value to hash the shard and not the id.

Outputs

row

  • Type: object
  • Required: ❌

rows

  • Type: array
  • SubType: object
  • Required: ❌

size

  • Type: integer
  • Required: ❌

total

  • Type: integer
  • Required: ❌

uri

  • Type: string
  • Required: ❌
  • Format: uri

Definitions

io.kestra.plugin.elasticsearch.ElasticsearchConnection

  • hosts
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ✔️
    • Min items: 1
  • basicAuth
  • headers
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • pathPrefix
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • strictDeprecationMode
    • Type: boolean
    • Dynamic: ❌
    • Required: ❌
  • trustAllSsl
    • Type: boolean
    • Dynamic: ❌
    • Required: ❌

io.kestra.plugin.elasticsearch.ElasticsearchConnection-BasicAuth

  • password
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • username
    • Type: string
    • Dynamic: ✔️
    • Required: ❌