PluginsCoreTasksFlow

WorkingDirectory​Working​Directory

yaml
type: "io.kestra.plugin.core.flow.WorkingDirectory"

Run tasks sequentially in the same working directory.

Tasks are stateless by default. Kestra will launch each task within a temporary working directory on a Worker. The WorkingDirectory task allows reusing the same file system's working directory across multiple tasks so that multiple sequential tasks can use output files from previous tasks without having to use the outputs.taskId.outputName syntax. Note that the WorkingDirectory only works with runnable tasks because those tasks are executed directly on the Worker. This means that using flowable tasks such as the Parallel task within the WorkingDirectory task will not work.

Examples

Clone a Git repository into the Working Directory and run a Python script in a Docker container.

yaml
id: git_python
namespace: company.team

tasks:
  - id: wdir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/examples
        branch: main

      - id: python
        type: io.kestra.plugin.scripts.python.Commands
        taskRunner:
          type: io.kestra.plugin.scripts.runner.docker.Docker
        containerImage: ghcr.io/kestra-io/pydata:latest
        commands:
          - python scripts/etl_script.py

Add input and output files within a Working Directory to use them in a Python script.

yaml
    id: api_json_to_mongodb
    namespace: company.team

    tasks:
      - id: wdir
        type: io.kestra.plugin.core.flow.WorkingDirectory
        outputFiles:
          - output.json
        inputFiles:
          query.sql: |
            SELECT sum(total) as total, avg(quantity) as avg_quantity
            FROM sales;
        tasks:
          - id: inline_script
            type: io.kestra.plugin.scripts.python.Script
            taskRunner:
              type: io.kestra.plugin.scripts.runner.docker.Docker
            containerImage: python:3.11-slim
            beforeCommands:
              - pip install requests kestra > /dev/null
            warningOnStdErr: false
            script: |
              import requests
              import json
              from kestra import Kestra

              with open('query.sql', 'r') as input_file:
                  sql = input_file.read()

              response = requests.get('https://api.github.com')
              data = response.json()

              with open('output.json', 'w') as output_file:
                  json.dump(data, output_file)

              Kestra.outputs({'receivedSQL': sql, 'status': response.status_code})

      - id: load_to_mongodb
        type: io.kestra.plugin.mongodb.Load
        connection:
          uri: mongodb://host.docker.internal:27017/
        database: local
        collection: github
        from: "{{ outputs.wdir.uris['output.json'] }}"
yaml
id: working_directory
namespace: company.team

tasks:
  - id: working_directory
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: first
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
        - 'echo "{{ taskrun.id }}" > {{ workingDir }}/stay.txt'
      - id: second
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
        - |
          echo '::{"outputs": {"stay":"'$(cat {{ workingDir }}/stay.txt)'"}}::''

A working directory with a cache of the node_modules directory.

yaml
id: node_with_cache
namespace: company.team

tasks:
  - id: working_dir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    cache:
      patterns:
        - node_modules/**
      ttl: PT1H
    tasks:
      - id: script
        type: io.kestra.plugin.scripts.node.Script
        beforeCommands:
          - npm install colors
        script: |
          const colors = require("colors");
          console.log(colors.red("Hello"));

Properties

cache

Cache configuration.

When a cache is configured, an archive of the files denoted by the cache configuration is created at the end of the execution of the task and saved in Kestra's internal storage. Then at the beginning of the next execution of the task, the archive of the files is retrieved and the working directory initialized with it.

errors

  • Type: array
  • SubType: Task
  • Dynamic: ❌
  • Required: ❌

List of tasks to run if any tasks failed on this FlowableTask.

inputFiles

  • Type:
    • object
    • string
  • Dynamic: ✔️
  • Required: ❌

The files to create on the local filesystem. It can be a map or a JSON object.

namespaceFiles

Inject namespace files.

Inject namespace files to this task. When enabled, it will, by default, load all namespace files into the working directory. However, you can use the include or exclude properties to limit which namespace files will be injected.

outputFiles

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

The files from the local filesystem to send to Kestra's internal storage.

Must be a list of glob) expressions relative to the current working directory, some examples: my-dir/, my-dir/*/ or my-dir/my-file.txt.

tasks

  • Type: array
  • SubType: Task
  • Dynamic: ❌
  • Required: ❌

Definitions

io.kestra.core.models.tasks.NamespaceFiles

  • enabled
    • Type: boolean
    • Dynamic: ❌
    • Required: ❌
    • Default: true
  • exclude
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌
  • include
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌

io.kestra.plugin.core.flow.WorkingDirectory-Cache

  • patterns
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ✔️
  • ttl
    • Type: string
    • Dynamic: ❌
    • Required: ❌
    • Format: duration