PluginsGoogle CloudTasksVertexai

CustomJob​Custom​Job

yaml
type: "io.kestra.plugin.gcp.vertexai.CustomJob"

Start a Vertex AI custom job.

Examples

yaml
id: gcp_vertexai_custom_job
namespace: company.team

tasks:
  - id: custom_job
    type: io.kestra.plugin.gcp.vertexai.CustomJob
    projectId: my-gcp-project
    region: europe-west1
    displayName: Start Custom Job
    spec:
      workerPoolSpecs:
      - containerSpec:
          imageUri: gcr.io/my-gcp-project/my-dir/my-image:latest
        machineSpec:
          machineType: n1-standard-4
        replicaCount: 1

Properties

delete

  • Type: boolean
  • Dynamic: ✔️
  • Required: ✔️
  • Default: true

Delete the job at the end.

displayName

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The job display name.

region

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The GCP region.

spec

The job specification.

wait

  • Type: boolean
  • Dynamic: ✔️
  • Required: ✔️
  • Default: true

Wait for the end of the job.

Allowing to capture job status & logs.

impersonatedServiceAccount

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP service account to impersonate.

projectId

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP project ID.

scopes

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: [ "https://www.googleapis.com/auth/cloud-platform" ]

The GCP scopes to be used.

serviceAccount

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP service account.

Outputs

createDate

  • Type: string
  • Required: ✔️
  • Format: date-time

endDate

  • Type: string
  • Required: ✔️
  • Format: date-time

name

  • Type: string
  • Required: ✔️

state

  • Type: string
  • Required: ✔️
  • Possible Values:
    • JOB_STATE_UNSPECIFIED
    • JOB_STATE_QUEUED
    • JOB_STATE_PENDING
    • JOB_STATE_RUNNING
    • JOB_STATE_SUCCEEDED
    • JOB_STATE_FAILED
    • JOB_STATE_CANCELLING
    • JOB_STATE_CANCELLED
    • JOB_STATE_PAUSED
    • JOB_STATE_EXPIRED
    • JOB_STATE_UPDATING
    • JOB_STATE_PARTIALLY_SUCCEEDED
    • UNRECOGNIZED

updateDate

  • Type: string
  • Required: ✔️
  • Format: date-time

Definitions

io.kestra.plugin.gcp.vertexai.models.ContainerSpec

  • imageUri
    • Type: string
    • Dynamic: ✔️
    • Required: ✔️
  • args
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • commands
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • env
    • Type: object
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.gcp.vertexai.models.CustomJobSpec

  • workerPoolSpecs
    • Type: array
    • SubType: WorkerPoolSpec
    • Dynamic: ✔️
    • Required: ✔️
    • Min items: 1
  • baseOutputDirectory
  • enableWebAccess
    • Type: boolean
    • Dynamic: ❌
    • Required: ❌
  • network
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • scheduling
  • serviceAccount
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • tensorboard
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.gcp.vertexai.models.GcsDestination

  • outputUriPrefix
    • Type: string
    • Dynamic: ❌
    • Required: ✔️

io.kestra.plugin.gcp.vertexai.models.WorkerPoolSpec

  • containerSpec
  • machineSpec
  • discSpec
  • pythonPackageSpec
  • replicaCount
    • Type: integer
    • Dynamic: ❌
    • Required: ❌

io.kestra.plugin.gcp.vertexai.models.PythonPackageSpec

  • args
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ✔️
  • envs
    • Type: object
    • SubType: string
    • Dynamic: ✔️
    • Required: ✔️
  • packageUris
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ✔️

io.kestra.plugin.gcp.vertexai.models.DiscSpec

  • bootDiskSizeGb
    • Type: integer
    • Dynamic: ❌
    • Required: ❌
    • Default: 100
  • bootDiskType
    • Type: string
    • Dynamic: ❌
    • Required: ❌
    • Default: PD_SSD
    • Possible Values:
      • PD_SSD
      • PD_STANDARD

io.kestra.plugin.gcp.vertexai.models.MachineSpec

  • machineType
    • Type: string
    • Dynamic: ✔️
    • Required: ✔️
  • acceleratorCount
    • Type: integer
    • Dynamic: ❌
    • Required: ❌
  • acceleratorType
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
    • Possible Values:
      • ACCELERATOR_TYPE_UNSPECIFIED
      • NVIDIA_TESLA_K80
      • NVIDIA_TESLA_P100
      • NVIDIA_TESLA_V100
      • NVIDIA_TESLA_P4
      • NVIDIA_TESLA_T4
      • NVIDIA_TESLA_A100
      • NVIDIA_A100_80GB
      • NVIDIA_L4
      • NVIDIA_H100_80GB
      • TPU_V2
      • TPU_V3
      • TPU_V4_POD
      • TPU_V5_LITEPOD
      • UNRECOGNIZED

io.kestra.plugin.gcp.vertexai.models.Scheduling

  • restartJobOnWorkerRestart
    • Type: boolean
    • Dynamic: ❌
    • Required: ✔️
  • timeOut
    • Type: string
    • Dynamic: ❌
    • Required: ✔️
    • Format: duration