PutRecords

yaml
type: "io.kestra.plugin.aws.kinesis.PutRecords"

Send multiple records to Amazon Kinesis Data Streams.

Examples

Send multiple records as maps to Amazon Kinesis Data Streams. Check the following AWS API reference for the structure of the PutRecordsRequestEntry request payload.

yaml
id: aws_kinesis_put_records
namespace: company.team

tasks:
  - id: put_records
    type: io.kestra.plugin.aws.kinesis.PutRecords
    accessKeyId: "<access-key>"
    secretKeyId: "<secret-key>"
    region: "eu-central-1"
    streamName: "mystream"
    records:
      - data: "user sign-in event"
        explicitHashKey: "optional hash value overriding the partition key"
        partitionKey: "user1"
      - data: "user sign-out event"
        partitionKey: "user1"

Send multiple records from an internal storage ion file to Amazon Kinesis Data Streams.

yaml
id: aws_kinesis_put_records
namespace: company.team

tasks:
  - id: put_records
    type: io.kestra.plugin.aws.kinesis.PutRecords
    accessKeyId: "<access-key>"
    secretKeyId: "<secret-key>"
    region: "eu-central-1"
    streamName: "mystream"
    records: kestra:///myfile.ion

Properties

`failOnUnsuccessfulRecords`

Type: boolean
Dynamic: ❌
Required: ✔️
Default: true

Mark the task as failed when sending a record is unsuccessful.

If true, the task will fail when any record fails to be sent.

`records`

Type:
- string
- array
Dynamic: ✔️
Required: ✔️

List of records (i.e., list of maps) or internal storage URI of the file that defines the records to be sent to AWS Kinesis Data Streams.

A list of at least one record with a map including data and partitionKey properties (those two are required arguments). Check the PutRecordsRequestEntry API reference for a detailed description of required fields.

`accessKeyId`

Type: string
Dynamic: ✔️
Required: ❌

Access Key Id in order to connect to AWS.

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

`compatibilityMode`

Type: boolean
Dynamic: ✔️
Required: ❌

`endpointOverride`

Type: string
Dynamic: ✔️
Required: ❌

The endpoint with which the SDK should communicate.

This property allows you to use a different S3 compatible storage backend.

`region`

Type: string
Dynamic: ✔️
Required: ❌

AWS region with which the SDK should communicate.

`secretKeyId`

Type: string
Dynamic: ✔️
Required: ❌

Secret Key Id in order to connect to AWS.

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

`sessionToken`

Type: string
Dynamic: ✔️
Required: ❌

AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource.

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

`streamArn`

Type: string
Dynamic: ✔️
Required: ❌

The ARN of the stream to push the records.

Make sure to set either streamName or streamArn. One of those must be provided.

`streamName`

Type: string
Dynamic: ✔️
Required: ❌

The name of the stream to push the records.

Make sure to set either streamName or streamArn. One of those must be provided.

`stsEndpointOverride`

Type: string
Dynamic: ✔️
Required: ❌

The AWS STS endpoint with which the SDKClient should communicate.

`stsRoleArn`

Type: string
Dynamic: ✔️
Required: ❌

AWS STS Role.

The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.