Blueprints

Run a Python script and generate outputs, metrics and files specified with a variable

Source

yaml
id: python-generate-outputs
namespace: company.team

variables:
  file: orders.csv

tasks:
  - id: analyze_orders
    type: io.kestra.plugin.scripts.python.Script
    warningOnStdErr: false
    beforeCommands:
      - pip install faker kestra > /dev/null
    outputFiles:
      - "*.csv"
    script: |
      import csv
      import random
      import time
      from faker import Faker
      from kestra import Kestra

      start_time = time.time()

      fake = Faker()

      # list of columns for the CSV file
      columns = [
          "order_id",
          "customer_name",
          "customer_email",
          "product_id",
          "price",
          "quantity",
          "total",
      ]

      filename = "{{ vars.file }}"
      tags = {'file': filename}

      # Generate 100 random orders
      orders = []
      for i in range(100):
          order_id = i + 1
          customer_name = fake.name()
          customer_email = fake.email()
          product_id = random.randint(1, 20)
          price = round(random.uniform(10.0, 200.0), 2)
          quantity = random.randint(1, 10)
          total = round(price * quantity, 2)
          orders.append(
              [order_id, customer_name, customer_email, product_id, price, quantity, total]
          )

      # Write the orders to a CSV file
      with open(filename, "w", newline="") as file:
          writer = csv.writer(file)
          writer.writerow(columns)
          writer.writerows(orders)

      # Calculate and print the sum and average of the "total" column
      total_sum = sum(order[6] for order in orders)
      average_order = round(total_sum / len(orders), 2)
      print(f"Total sum: {total_sum}")
      print(f"Average Order value: {average_order}")

      Kestra.outputs({"total_sum": total_sum, "average_order": average_order})
      Kestra.counter('total_sum', total_sum, tags)
      Kestra.counter('average_order', average_order, tags)

      end_time = time.time()  
      processing_time = end_time - start_time
      Kestra.timer('processing_time', processing_time, tags)

      print(f"The script execution took: {processing_time} seconds")

About this blueprint

Metrics pip Python Software Engineering Outputs

This flow generates a CSV file with 100 random orders and then calculates the sum and average of the "total" column. It then reports the results as outputs and metrics. The CSV file generated by a Python task is set as outputFiles, allowing you to download the file from the UI's Execution page. It is helpful to share the results of your workflow with business stakeholders who can download the file from the UI and use it in their processes. To avoid hardcoding values, the filename orders.csv is specified as a variable.

Script

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra