Skip to content

Pipeline SDK

The FlaskTrack Pipeline SDK lets Python and R processing blocks read their declared inputs and publish structured outputs. It is already available inside the FlaskTrack pipeline runtime; you do not need to configure credentials or connect to FlaskTrack from your script.

Use the SDK whenever a processing block needs to:

  • read an Arrow table produced by a source or earlier block;
  • read JSON configuration or metadata;
  • open a declared file input;
  • publish a table for another block or for reporting;
  • publish JSON or a generated file; or
  • add useful progress details to the execution log.

The basic pattern

Every script follows the same three-part pattern:

  1. Read each declared input by its port name.
  2. Process the data with Python or R libraries.
  3. Publish each declared output by its port name.

Python

import flasktrack as ft

rows = ft.input_table("rows")
result = rows.slice(0, 100)
ft.output_table("result", result)

R

source("/opt/flasktrack/flasktrack.R")

rows <- ft_input_table("rows")
result <- head(rows, 100)
ft_output_table("result", result)

The names rows and result must match the ports declared on the block. The input and output helper must also match the port's data type.

Important: AI-generated scripts are unreviewed drafts. Review their port names, library syntax, calculations, and output types before saving or publishing a pipeline version.

Where to go next