Skip to content

Configure instrument connectors

A connector defines how results reach FlaskTrack. Choose the instrument and its assigned agent, then set the source and the matching server parser. Every collector in this agent is compiled Rust. No Python, Node.js, interpreter, or helper package installation is required.

Instrument connectors page
Connections specify the source, agent, and parser for instrument results.

Using the guided connection form

Choose the connection type first. FlaskTrack shows the source fields and settings relevant to that type. The form keeps the saved settings when you switch types.

  • File watcher / Network share: choose the folder, file pattern, stability delay, and source-file handling.
  • REST / Vendor HTTP: set the endpoint, authentication environment references, and optional result-array/pagination pointers.
  • SFTP: select key-file or password-variable authentication and enter the verified SSH host fingerprint.
  • Serial: select port, baud rate, framing, and result format.
  • MQTT: enter topics one per line, delivery quality, broker credentials, and TLS files.
  • SDK / Custom: provide the installed adapter path and literal arguments.

The guided size field uses MiB; FlaskTrack saves the corresponding byte count in Settings JSON. CSV column mappings use add/remove rows instead of requiring you to write an object by hand. JSON pointers describe where a value is located in each result record: /well selects the well field.

Advanced integration settings keeps the full JSON object available. Existing keys outside the guided fields are retained, but arbitrary keys do not add agent features. Edit the JSON and leave the field to refresh the guided controls. Invalid JSON must be corrected before saving.

Switching vendor mode to HTTP clears the executable settings; changing SFTP authentication clears the other authentication method. These explicit choices prevent stale settings from selecting the wrong transport.

Saving validates required fields and basic formats in the browser. A valid form does not confirm remote connectivity or parser compatibility. Complete the agent checks below.

Connection types

Type Implementation Source fields
Manual upload Operator workflow; no automatic collector Parser/profile chosen by upload flow
File watcher Existing native file scan watch_path, file_pattern
Network share Existing native file scan OS-mounted/authenticated watch_path, file_pattern
REST API Native HTTP(S) polling, raw response or paginated JSON records base_url = endpoint
Vendor API Native HTTP polling or an installed executable adapter base_url or settings.executable
SFTP Native SSH/SFTP with pinned host-key verification sftp://host[:port], remote watch_path, file_pattern
Serial Persistent native serial-port worker settings.port and framing options
MQTT Persistent native MQTT 3.1.1 subscription worker mqtt://host or mqtts://host, topics
Opentrons Existing native Robot Server result capture Robot Server base_url
Chromeleon SDK Native supervision/ingestion of an installed SDK exporter settings.executable
Custom Native executable adapter contract settings.executable

Chromeleon SDK APIs are vendor-specific and proprietary. This package supplies the agent-side SDK-exporter contract and supervision, not a licensed SDK or an implementation of a particular Chromeleon SDK version. Your installed exporter must use that SDK and satisfy the contract below. The native demo exporter is a protocol example, not a Chromeleon integration.

For every automatic collector, assign the agent with source access. File paths refer to that host, not the FlaskTrack cloud server. Inbound is supported. Bidirectional collects only its inbound leg; this update does not send device commands, start protocols, or push worklists. Outbound-only configurations are not ingestible.

Parser configuration

A collected artifact still needs a compatible server parser. Configure parser key/version on the connector and an enabled matching parser profile. The agent configuration resolves an enabled matching profile, preferring an instrument-specific match over a shared profile. Avoid multiple active profiles with the same instrument, parser key, and version. Collecting a new source format does not automatically install its server parser.

For CSV, use generic.csv.long.v1, version 1, and a profile matching the exported headers. JSON/raw vendor files require their own deployed parser or transformation. Live Opentrons capture is JSON; the virtual simulator's CSV configuration does not parse live Robot Server artifacts.

Connector Settings JSON controls collection. Parser-profile Configuration JSON controls column interpretation. The top-level credential_reference field is not delivered/resolved by this agent; use the documented host environment references.

Common settings for the new collectors

Key Type/default Meaning
timeout_seconds integer 20, range 1–300 Source request/adapter timeout; MQTT initial startup deadline
max_file_bytes integer 67108864, range 1–2147483648 Captured artifact and relevant response/frame size bound
source_filename basename string Artifact name; no path separators permitted
content_type string Optional raw MIME type override
output_format json default, or csv JSON record output format
csv_columns object CSV header → JSON pointer; required for CSV projection

JSON pointers use /field/nested, array indexes, and ~1/~0 escaping. Missing CSV pointer values become blank cells; the server parser still validates required result fields. Projection preserves configured columns only: include run/sample IDs, units, timestamps, and event context when needed.

Default filenames: JSON results use result.json; CSV projections use result.csv; raw HTTP uses http-result.bin; raw serial uses serial-result.txt; raw MQTT uses mqtt-result.bin. Override filenames to match your parser.

Manual upload

No worker is started. The agent health describes this as operator-driven. Use the existing application upload surface; choosing this type does not create one or bypass API authentication. Settings JSON is {}.

File watcher and Network share

Use an accessible path such as C:\Reader\Exports, /var/lib/flasktrack/inbox, \\LAB-NAS\Results, or /mnt/lab-results. Network shares must already be mounted/authenticated for the service account. Patterns match filenames with * and ?; multiple patterns are separated by ; or ,.

Key Default/type Behavior
recursive false / boolean Scan nested directories; symlinks and .flasktrack-* directories skipped
settle_seconds 2 / integer Stable size/mtime time; existing collector caps at 300 seconds
max_file_bytes 2147483648 / integer Per-file limit
source_file_action leave / string leave, archive, or delete after upload receipt
archive_dir .flasktrack-archive / string Relative to watch root or absolute
archive_processed_files false / boolean Legacy archive request, overridden by source_file_action

These existing collectors and their source-cleanup behavior are preserved. Upload receipt is not parser acceptance. Use leave while validating retention requirements; a file may later quarantine. Keep custom archive directories and agent state outside recursively watched folders to avoid re-ingestion. Collection settings do not supply parser support for every file extension.

file_watcher settings

{
  "recursive": false,
  "settle_seconds": 2,
  "source_file_action": "leave"
}

network_share settings

{
  "recursive": true,
  "settle_seconds": 10,
  "source_file_action": "leave"
}

REST API

Set base_url to the complete initial endpoint. The collector supports GET or POST query endpoints. HTTPS uses certificate verification; redirects are refused. Pagination next links must retain the original URL origin.

Key Default/type Meaning
endpoint_url base_url / string Optional complete endpoint override
http_method GET / string GET or POST; use side-effect-free query endpoints
request_json {} / JSON POST query body
headers object Nonsensitive header values
header_env object Header name → host environment variable
bearer_token_env string Host variable containing bearer token
records_pointer string JSON array to capture record-by-record
id_pointer string Identity within each record
next_pointer string Next-page URL or null; pointer must exist in response
max_pages 100 / integer, 1–10000 Bound a pagination traversal

Without records_pointer/next_pointer, capture the raw bounded response. Record polling restarts from the initial page each cycle and deduplicates records by identity/content. A changed record produces a new artifact. There is no server-specific incremental cursor protocol. Cycles and cross-origin links fail health. Previously staged pages remain durable.

Example base_url: https://reader.example/results.

{
  "records_pointer": "/data",
  "next_pointer": "/next",
  "id_pointer": "/id",
  "bearer_token_env": "READER_TOKEN",
  "max_pages": 100,
  "output_format": "csv",
  "source_filename": "results.csv",
  "csv_columns": {
    "run_id": "/runId",
    "position": "/well",
    "measurement": "/metric",
    "value": "/reading",
    "unit": "/unit"
  }
}

Vendor API

Uses the same HTTP options as REST when executable is absent. With executable, it uses the native adapter contract. You must supply the actual vendor endpoint/payload schema; choosing a vendor type does not infer a proprietary protocol.

Example: base_url points to a CSV export endpoint:

{
  "headers": {
    "Accept": "text/csv"
  },
  "header_env": {
    "X-API-Key": "VENDOR_API_KEY"
  },
  "source_filename": "vendor-export.csv",
  "content_type": "text/csv"
}

SFTP

Set base_url to sftp://host[:port], watch_path to the remote directory, and file_pattern to the export pattern. The agent implements SSH internally; it does not invoke system ssh/sftp commands.

Key Default/type Meaning
username required string SSH username
host_key_sha256 required string Independently verified OpenSSH SHA256 host-key fingerprint
password_env string Environment variable holding password
private_key_file string Local private-key path; used instead of password auth when set
private_key_passphrase_env string Environment variable for encrypted private key
recursive false / boolean Remote nested-directory traversal
settle_seconds 2 / integer, 0–300 Minimum age of remote mtime before read
max_files_per_poll 1000 / integer Maximum newly captured files per cycle; resume unseen files on the next cycle
session_timeout_seconds 120 / integer, 1–600 Whole-session deadline

Pin the fingerprint after verifying it through an independent channel. There is no auto-trust/accept-any-host switch. Remote size and mtime are compared before/after read; changed files are deferred. Symlinks are not followed. Source files are left remotely, and source archive/delete settings do not apply. Completed downloads remember path/size/mtime so bounded batches progress rather than re-reading the first batch. An in-place change that preserves both size and mtime is not detected; use unique export names. Clock skew can affect the mtime-age check.

{
  "username": "labreader",
  "host_key_sha256": "SHA256:REPLACE_WITH_VERIFIED_HOST_FINGERPRINT",
  "private_key_file": "/etc/flasktrack-agent/id_ed25519",
  "recursive": false,
  "settle_seconds": 10,
  "session_timeout_seconds": 120
}

Serial

A dedicated Rust worker keeps the serial port open between polling ticks. Choose a delimiter-terminated protocol; incomplete frames remain buffered while connected, with a bounded size. This collector does not send initialization/query commands.

Key Default/type Meaning
port required string COM3, /dev/ttyUSB0, or appropriate host device
baud_rate 9600 / integer Serial speed
data_bits 8 / integer 5–8
parity none / string none, even, odd
stop_bits 1 / integer 1 or 2
flow_control none / string none, hardware, software
delimiter required string Frame terminator, such as newline or CRLF
frame_format raw default / string raw or json
id_pointer string JSON frame's stable event identity

DTR is requested off on open, but some drivers/devices may still transiently toggle it. Doctor opens/closes the port; use it only when that is safe for your instrument. Native exclusive port access is used. Raw frames receive session/sequence identities so repeated values remain distinct. JSON can use id_pointer for stable event identities. Incomplete frames are lost if the connection/worker restarts; include timestamps/event IDs where possible.

{
  "port": "COM3",
  "baud_rate": 9600,
  "data_bits": 8,
  "parity": "none",
  "stop_bits": 1,
  "flow_control": "none",
  "delimiter": "\n",
  "frame_format": "json",
  "id_pointer": "/eventId",
  "output_format": "csv",
  "source_filename": "serial.csv",
  "csv_columns": {
    "run_id": "/runId",
    "position": "/position",
    "measurement": "/measurement",
    "value": "/value",
    "unit": "/unit"
  }
}

MQTT

Set base_url to mqtt://host[:port] or mqtts://host[:port]. A Rust subscription worker stays connected; QoS 1 messages are acknowledged only after durable local capture. Broker persistence and service lifetime remain part of delivery guarantees.

Key Default/type Meaning
topics required string array At most 128 subscription filters
qos 1 / integer 0 or 1; QoS 2 is rejected
client_id flasktrack-connector-ID / string Stable unique broker identity
keepalive_seconds 60 / integer 5–3600 seconds
username string Optional broker account
password_env string Host variable containing broker password
ca_file required for mqtts / path Trusted PEM CA bundle
client_cert_file + client_key_file optional paired paths PEM TLS client credentials
payload_format json default / string json or raw
id_pointer string Event ID within JSON payload

All subscription acknowledgments must succeed before health is Online. Invalid JSON or unresolved IDs fail collection without acknowledging that message. There is no separate MQTT poison-message quarantine. Workers reconnect after failure; no distributed exactly-once guarantee is made. Identical raw bytes/topic or JSON without distinct event IDs may deduplicate; use unique event IDs for repeated measurements.

{
  "topics": [
    "lab/reader/results"
  ],
  "client_id": "flasktrack-reader-01",
  "qos": 1,
  "username": "labreader",
  "password_env": "MQTT_PASSWORD",
  "ca_file": "C:\\FlaskTrack\\broker-ca.pem",
  "payload_format": "json",
  "id_pointer": "/eventId",
  "output_format": "csv",
  "csv_columns": {
    "run_id": "/runId",
    "position": "/position",
    "measurement": "/measurement",
    "value": "/value",
    "unit": "/unit"
  }
}

Opentrons

The existing native collector is preserved. Base URL is the Robot Server, for example http://robot.local:31950. It captures terminal succeeded/failed/stopped runs as JSON. Discovery and command retrieval currently fetch one page each; historical pagination is not implemented. It does not execute protocols or send worklists.

Settings: opentrons_version string default *, vendor_timeout_seconds integer default 20/minimum 5, opentrons_page_length integer default 50/range 1–200. Use a parser for the captured Robot Server JSON wrapper, not generic CSV.

{
  "opentrons_version": "*",
  "vendor_timeout_seconds": 20,
  "opentrons_page_length": 50
}

Chromeleon SDK and Custom

These collectors supervise an installed native executable through a bounded stdin/stdout contract. The executable implements the instrument/vendor SDK calls; it may be written in Rust or use the vendor-supported SDK runtime. A supported vendor runtime is a vendor dependency, not a new requirement for the agent itself. A licensed SDK exporter is not bundled here.

Key Type/default Meaning
executable required absolute path Installed adapter executable
args string array, default [] Literal arguments, with no shell expansion
timeout_seconds integer 20 Check/collection deadline
max_file_bytes integer 67108864 Per-artifact bound; output envelope is capped at twice this size

Adapters run with the service account's permissions. Restrict connector editing to authorized administrators. Timeouts kill the directly spawned process; the adapter must manage descendants and close its output pipes. Diagnostics on stderr are not forwarded as potentially secret-containing health messages.

chromeleon_sdk settings

{
  "executable": "C:\\FlaskTrack\\ChromeleonExporter.exe",
  "args": [],
  "timeout_seconds": 60
}

custom settings

{
  "executable": "/opt/flasktrack-agent/adapters/my-adapter",
  "args": [],
  "timeout_seconds": 20
}

Native adapter contract

Read a single JSON object from stdin:

{"operation":"collect","connector":{"id":"...","connector_type":"custom","settings":{}}}

The operation is check or collect. Check must verify source/prerequisites without modifying the instrument. Exit nonzero on failure. Write JSON only on stdout:

{"ok":true,"artifacts":[{"id":"run-001","filename":"results.csv","content_type":"text/csv","payload_base64":"...","metadata":{"source":"sdk"}}]}

Artifact IDs must be stable and nonempty; filenames must be basenames. Base64 content is size-checked and handed to the normal spool. The supplied examples/demo_exporter.rs is a native Rust contract demonstration:

cargo build --package flasktrack-agent --example demo_exporter

Point a Custom connector at the resulting absolute executable path for a demo. It does not implement a proprietary SDK.

Operation and diagnostics

The existing CLI/config/service setup is unchanged. Credential environment variables must be supplied to the daemon service account. doctor performs source checks, including MQTT subscription acceptance; REST POST checks issue a POST, so use read-only query endpoints. Source reachability is not parser acceptance.

New persistent collectors start during once; it waits for initial source health, drains captured artifacts into the existing spool, uploads, sends heartbeat, and exits. Use run for continuous Serial/MQTT reception. A Serial/MQTT connection that receives no message can still be healthy.

Configuration refresh cancels replaced/removed workers and clears stale schedules/health. A replacement waits for the previous worker to finish before opening the same source. Source timeouts bound HTTP/SSH/adapter operations; native streams check cancellation frequently. Inbound data already captured/spooled may still upload after disabling a connector.

Intermediate native captures live under state/captures. They contain bytes, checksums, stable identities, and the collection's parser/instrument configuration snapshot. Once committed into the existing spool, their payload/manifests are removed; .seen markers preserve polling deduplication. Do not manually delete state to retry uploads. The existing API retry/dead-letter behavior applies. No total disk quota or capture-history compaction is implemented; monitor free space.

Keep configuration, captures, spool, and local credentials accessible only to the service account. Report/pipeline queries consume structured results after successful server parsing and mapping; this source update does not change reporting schema exposure.