Configure instrument connectors
A connector defines how results reach FlaskTrack. Choose the instrument and its assigned agent, then set the source and the matching server parser. Every collector in this agent is compiled Rust. No Python, Node.js, interpreter, or helper package installation is required.
Using the guided connection form
Choose the connection type first. FlaskTrack shows the source fields and settings relevant to that type. The form keeps the saved settings when you switch types.
- File watcher / Network share: choose the folder, file pattern, stability delay, and source-file handling.
- REST / Vendor HTTP: set the endpoint, authentication environment references, and optional result-array/pagination pointers.
- SFTP: select key-file or password-variable authentication and enter the verified SSH host fingerprint.
- Serial: select port, baud rate, framing, and result format.
- MQTT: enter topics one per line, delivery quality, broker credentials, and TLS files.
- SDK / Custom: provide the installed adapter path and literal arguments.
The guided size field uses MiB; FlaskTrack saves the corresponding byte count in Settings JSON. CSV column mappings use add/remove rows instead of requiring you to write an object by hand. JSON pointers describe where a value is located in each result record: /well selects the well field.
Advanced integration settings keeps the full JSON object available. Existing keys outside the guided fields are retained, but arbitrary keys do not add agent features. Edit the JSON and leave the field to refresh the guided controls. Invalid JSON must be corrected before saving.
Switching vendor mode to HTTP clears the executable settings; changing SFTP authentication clears the other authentication method. These explicit choices prevent stale settings from selecting the wrong transport.
Saving validates required fields and basic formats in the browser. A valid form does not confirm remote connectivity or parser compatibility. Complete the agent checks below.
Connection types
| Type | Implementation | Source fields |
|---|---|---|
| Manual upload | Operator workflow; no automatic collector | Parser/profile chosen by upload flow |
| File watcher | Existing native file scan | watch_path, file_pattern |
| Network share | Existing native file scan | OS-mounted/authenticated watch_path, file_pattern |
| REST API | Native HTTP(S) polling, raw response or paginated JSON records | base_url = endpoint |
| Vendor API | Native HTTP polling or an installed executable adapter | base_url or settings.executable |
| SFTP | Native SSH/SFTP with pinned host-key verification | sftp://host[:port], remote watch_path, file_pattern |
| Serial | Persistent native serial-port worker | settings.port and framing options |
| MQTT | Persistent native MQTT 3.1.1 subscription worker | mqtt://host or mqtts://host, topics |
| Opentrons | Existing native Robot Server result capture | Robot Server base_url |
| Chromeleon SDK | Native supervision/ingestion of an installed SDK exporter | settings.executable |
| Custom | Native executable adapter contract | settings.executable |
Chromeleon SDK APIs are vendor-specific and proprietary. This package supplies the agent-side SDK-exporter contract and supervision, not a licensed SDK or an implementation of a particular Chromeleon SDK version. Your installed exporter must use that SDK and satisfy the contract below. The native demo exporter is a protocol example, not a Chromeleon integration.
For every automatic collector, assign the agent with source access. File paths refer to that host, not the FlaskTrack cloud server. Inbound is supported. Bidirectional collects only its inbound leg; this update does not send device commands, start protocols, or push worklists. Outbound-only configurations are not ingestible.
Parser configuration
A collected artifact still needs a compatible server parser. Configure parser key/version on the connector and an enabled matching parser profile. The agent configuration resolves an enabled matching profile, preferring an instrument-specific match over a shared profile. Avoid multiple active profiles with the same instrument, parser key, and version. Collecting a new source format does not automatically install its server parser.
For CSV, use generic.csv.long.v1, version 1, and a profile matching the exported headers. JSON/raw vendor files require their own deployed parser or transformation. Live Opentrons capture is JSON; the virtual simulator's CSV configuration does not parse live Robot Server artifacts.
Connector Settings JSON controls collection. Parser-profile Configuration JSON controls column interpretation. The top-level credential_reference field is not delivered/resolved by this agent; use the documented host environment references.
Common settings for the new collectors
| Key | Type/default | Meaning |
|---|---|---|
| timeout_seconds | integer 20, range 1–300 | Source request/adapter timeout; MQTT initial startup deadline |
| max_file_bytes | integer 67108864, range 1–2147483648 | Captured artifact and relevant response/frame size bound |
| source_filename | basename string | Artifact name; no path separators permitted |
| content_type | string | Optional raw MIME type override |
| output_format | json default, or csv | JSON record output format |
| csv_columns | object | CSV header → JSON pointer; required for CSV projection |
JSON pointers use /field/nested, array indexes, and ~1/~0 escaping. Missing CSV pointer values become blank cells; the server parser still validates required result fields. Projection preserves configured columns only: include run/sample IDs, units, timestamps, and event context when needed.
Default filenames: JSON results use result.json; CSV projections use result.csv; raw HTTP uses http-result.bin; raw serial uses serial-result.txt; raw MQTT uses mqtt-result.bin. Override filenames to match your parser.
Manual upload
No worker is started. The agent health describes this as operator-driven. Use the existing application upload surface; choosing this type does not create one or bypass API authentication. Settings JSON is {}.
File watcher and Network share
Use an accessible path such as C:\Reader\Exports, /var/lib/flasktrack/inbox, \\LAB-NAS\Results, or /mnt/lab-results. Network shares must already be mounted/authenticated for the service account. Patterns match filenames with * and ?; multiple patterns are separated by ; or ,.
| Key | Default/type | Behavior |
|---|---|---|
| recursive | false / boolean | Scan nested directories; symlinks and .flasktrack-* directories skipped |
| settle_seconds | 2 / integer | Stable size/mtime time; existing collector caps at 300 seconds |
| max_file_bytes | 2147483648 / integer | Per-file limit |
| source_file_action | leave / string | leave, archive, or delete after upload receipt |
| archive_dir | .flasktrack-archive / string | Relative to watch root or absolute |
| archive_processed_files | false / boolean | Legacy archive request, overridden by source_file_action |
These existing collectors and their source-cleanup behavior are preserved. Upload receipt is not parser acceptance. Use leave while validating retention requirements; a file may later quarantine. Keep custom archive directories and agent state outside recursively watched folders to avoid re-ingestion. Collection settings do not supply parser support for every file extension.
file_watcher settings
network_share settings
REST API
Set base_url to the complete initial endpoint. The collector supports GET or POST query endpoints. HTTPS uses certificate verification; redirects are refused. Pagination next links must retain the original URL origin.
| Key | Default/type | Meaning |
|---|---|---|
| endpoint_url | base_url / string | Optional complete endpoint override |
| http_method | GET / string | GET or POST; use side-effect-free query endpoints |
| request_json | {} / JSON | POST query body |
| headers | object | Nonsensitive header values |
| header_env | object | Header name → host environment variable |
| bearer_token_env | string | Host variable containing bearer token |
| records_pointer | string | JSON array to capture record-by-record |
| id_pointer | string | Identity within each record |
| next_pointer | string | Next-page URL or null; pointer must exist in response |
| max_pages | 100 / integer, 1–10000 | Bound a pagination traversal |
Without records_pointer/next_pointer, capture the raw bounded response. Record polling restarts from the initial page each cycle and deduplicates records by identity/content. A changed record produces a new artifact. There is no server-specific incremental cursor protocol. Cycles and cross-origin links fail health. Previously staged pages remain durable.
Example base_url: https://reader.example/results.
{
"records_pointer": "/data",
"next_pointer": "/next",
"id_pointer": "/id",
"bearer_token_env": "READER_TOKEN",
"max_pages": 100,
"output_format": "csv",
"source_filename": "results.csv",
"csv_columns": {
"run_id": "/runId",
"position": "/well",
"measurement": "/metric",
"value": "/reading",
"unit": "/unit"
}
}
Vendor API
Uses the same HTTP options as REST when executable is absent. With executable, it uses the native adapter contract. You must supply the actual vendor endpoint/payload schema; choosing a vendor type does not infer a proprietary protocol.
Example: base_url points to a CSV export endpoint:
{
"headers": {
"Accept": "text/csv"
},
"header_env": {
"X-API-Key": "VENDOR_API_KEY"
},
"source_filename": "vendor-export.csv",
"content_type": "text/csv"
}
SFTP
Set base_url to sftp://host[:port], watch_path to the remote directory, and file_pattern to the export pattern. The agent implements SSH internally; it does not invoke system ssh/sftp commands.
| Key | Default/type | Meaning |
|---|---|---|
| username | required string | SSH username |
| host_key_sha256 | required string | Independently verified OpenSSH SHA256 host-key fingerprint |
| password_env | string | Environment variable holding password |
| private_key_file | string | Local private-key path; used instead of password auth when set |
| private_key_passphrase_env | string | Environment variable for encrypted private key |
| recursive | false / boolean | Remote nested-directory traversal |
| settle_seconds | 2 / integer, 0–300 | Minimum age of remote mtime before read |
| max_files_per_poll | 1000 / integer | Maximum newly captured files per cycle; resume unseen files on the next cycle |
| session_timeout_seconds | 120 / integer, 1–600 | Whole-session deadline |
Pin the fingerprint after verifying it through an independent channel. There is no auto-trust/accept-any-host switch. Remote size and mtime are compared before/after read; changed files are deferred. Symlinks are not followed. Source files are left remotely, and source archive/delete settings do not apply. Completed downloads remember path/size/mtime so bounded batches progress rather than re-reading the first batch. An in-place change that preserves both size and mtime is not detected; use unique export names. Clock skew can affect the mtime-age check.
{
"username": "labreader",
"host_key_sha256": "SHA256:REPLACE_WITH_VERIFIED_HOST_FINGERPRINT",
"private_key_file": "/etc/flasktrack-agent/id_ed25519",
"recursive": false,
"settle_seconds": 10,
"session_timeout_seconds": 120
}
Serial
A dedicated Rust worker keeps the serial port open between polling ticks. Choose a delimiter-terminated protocol; incomplete frames remain buffered while connected, with a bounded size. This collector does not send initialization/query commands.
| Key | Default/type | Meaning |
|---|---|---|
| port | required string | COM3, /dev/ttyUSB0, or appropriate host device |
| baud_rate | 9600 / integer | Serial speed |
| data_bits | 8 / integer | 5–8 |
| parity | none / string | none, even, odd |
| stop_bits | 1 / integer | 1 or 2 |
| flow_control | none / string | none, hardware, software |
| delimiter | required string | Frame terminator, such as newline or CRLF |
| frame_format | raw default / string | raw or json |
| id_pointer | string | JSON frame's stable event identity |
DTR is requested off on open, but some drivers/devices may still transiently toggle it. Doctor opens/closes the port; use it only when that is safe for your instrument. Native exclusive port access is used. Raw frames receive session/sequence identities so repeated values remain distinct. JSON can use id_pointer for stable event identities. Incomplete frames are lost if the connection/worker restarts; include timestamps/event IDs where possible.
{
"port": "COM3",
"baud_rate": 9600,
"data_bits": 8,
"parity": "none",
"stop_bits": 1,
"flow_control": "none",
"delimiter": "\n",
"frame_format": "json",
"id_pointer": "/eventId",
"output_format": "csv",
"source_filename": "serial.csv",
"csv_columns": {
"run_id": "/runId",
"position": "/position",
"measurement": "/measurement",
"value": "/value",
"unit": "/unit"
}
}
MQTT
Set base_url to mqtt://host[:port] or mqtts://host[:port]. A Rust subscription worker stays connected; QoS 1 messages are acknowledged only after durable local capture. Broker persistence and service lifetime remain part of delivery guarantees.
| Key | Default/type | Meaning |
|---|---|---|
| topics | required string array | At most 128 subscription filters |
| qos | 1 / integer | 0 or 1; QoS 2 is rejected |
| client_id | flasktrack-connector-ID / string | Stable unique broker identity |
| keepalive_seconds | 60 / integer | 5–3600 seconds |
| username | string | Optional broker account |
| password_env | string | Host variable containing broker password |
| ca_file | required for mqtts / path | Trusted PEM CA bundle |
| client_cert_file + client_key_file | optional paired paths | PEM TLS client credentials |
| payload_format | json default / string | json or raw |
| id_pointer | string | Event ID within JSON payload |
All subscription acknowledgments must succeed before health is Online. Invalid JSON or unresolved IDs fail collection without acknowledging that message. There is no separate MQTT poison-message quarantine. Workers reconnect after failure; no distributed exactly-once guarantee is made. Identical raw bytes/topic or JSON without distinct event IDs may deduplicate; use unique event IDs for repeated measurements.
{
"topics": [
"lab/reader/results"
],
"client_id": "flasktrack-reader-01",
"qos": 1,
"username": "labreader",
"password_env": "MQTT_PASSWORD",
"ca_file": "C:\\FlaskTrack\\broker-ca.pem",
"payload_format": "json",
"id_pointer": "/eventId",
"output_format": "csv",
"csv_columns": {
"run_id": "/runId",
"position": "/position",
"measurement": "/measurement",
"value": "/value",
"unit": "/unit"
}
}
Opentrons
The existing native collector is preserved. Base URL is the Robot Server, for example http://robot.local:31950. It captures terminal succeeded/failed/stopped runs as JSON. Discovery and command retrieval currently fetch one page each; historical pagination is not implemented. It does not execute protocols or send worklists.
Settings: opentrons_version string default *, vendor_timeout_seconds integer default 20/minimum 5, opentrons_page_length integer default 50/range 1–200. Use a parser for the captured Robot Server JSON wrapper, not generic CSV.
Chromeleon SDK and Custom
These collectors supervise an installed native executable through a bounded stdin/stdout contract. The executable implements the instrument/vendor SDK calls; it may be written in Rust or use the vendor-supported SDK runtime. A supported vendor runtime is a vendor dependency, not a new requirement for the agent itself. A licensed SDK exporter is not bundled here.
| Key | Type/default | Meaning |
|---|---|---|
| executable | required absolute path | Installed adapter executable |
| args | string array, default [] | Literal arguments, with no shell expansion |
| timeout_seconds | integer 20 | Check/collection deadline |
| max_file_bytes | integer 67108864 | Per-artifact bound; output envelope is capped at twice this size |
Adapters run with the service account's permissions. Restrict connector editing to authorized administrators. Timeouts kill the directly spawned process; the adapter must manage descendants and close its output pipes. Diagnostics on stderr are not forwarded as potentially secret-containing health messages.
chromeleon_sdk settings
custom settings
Native adapter contract
Read a single JSON object from stdin:
The operation is check or collect. Check must verify source/prerequisites without modifying the instrument. Exit nonzero on failure. Write JSON only on stdout:
{"ok":true,"artifacts":[{"id":"run-001","filename":"results.csv","content_type":"text/csv","payload_base64":"...","metadata":{"source":"sdk"}}]}
Artifact IDs must be stable and nonempty; filenames must be basenames. Base64 content is size-checked and handed to the normal spool. The supplied examples/demo_exporter.rs is a native Rust contract demonstration:
Point a Custom connector at the resulting absolute executable path for a demo. It does not implement a proprietary SDK.
Operation and diagnostics
The existing CLI/config/service setup is unchanged. Credential environment variables must be supplied to the daemon service account. doctor performs source checks, including MQTT subscription acceptance; REST POST checks issue a POST, so use read-only query endpoints. Source reachability is not parser acceptance.
New persistent collectors start during once; it waits for initial source health, drains captured artifacts into the existing spool, uploads, sends heartbeat, and exits. Use run for continuous Serial/MQTT reception. A Serial/MQTT connection that receives no message can still be healthy.
Configuration refresh cancels replaced/removed workers and clears stale schedules/health. A replacement waits for the previous worker to finish before opening the same source. Source timeouts bound HTTP/SSH/adapter operations; native streams check cancellation frequently. Inbound data already captured/spooled may still upload after disabling a connector.
Intermediate native captures live under state/captures. They contain bytes, checksums, stable identities, and the collection's parser/instrument configuration snapshot. Once committed into the existing spool, their payload/manifests are removed; .seen markers preserve polling deduplication. Do not manually delete state to retry uploads. The existing API retry/dead-letter behavior applies. No total disk quota or capture-history compaction is implemented; monitor free space.
Keep configuration, captures, spool, and local credentials accessible only to the service account. Report/pipeline queries consume structured results after successful server parsing and mapping; this source update does not change reporting schema exposure.