HF RL Explorer

Pulsar stream ingestion: support extracting message properties (headers), record keys, and selected metadata

Pulsar stream ingestion: support extracting message properties (headers), record keys, and selected metadata: a task in LegoFlow-SWE (Harbor dataset). Currently, the Pulsar connector does not expose message properties (headers) or extract Pulsar‑specific metadata (message key, message id, etc.)…

The task

Currently, the Pulsar connector does not expose message properties (headers) or extract Pulsar‑specific metadata (message key, message id, etc.) when `metadata.populate` is enabled. This prevents users from ingesting these values as regular table columns, unlike other stream connectors.

Part of Lego-X/LegoFlow-SWE.