Manager UI and connector configuration

The Manager UI is a small browser-based Visual Pipeline Builder. It generates the same JSON document accepted by POST /api/v1/versions; the Manager validates and serializes that document into the Data Plane YAML artifact.

Open the UI at http://127.0.0.1:8091/ after starting the Manager as described in Control Plane and Manager. See the Connector reference for the complete input/output matrix and the boundary between UI-configurable and YAML-configured connectors.

UI sections

Pipeline

Set the pipeline name and match expression. The default match expression is true, which is useful for a local smoke test. Use an expression appropriate to the event shape in a real configuration.

Sequential transformations

Add transformations in the order in which they should run:

• Grok Parsing — parses a source field with a Grok pattern;
• Data Masking — applies configured regular-expression masking rules;
• Expr Filter — evaluates an expression;
• Keep — retains selected fields;
• Drop — removes selected fields.

Use the up/down controls to change order and Remove to delete a step. Confirm the resulting order in Live Preview.

Routes

Create any number of routes. Each route has a name, condition and one or more checked destinations. Routes can fan out to multiple outputs. The Manager serializes the compatible Data Plane keys routes and route emitters; these names are not changing in the YAML/API contract.

The Set up Front Door HIT split button seeds two destinations and two mutually exclusive example routes. It does not fill in tenant-specific Azure resource values. Verify actual field spelling and value casing from your Front Door events before saving a production configuration.

Destinations / Outputs

Add multiple destinations and select each destination's type. The UI currently exposes:

UI type	Required fields	Data Plane settings
Azure Log Analytics (DCR)	Ingestion endpoint, immutable rule ID, stream, and an authentication mode (Client secret, Managed Identity or AKS Workload Identity)	sentinel_dcr.dcr
Azure Blob Storage	Account URL, container	azure_blob.cloud
Splunk HEC	Endpoint, Splunk token	auth
AWS S3	Bucket	cloud
AWS SQS	Queue URL	cloud
GCP Cloud Storage	Bucket	cloud
GCP Pub/Sub	Project ID, subscription	cloud
Kafka	Brokers (comma or newline separated), topic	brokers, topic
RabbitMQ	AMQP/AMQPS URL and queue or exchange; routing key optional	rabbitmq
SQL	Driver, table, DSN, insert query	sql
JSONL file	Path	emitter path

Batch size and flush interval are available for each destination. RabbitMQ emits persistent JSON messages and only marks an event accepted after a positive publisher confirm; negative or ambiguous confirms follow the retry/DLQ path. Azure Blob output uses the Data Plane's Azure SDK default credential chain (for example, managed identity). DCR authentication can use Client secret, Managed Identity or AKS Workload Identity. Client-secret mode requires tenant ID, client ID and client secret; pass production secrets as environment placeholders such as ${ENV:AZURE_CLIENT_SECRET}, which the Data Plane expands at load time. Managed Identity mode omits all client-secret fields; leave the optional managed identity client ID empty for a system-assigned identity, or set it to select a user-assigned identity. AKS Workload Identity mode also omits all client-secret fields and resolves the federated identity settings (client ID, tenant ID and federated token file) from the AKS workload identity webhook environment, with the same optional client ID override. The selected identity must be attached to (or federated for) the Data Plane workload (not the Manager) and have the required Azure Monitor Logs Ingestion permissions on the DCR. Avoid saving real production credentials into Manager versions or sharing previews containing credentials. For RabbitMQ, reference the broker URL through an environment variable (for example ${RABBITMQ_URL}) and do not persist production credentials in Manager versions.

Inputs / Sources

The form supports multiple individual cloud, Kafka-compatible and RabbitMQ inputs at once, plus optional global HTTP and Syslog listeners. JSON schema/sample generation is local to the browser and does not select a source. The Manager preserves the Data Plane YAML contract: individual inputs serialize under ingestors, while listeners serialize under global.http and global.syslog.

Input	Required fields	YAML location
Azure Blob / Event Grid	Name, account URL, container	ingestors[].storage (optional webhook listen/path)
Azure Event Hubs (Kafka API)	Name, brokers, topic, consumer group	ingestors[]
Kafka-compatible	Name, brokers, topic, consumer group	ingestors[]
RabbitMQ	Name, AMQP/AMQPS URL, queue	ingestors[].rabbitmq
AWS S3	Name, bucket	ingestors[].cloud
AWS SQS	Name, queue URL	ingestors[].cloud
GCP GCS	Name, bucket	ingestors[].cloud
GCP Pub/Sub	Name, project ID, subscription	ingestors[].cloud
HTTP listener	Listen address	global.http (timeouts and max body size are optional)
Syslog listener	TCP and/or UDP listen address	global.syslog (message size and multiline options are optional)

Add HTTP or Syslog through the global-listener buttons; they are not ingestors. The Syslog form exposes TCP/UDP, message size and multiline settings supported by the runtime. TLS Syslog is intentionally omitted because it is not wired by the current runtime server. Configure provider credentials and transport security through the Data Plane deployment environment; for RabbitMQ, supply the broker URL via environment interpolation (for example ${RABBITMQ_URL}) rather than storing credentials in a Manager version. Do not enter secrets into sample/schema fields.

A source definition and destinations are independent: schema/sample preview does not select or configure the input source.

Front Door HIT routing walkthrough

This example sends Front Door cache HIT events to Azure Blob Storage and all non-HIT events to a Sentinel Log Analytics DCR. “All” here means all events where cacheStatus is not exactly the string Hit; confirm this semantics against real records and handle missing/null values if needed.

1. In JSON sample → schema & table, paste representative Front Door access log JSON (one object or an array) and choose Infer schema and preview. This schema/table remains in the browser; it is not included in the saved YAML.
2. Optionally choose fields and generate a KEEP transform. Add any parser/masking steps needed and review their order. The input source is configured separately.
3. Click Set up Front Door HIT split under Routes. This creates cacheStatus == "Hit" to frontdoor-hit-blob and cacheStatus != "Hit" to frontdoor-other-dcr.
4. Under Destinations / Outputs, fill the Blob account URL and container. Grant the Data Plane identity the required Blob data-plane role on the storage account/container.
5. Fill the DCR ingestion endpoint, immutable rule ID and stream name. Select Managed Identity to use an identity attached to the Data Plane workload (omit the client ID for system-assigned identity, or enter the client ID of an attached user-assigned identity), select AKS Workload Identity for pods running on AKS with workload identity enabled (the Data Plane then resolves the webhook-injected AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE, and the optional client ID override selects a specific federated identity), or select Client secret and provide the Entra tenant ID, client ID and an environment reference such as ${ENV:AZURE_CLIENT_SECRET}. Grant the Data Plane identity or service principal the required DCR ingestion role. The Manager identity is not used to send events to Azure Monitor.
6. Add the desired source under Inputs / Sources. Configure an Azure Blob/Event Grid, Event Hubs/Kafka, AWS or GCP input as appropriate, or add global HTTP/Syslog listeners; these are independent of the JSON sample.
7. Review the YAML preview and validation status. Ensure each route references its intended output and the routing condition matches the actual field/value in your sample.
8. Click Create configuration version. This saves a version only; it does not perform a rollout. Inspect the stored version and explicitly initiate a rollout later, after review and testing.

The DCR stream/table schema must be compatible with the fields emitted by the pipeline. Configure the corresponding custom table and DCR transformation in Azure before enabling production delivery.

Live Preview and validation

Live Preview shows the YAML representation of the current form. The preview is generated in the browser for immediate feedback; the Manager performs the authoritative serialization and validation when the version is saved.

The Create configuration version button remains disabled when required fields are missing. Examples include:

• missing Splunk endpoint or token;
• missing cloud bucket;
• missing SQS queue URL;
• missing Pub/Sub project or subscription;
• missing SQL table or insert query;
• missing ingestor name/type or required cloud fields;
• missing Kafka input brokers, topic or consumer group;
• missing Kafka destination brokers or topic;
• a global listener setting without its listen address.

Always review the preview before saving. The artifact intentionally has no runtime version key; the version is tracked by the Manager database.

Save and review a version

1. Configure the pipeline, transformations, route and emitter.
2. Review the Live Preview.
3. Click Create configuration version.
4. Confirm the saved version ID and SHA-256 value.
5. Click Refresh versions and inspect the history.
6. Roll out only after reviewing the stored YAML and registering the intended Data Plane node.

The UI also displays registered nodes and their latest rollout state.

Local smoke test

From the repository root, use a disposable database and a non-production port:

cd slimstream-manager
go run ./cmd/manager -db /tmp/logstrm-manager-test.db -listen :8091

Then:

1. Open http://127.0.0.1:8091/.
2. Select JSONL file and use /tmp/logstrm-events.jsonl.
3. Add one transformation, such as Add Keep.
4. Confirm the YAML preview changes.
5. Clear the required path and verify that saving is blocked.
6. Restore the path and save a version.
7. Verify the result with:

curl -s http://127.0.0.1:8091/api/v1/health
curl -s http://127.0.0.1:8091/api/v1/versions
curl -s http://127.0.0.1:8091/api/v1/nodes

Expected results are {"status":"ok"} for health and a collection of saved versions after the first successful save. An empty versions or nodes collection is not an error.

What this test does not prove

Saving a version proves UI rendering, client-side validation, Manager serialization and SQLite persistence. It does not prove delivery to Splunk, AWS, GCP or SQL. End-to-end connector testing requires a reachable test destination, credentials supplied through the deployment environment and an enabled Data Plane node. Destination coverage remains separate: inputs/listeners added here do not imply that every Data Plane output has a Manager form. See the Connector reference for the current distinction between Data Plane runtime capability and Manager UI coverage.