Control Plane and Manager

Logstrm Manager is the optional Control Plane for authoring, versioning and rolling out Data Plane configuration. The Data Plane can run independently; a Manager outage does not stop an already configured Data Plane node.

Responsibilities

The Manager provides:

• a browser-based Visual Pipeline Builder;
• configuration version history stored in SQLite;
• YAML generation compatible with the Data Plane config_v4.yaml artifact shape;
• Config v1 metadata validation and bounded version diffs;
• Data Plane node registration;
• authenticated rollouts with bounded timeouts and response sizes;
• rollback by creating a new version linked to an earlier version;
• audit logging for mutating requests.

The Manager does not process telemetry itself. It sends a versioned configuration to enabled Data Plane nodes through the rollout API. Connector coverage and the distinction between UI-configurable and YAML-configured integrations are documented in the Connector reference.

Local start

From the repository root, start the Manager:

cd slimstream-manager
go run ./cmd/manager -db /tmp/logstrm-manager-test.db -listen :8091

Open the UI at http://127.0.0.1:8091/. The default port is 8090; use another port when the Data Plane or another local service already occupies it.

The database path controls the local history. Use a persistent path for a real environment and a temporary path for disposable tests.

API endpoints

Endpoint	Method	Purpose
/api/v1/health	GET	Unauthenticated health check
/api/v1/versions	GET	List configuration versions, including schema_version metadata
/api/v1/versions	POST	Validate, serialize and store a configuration version
/api/v1/versions/{id}/diff	GET	Return a bounded diff against the parent version, or an explicit ?against={id} version
Data Plane /api/v1/config/validate	PUT	Validate candidate YAML without changing active Data Plane state
/api/v1/versions/{id}/rollback	POST	Create a new version from an earlier version
/api/v1/nodes	GET	List registered Data Plane nodes
/api/v1/nodes	POST	Register or update a node
/api/v1/nodes/{id}	DELETE	Remove a node
/api/v1/rollouts	POST	Roll out a version to enabled nodes

When OIDC is configured, all API endpoints except health require Authorization: Bearer <token>. Setting SLIMSTREAM_OIDC_ISSUER automatically enables required authentication; invalid or unavailable OIDC configuration fails closed. Admin may mutate state; Viewer may read state only.

Config v1 and version diffs

Manager configuration requests use schema version 1. Omitting schema_version selects the current version. Unsupported values are rejected with 400 Bad Request before persistence. The response and SQLite history include schema_version.

schema_version is Manager metadata only. It is deliberately excluded from the generated Data Plane config_v4.yaml artifact so existing Data Plane parsers and rollout contracts remain compatible.

A version diff response has this shape:

{
  "version_id": 2,
  "against": 1,
  "diff": "--- version-1\\n+++ version-2\\n ..."
}

Without against, the endpoint compares the requested version with its parent_id. A root version returns an empty diff. The formatter is line-oriented, deterministic and bounded; oversized input or output is rejected rather than allowing unbounded Manager CPU or memory use.

curl -s http://127.0.0.1:8091/api/v1/versions/2/diff
curl -s 'http://127.0.0.1:8091/api/v1/versions/2/diff?against=1'

Node registration and rollout

Before enabling a production rollout, use the Data Plane validation endpoint against the exact generated YAML. It parses and compiles the candidate using the same path as reload, but does not create emitters, update routing, or change the active version/hash. The endpoint uses the node's configured bearer token and returns {"valid":true,"sha256":"..."} on success.

Manager rollout performs this preflight automatically for every enabled node at /api/v1/config/validate. It verifies that the returned SHA-256 matches the stored version hash before sending the mutating /api/v1/config/reload request. A validation failure is recorded for that node and no reload is sent; other nodes continue independently. This makes rollout validation a per-node safety gate rather than an operator-only check.

Register a node with its Data Plane base URL and bearer token. Keep credentials out of shell history in production; the example uses test values only:

curl -s -X POST http://127.0.0.1:8091/api/v1/nodes \
  -H 'content-type: application/json' \
  -d '{"name":"dataplane-test","url":"http://127.0.0.1:8090","auth_token":"test-token","tls_verify":true,"enabled":false}'

Set enabled to true only when the node is ready to receive configuration. Start a rollout with a saved version ID:

curl -s -X POST http://127.0.0.1:8091/api/v1/rollouts \
  -H 'content-type: application/json' \
  -d '{"version_id":1}'

A rollout targets enabled nodes only. Each node must pass read-only validation before it receives the mutating reload request. If the request is cancelled, nodes that have not been started are returned as cancelled and are not pushed. Results are persisted with the version and node status. A response header X-Logstrm-Persistence: failed means that persistence failed after the push; inspect the Manager log before treating the stored rollout history as complete.

OIDC and roles

Configure the verifier with:

• SLIMSTREAM_OIDC_ISSUER;
• SLIMSTREAM_OIDC_CLIENT_ID;
• optional SLIMSTREAM_OIDC_ROLES_CLAIM (default roles);
• optional SLIMSTREAM_OIDC_GROUPS_CLAIM (default groups).

The exact claim values Admin and Viewer are accepted. A user with both roles retains Admin permissions. Do not paste production tokens into screenshots, tickets or documentation.

Empty collections

A new Manager database has no versions and no nodes. Depending on the current JSON serialization path, an empty collection may be displayed as null rather than []; this is not a health failure. The recommended future API refinement is to normalize empty collections to [] for a more predictable client contract.

Operational guidance

• Back up the Manager SQLite database; it contains configuration history and rollout state.
• Treat generated YAML as production configuration: review it before rollout.
• Use environment variables or deployment secrets for credentials.
• Test with disabled nodes before enabling a rollout target.
• Keep the previous working version available for rollback.