Skip to content

Production hardening

A default installation is not a hardened installation. This checklist lists the controls an operator must apply, and the verification that shows each one is actually in effect.

Identity and access

  • For multi-tenant installations, use a dedicated namespace/release, Manager database, OIDC client, reload token, TLS Secrets and Data Plane identity per tenant/trust domain. Do not rely on Manager node roles as tenant isolation; current rollouts send one selected version to every enabled node in that Manager.
  • Configure Manager OIDC (manager.oidc.issuer and manager.oidc.clientID in Helm, or SLIMSTREAM_OIDC_ISSUER and SLIMSTREAM_OIDC_CLIENT_ID for direct deployments); require authentication with SLIMSTREAM_OIDC_REQUIRED=true. Both direct startup and Helm require OIDC by default. The explicit SLIMSTREAM_ALLOW_UNAUTHENTICATED_LOOPBACK=true escape hatch is for isolated local development only; never proxy or tunnel that mode. Verify with an unauthenticated request, which must be rejected.
  • Map identity-provider groups to Viewer, Operator and Admin, and keep Admin membership minimal. Verify by attempting an Admin-only operation with an Operator token.
  • Use auth_mode: managed_identity or auth_mode: workload_identity for Sentinel. Verify that no client_secret appears in any configuration version.
  • Scope Monitoring Metrics Publisher to the specific Data Collection Rule, not the subscription or resource group. Verify with a role-assignment listing at that scope.
  • Issue a distinct Data Plane reload token per node and record its rotation owner.
  • Keep Live Tail disabled unless needed. Require API authentication, leave raw event capture off, and verify anonymous ticket/WebSocket requests fail. See Live Tail.
  • Configure the Blob/Event Grid webhook bearer delivery header and scope the Azure identity to its configured container. Verify foreign account/container notifications are rejected. See Blob ingestion.
  • Set bounded emitter queues for the workload and verify overload handling at sources. HTTP 202 is volatile acceptance; 503 may follow partial delivery. See Delivery contract.

Network exposure

  • Keep the Data Plane API (logstrm:8090) on its private ClusterIP; the Helm chart enables TLS there and on global HTTP ingestion by default. Provision the server/client TLS Secrets and verify the Manager validates the certificate and hostname.
  • Register Data Plane nodes only with HTTPS URLs, hostname-matching server certificates and TLS verification enabled. The Manager rejects HTTP/unverified node registrations and rollouts.
  • Terminate TLS at an ingress or reverse proxy for the Manager UI/API; the Manager listener itself does not terminate TLS. Verify the browser endpoint negotiates TLS and rejects plaintext.
  • Keep the Manager off the public internet. Expose it through a private endpoint, VPN or identity-aware proxy.
  • Keep the Data Plane API Service internal (ClusterIP, enforced by the chart). If external HTTP ingestion needs a direct address, expose only dataplane.ingestService.type=LoadBalancer and restrict source networks.
  • Configure a hot-reload token from a Kubernetes Secret. The Helm chart now rejects missing or conflicting token configuration, and Secret references are required by both workloads; never rely on an unauthenticated reload endpoint.
  • Keep Syslog TCP/UDP private; it has no in-process TLS. Use a TLS relay for untrusted networks. TCP admission is capped at 1,024 connections with a two-minute idle read deadline; reconnect after idle closure. Shutdown closes active connections before waiting for readers.
  • Enable the chart's NetworkPolicy (networkPolicy.enabled=true), which is disabled by default, and restrict ingress to the Manager and egress to the required destinations. Verify that a denied path actually fails.
  • Reject insecure_skip_verify and plain http:// endpoints for the Elasticsearch emitter in configuration review.
  • Configure Splunk HEC only with HTTPS endpoints; keep its token in a runtime Secret referenced as an environment variable, never as a literal Manager-version value.
  • Do not use the generic type: kafka connector over untrusted networks; it currently has no TLS/SASL security configuration. Use the Azure Event Hubs connector only with its TLS plus Entra OAuth mode, or add verified Kafka security configuration before using a generic broker.

Private connectivity to Azure Monitor

Private connectivity is a property of the deployment, not a Logstrm feature. Logstrm sends HTTPS to whatever Data Collection Endpoint it is given, so it inherits whatever path the network provides.

  • Place the Data Plane in the customer VNet or an AKS cluster with controlled egress.
  • Configure an Azure Monitor Private Link Scope with the Data Collection Endpoint and the workspace, and validate that private DNS resolves the ingestion hostname to the private endpoint address from inside the workload.
  • Confirm ingestion still succeeds with public network access disabled on the relevant resources, and record that test as the evidence.
  • Inventory the dependencies that remain outside the private path, such as Entra token endpoints, the container registry and the customer's identity provider, and permit them explicitly in egress rules.

Do not state "Private Link supported" without the resolution and ingestion evidence above; the distinction between private data-plane traffic and authentication egress is the one assessors check.

Data at rest

  • Place the DLQ directory, any JSONL archive and the Manager database on encrypted volumes, with customer-managed keys where policy requires them.
  • Apply the same encryption and access control to volume backups and snapshots.
  • Configure masking or field removal in the pipeline so that buffered payloads contain no unnecessary sensitive data.
  • Set DLQ retention and archive rotation to match the data-classification policy, and monitor DLQ growth.
  • When the Event Hubs durable outbox is enabled, use a dedicated encrypted PVC per tenant, keep the configured payload cap well below PVC capacity, monitor free space/outbox gauges, test restore, and never share the SQLite PVC between replicas.

See Data protection and encryption for what the product does and does not encrypt.

Workload and supply chain

  • Keep the chart's container defaults: non-root user, read-only root filesystem, no privilege escalation, all capabilities dropped, RuntimeDefault seccomp, automountServiceAccountToken: false and resource limits. Verify on the running pod specification rather than on values files alone.
  • Deploy images by digest, not by mutable tag.
  • Verify the Cosign signature and SLSA provenance of every digest before promotion, as described in Image signatures and build provenance.
  • Restrict registry pull credentials to the images required.

Operations

  • Review configuration changes before rollout; treat configuration as production code and keep the previous version available for rollback.
  • Ship Manager and Data Plane logs and metrics to monitoring, and alert on delivery failure, DLQ growth and authentication errors. See Observability.
  • Test restoration of the Manager database and confirm which configuration state the restore produces.
  • Define ownership for vulnerability remediation, image rebuilds and upgrade cadence.
  • Record residual risks that this checklist cannot remove, such as the absence of application-level encryption and of in-process TLS, and obtain explicit acceptance for them.

Validation scope

Published Azure identity validations are controlled functional smoke tests: they demonstrate that an authentication mode successfully delivers a schema-matching event. They are not throughput, resilience, failover or production-readiness certifications, and they do not replace the customer's own acceptance testing in the target environment.

Ingestion, output and DLQ limits

Global HTTP ingestion allows four concurrent decode/route operations. Excess requests receive 503 and Retry-After: 1. The configured request byte limit still applies (default 10 MiB); arrays exceeding 1,000 events receive 413 before routing. Send smaller batches and retry overload responses with backoff; retries after partial emitter admission can duplicate deliveries. Heap expansion and active output queues mean these limits are not a 50 MB memory guarantee.

Sentinel DCR, Splunk HEC/ACK, Elasticsearch and HTTP JSON output clients reject redirects and have 15-second request deadlines. Use final URLs, including the final token endpoint for client-secret authentication. SDK-managed identity acquisition requires separate cloud validation. Review retry budgets alongside pod termination grace periods.

DLQ sealing requires a filesystem supporting hard links and directory fsync. Startup rejects non-regular active/sealed paths and preserves conflicting files instead of replacing sealed history. An interrupted seal with active/sealed aliases for the same inode recovers automatically. Keep the directory private and ensure one writer per directory. File-count retention can still evict undelivered records; provision retention and capacity from measured serialized event size and outage duration. A bounded DLQ alone is not a lossless delivery guarantee.

Profiling is disabled unless observability.enable_profiling: true. When enabled, pprof listens only on 127.0.0.1:6060; do not expose it through ingress or a shared proxy.