400,000 lines, batch 1,000 × 16 workers.
Benchmark source →Keep the interface.
Scale the evidence.
Beacon adds authenticated ingest, ClickHouse storage, remote query, and alert delivery behind the Lantern TUI. Start local; add the platform when an investigation needs durable, shared history.
01 / ARCHITECTURE
A direct path, with a buffer when needed.
Direct ingest is the simplest path for a single node or modest collector. Buffered ingest absorbs bursts and ClickHouse outages in Redpanda; the loader commits only after ClickHouse accepts a batch.
Agent→Gateway→ClickHouse
Lantern TUI→Query→Sidecar→ClickHouse
OPTIONAL BUFFERGateway→Redpanda→Loader→ClickHouse
Scheduler→Alert delivery
Delivery is at least once, not exactly once. A crash can replay the last ingest batch, and retries or rare alert failover overlap can produce duplicates. Consumers and alert receivers should deduplicate. Read the delivery contract →
02 / MEASURED EVIDENCE
Numbers with their test conditions attached.
These repository benchmarks run the real gateway, query service, Python sidecar, and ClickHouse paths.
Reference environment4 vCPU, 16 GiB RAM, co-located services; single node, shared cores.Method and honest caveats →
No worse than 94 ms across the documented query classes.
Benchmark source →Cursor polling throughout the query benchmark.
Benchmark source →03 / OPERATING MODEL
Durability and isolation are explicit.
Each control is configurable; none replaces deployment-specific review.
Agent WAL
The optional on-disk write-ahead log records lines before send and replays after restart. When full, the agent stops reading instead of dropping new lines.
Durability details →Tenant resolution
Bearer tokens map to tenant IDs. The gateway server-stamps the tenant and never trusts an agent-supplied value.
Authentication details →Layered isolation
Optional per-tenant ClickHouse users and row policies provide a second isolation layer beyond server-side predicates.
Architecture decision →mTLS ingest
The gateway can require agent certificates signed by the deployment CA, and the agent can pin that CA.
TLS configuration →Prometheus signals
Services expose liveness, readiness, and metrics for ingest, queries, lag, outbox depth, dead letters, and failures.
Metrics reference →Alert outbox
The scheduler persists pending notifications. PostgreSQL supports failover; delivery remains at least once.
Outbox behavior →Kubernetes deployment
The Helm chart deploys autoscalable services, an agent DaemonSet, and scheduler, while external operators remain your responsibility.
Deployment guide →A deployment is not turnkey production-ready.
Before production, validate your environment's secret management, certificate rotation, storage-tier moves, operator upgrades, network-partition behavior, alert receiver deduplication, and live autoscaling. Repository tests cover component and failover behavior; they do not replace a deployment-specific load and recovery exercise.
Read all deployment caveats →