How data moves
agent → gateway → Kafka → collector → ClickHouse → core → web. Every stage scales on its own.
Capture and query are separate. Writes do not block reads.
One data path
The agent captures on the host and sends over gRPC or OTLP to the gateway. The gateway handles auth, rate limiting and protocol conversion, then writes to Kafka. The collector consumes and writes columnar data into ClickHouse. Core serves the API and query orchestration; the web only presents.
Each stage scales independently: gateways are stateless, collectors scale by partition, ClickHouse by shard.
| Component | Role | Default port |
|---|---|---|
| agent | Capture, protocol parsing, local aggregation | outbound gRPC |
| gateway | Auth, rate limiting, OTLP and gRPC ingress | 9999 / 4317 / 4318 |
| Kafka | Buffering and decoupling | 9092 |
| collector | Consume and write columnar | 7777 |
| ClickHouse | Columnar storage and query | 8123 |
| core | API, query orchestration, permissions | 7005 |
| web | Interface | 80 |
Why split it this way
Capture must not stall because queries are slow, and queries must not time out because writes spike. Kafka separates the two, and collectors scale as needed.
In an offline deployment the whole path lives inside the bundle, with no external service.
Deployment shapes
- Standalone: every component on one machine, for evaluation and edge
- Performance cluster: capture, storage and query scale separately
- HA cluster: replicas, failover, rolling upgrades
Bring your system in
Tell us what you run: OS, kernel, public network or not, virtual machines or clusters. We will map the path in.