Read-only observability

Kafka Pipeline

Loading pipeline state…

Kafka — broker reachability

Market — server-owned state

Topics — configured / observed

Consumer lag — max(0, end − committed)

Messages/minute — derived from offsets

Last pipeline activity — offset movement

Pipeline topology

How DSEverse data moves

Market Scraper, Market Normalizer and ClickHouse Sink are one service each, shared by both pipelines. Producer edges are labelled "Configured producer": offset movement proves topic activity but not a friendly producer identity. Consumer groups are verified.

Allowlisted topics only

Topics

Topic	Partitions	Latest offset	Messages	Rate (msg/min)	Last activity	Status

Allowlisted groups only

Consumer groups

Group	State	Members	Assigned partitions	Total lag	Status

Meaning & limitations

About this view

Read-only: this page never creates or deletes topics, alters offsets, resets groups, or consumes message payloads.
Offsets and lag prove activity; they do not identify which application produced a message. Producer edges are therefore labelled "Configured producer".
Only the DSEverse allowlist of topics and consumer groups is shown; no internal broker addresses, credentials, client ids, or hosts are exposed.
Message rates are derived from offset deltas between samples and can under-report bursts between polls.
While the market is closed the page stops full sampling: one snapshot on load, then only a low-frequency status check until the next session.