How to monitor your Parseable metadata in a Grafana dashbaord
As Parseable deployments in the wild are handling larger and larger volumes of logs, we needed a way to enable users to monitor their Parseable instances.
Typically this would mean setting up Prometheus to capture Parseable ingest and query node metrics and visualize those metrics on a Grafana dashboard. We added Prometheus metrics support in Parseable to enable this use case.
But we wanted a simpler, self-contained approach that allows users to monitor their Parseable instances without needing to set up Prometheus.
This led us to figuring out a way to store Parseable server's internal metrics in a special log stream called pmeta
. This stream keeps track of important information about all of the ingestors in the cluster. This includes information like the URL of the ingestor, Commit id of that ingestor, number of events processed by the ingestor, and staging file location and size.