Apache Pulsar monitoring with Netdata
Apache Pulsar is an open-source distributed pub-sub messaging system.
This module will monitor one or more Apache Pulsar instances, depending on your configuration.
It collects broker statistics from the prometheus endpoint.
pulsar module is tested on the following versions:
- v2.5.0
Charts#
It produces the following charts:
Summary#
- Broker Components in
num - Messages Rate in
messages/s - Throughput Rate in
KiB/s - Storage Size in
KiB - Messages Backlog Size in
messages - Storage Write Latency Histogram in
entries/s - Entry Size Histogram in
entries/s - Subscriptions Delayed for Dispatching in
message batches
If exposeTopicLevelMetricsInPrometheus is set to true:
- Subscriptions Redelivered Message Rate in
messages/s - Subscriptions Blocked On Unacked Messages in
subscriptions
If replication is configured and replicationMetricsEnabled is set to true:
- Replication Rate in
messages/s - Replication Throughput Rate in
messages/s - Replication Backlog in
messages
Namespace#
- Broker Components in
num - Messages Rate in
messages/s - Throughput Rate in
KiB/s - Storage Size in
KiB - Storage Read/Write Operations Rate in
message batches/s - Messages Backlog Size in
messages - Storage Write Latency Histogram in
entries/s - Entry Size Histogram in
entries/s - Subscriptions Delayed for Dispatching in
message batches
If exposeTopicLevelMetricsInPrometheus is set to true:
- Subscriptions Redelivered Message Rate in
messages/s - Subscriptions Blocked On Unacked Messages in
subscriptions
If replication is configured and replicationMetricsEnabled is set to true:
- Replication Rate in
messages/s - Replication Throughput Rate in
messages/s - Replication Backlog in
messages
Topic#
Topic charts are only available when exposeTopicLevelMetricsInPrometheus is set to true. In addition, you need to
set topic_filer configuration option. If you have a lot of topics this is highly unrecommended.
- Producers in
producers - Subscriptions in
producers - Consumers in
producers - Publish Messages Rate in
publishes/s - Dispatch Messages Rate in
dispatches/s - Publish Throughput Rate in
KiB/s - Dispatch Throughput Rate in
KiB/s - Storage Size in
KiB - Storage Read Rate in
message batches/s - Storage Write Rate in
message batches/s - Messages Backlog Size in
messages - Subscriptions Delayed for Dispatching in
message batches - Subscriptions Redelivered Message Rate in
messages/s - Subscriptions Blocked On Unacked Messages in
blocked subscriptions
If replication is configured and replicationMetricsEnabled is set to true:
- Topic Replication Rate From Remote Cluster in
messages/s - Topic Replication Rate To Remote Cluster in
messages/s - Topic Replication Throughput Rate From Remote Cluster in
KiB/s - Topic Replication Throughput Rate To Remote Cluster in
KiB/s - Topic Replication Backlog in
KiB/s
Configuration#
Edit the go.d/pulsar.conf configuration file using edit-config from the
Netdata config directory, which is typically at /etc/netdata.
Needs only url to server's /metrics endpoint. Here is an example for 2 servers:
For all available options please see module configuration file.
Topic filtering#
By default module collects data for all topics but it supports topic filtering. Filtering doesnt exclude a topic stats from the summary/namespace stats, it only removes the topic from the topic charts.
To check matcher syntax see matcher documentation.
Update every#
Module default update_every is 60.
Apache Pulsar doesnt expose raw counters, it exposes rate. It counts rates every statsUpdateFrequencyInSecs. Default
value is 60 seconds.
Module update_every should be equal to statsUpdateFrequencyInSecs.
Troubleshooting#
To troubleshoot issues with the pulsar collector, run the go.d.plugin with the debug option enabled. The output
should give you clues as to why the collector isn't working.
First, navigate to your plugins directory, usually at /usr/libexec/netdata/plugins.d/. If that's not the case on your
system, open netdata.conf and look for the setting plugins directory. Once you're in the plugin's directory, switch
to the netdata user.
You can now run the go.d.plugin to debug the collector: