Export metrics to Google Cloud Pub/Sub Service

Prerequisites

To use the Pub/Sub service for metric collecting and processing, you should first install Google Cloud Platform C++ Proto Libraries. Pub/Sub support is also dependent on the dependencies of those libraries, like protobuf, protoc, and grpc. Next, Netdata should be re-installed from the source. The installer will detect that the required libraries are now available.

You cannot compile Netdata with Pub/Sub support enabled using grpc 1.32 or higher.

Some distributions don't have .cmake files in packages. To build the C++ Proto Libraries on such distributions we advise you to delete protobuf, protoc, and grpc related packages and install grpc with its dependencies from source.

Configuration

To enable data sending to the Pub/Sub service, run ./edit-config exporting.conf in the Netdata configuration directory and set the following options:

[pubsub:my_instance]

enabled = yes

destination = pubsub.googleapis.com

credentials file = /etc/netdata/google_cloud_credentials.json

project id = my_project

topic id = my_topic

Set the destination option to a Pub/Sub service endpoint. pubsub.googleapis.com is the default one.

Next, create the credentials JSON file by following Google Cloud's authentication guide. The user running the Agent (typically netdata) needs read access to google_cloud_credentials.json, which you can set with chmod 400 google_cloud_credentials.json; chown netdata google_cloud_credentials.json. Set the credentials file option to the full path of the file.