Tutorial: Shipping Docker Metrics to Logz.io

• Home
• Blog
• DevOps
• Tutorial: Shipping Docker Metrics to Logz.io

Docker is an essential bridge in modern DevOps. Despite Kubernetes overtaking Docker on orchestrating containers, the Docker container itself remains the standard and likely will for the foreseeable future. We developed the Docker Metrics collector to operate as its own container that will run Metricbeat using the modules you are running in real time. Now, in addition to the Docker module, we are now releasing an AWS module for operations in the cloud.

The docker module collects the following metrics: container , cpu , diskio , healthcheck , info , memory , and network .

In an earlier tutorial, we covered shipping Docker metrics through Metricbeat to Elasticsearch and on to Kibana. This walkthrough will work with the Logzio Docker Metrics Collector and shipping straight to Logz.io.

In order to install the Docker Metrics Collector, stop your container and add docker to the LOGZIO_MODULES environment variable. At that point, restart.

Docker Metrics Configuration

Pull the Docker image. Download the Docker Metrics Collector image:

docker pull logzio/docker-collector-metrics

Run the container.

For a complete list of options, see the parameters below the code block.:point_down:

docker run --name docker-collector-metrics \
--env LOGZIO_TOKEN="<<SHIPPING-TOKEN>>" \
--env LOGZIO_URL="<<LISTENER-HOST>>" \
--env LOGZIO_MODULES="docker" \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
logzio/docker-collector-metrics

Module Parameters

There are 16 parameters you can (or must) set in the module. The first two on the list are mandatory, otherwise it won’t work:

1. 1. LOGZIO_TOKEN —This is your Logz.io account token. Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.
   2. LOGZIO_URL —This is the Logz.io listener host to ship the metrics to. Replace <<LISTENER-HOST>> with your region’s listener host (for example, listener.logz.io). For more information on finding your account’s region, see Account region.
   3. LOGZIO_MODULES —This is a comma-separated list of Metricbeat modules to be enabled on this container (formatted as module1,module2,module3 ). To use a custom module configuration file, mount its folder to /logzio/logzio_modules .
   4. LOGZIO_TYPE —In this instance, it will be something like docker-collector-metrics . The log type you’ll use with this Docker. This is shown under the type field in Kibana. Logz.io applies parsing based on type .
   5. LOGZIO_LOG_LEVEL —This is the log level the module startup scripts will generate.
   6. LOGZIO_ADDITIONAL_FIELDS —A semicolon-separated list of additional fields to be included with each message sent (formatted as fieldName1=value1;fieldName2=value2 ). To use an environment variable as a value, format as fieldName=$ENV_VAR_NAME . Environment variables must be the only value in the field. Where an environment variable can’t be resolved, the field is omitted.
   7. DOCKER_MATCH_CONTAINER_NAME —This is a comma-separated list of containers you want to collect the metrics from. If a container’s name partially matches a name on the list, that container’s metrics are shipped. Otherwise, its metrics are ignored. Note: This can’t be used with DOCKER_SKIP_CONTAINER_NAME .
   8. DOCKER_SKIP_CONTAINER_NAME —This is a comma-separated list of containers you want to ignore. If a container’s name partially matches a name on the list, that container’s metrics are ignored. Otherwise, its metrics are shipped. Note: This be used with DOCKER_MATCH_CONTAINER_NAME .
   9. DOCKER_PERIOD —Sampling rate of metrics. The Docker API takes up to 2 seconds to respond, so we recommend setting this to 3s or longer.
   10. DOCKER_CERTIFICATE_AUTHORITY —This is the file path to certificate authority for connecting to Docker over TLS.
   11. DOCKER_CERTIFICATE —This is the file path to CA certificate for connecting to Docker over TLS.
   12. DOCKER_KEY —This is the file path to Docker key for connecting to Docker over TLS.

It will take a few minutes for the metrics to appear in our system. Beforehand, you should have a dashboard ready to go. If you want to customize a dashboard but need to go with something to display things in the meantime, we have a pre-fab set of dashboards for different platforms in Logz.io, including Docker.