Netdata v1.21.0

Release v1.21.0 contains 2 new collectors, 3 new exporting connectors, 37 bug fixes, 46 improvements, and 25 documentation updates. We also made 26 bug fixes or improvements related to the upcoming release of Netdata Cloud.

At a glance

We added a new collector for Apache Pulsar, a popular open-source distributed pub-sub messaging system. We use Pulsar in our Netdata Cloud infrastructure (more on that later this month!), and are excited to start sharing metrics about our own Pulsar systems when the time comes. The Pulsar collector attempts to auto-detect any running Pulsar processes, but you can always configure the collector based on your setup.

Also new in v1.21 is a VerneMQ collector. We use the open-source MQ Telemetry Transport (MQTT) broker for Netdata Cloud as well. As with Pulsar, you can configure the VerneMQ collector to auto-detect your installation in just a few steps.

Our experimental exporting engine received significant updates with new connectors for Prometheus remote write, MongoDB, and AWS Kinesis Data Streams. You can now send Netdata metrics to more than 20 additional external storage providers for long-term archiving and deeper analysis. Learn more about the exporting engine in our documentation.

We upgraded our TLS compatibility to include 1.3, which applies to HTTPS for both Netdata’s web server and streaming connections. TLS 1.3 is the most up-to-date version of the TLS protocol, and contains important fixes and improvements to ensure strong encryption. If you enabled TLS in the web server or streaming, Netdata attempts to use 1.3 by default, but you can also set the version and ciphers explicitly. Learn more in the documentation.

The Netdata dashboard has been completely re-written in React. While the look and behavior hasn’t changed, these under-the-hood changes enable a suite of new features, UX improvements, and design overhauls. With React, we’ll be able to work faster and better resource our talented engineers.

As part of the ongoing work to polish our eBPF collector tech preview, we’ve now proven the collector’s performance is very good, and have vastly expanded the number of operating system versions the collector works on. Learn how to enable it in our documentation. We’ve also extensively stress-tested the eBPF collector and found that it’s impressively fast given the depth of metrics it collects! Read up on our benchmarking analysis on GitHub.

Acknowledgments

Jiab77 for helping remove extra printed \n in various installation methods.
SamK for fixing missing folders in /var/ for .deb installations.
kevenwyld for improving Netdata’s support of RHEL distributions.
WoozyMasta for adding in the ability to get Kubernetes pod names with kubectl in bare-metal deployments.
paulmezz for adding the ability to to connect to non-admin user IDs when trying to collect metrics from a Ceph storage cluster.
ManuelPombo for adding additional charts to our Postgres collector, and anayrat for helping review the changes.
Default for adding lsyncd to the backup group in apps.plugin.
bceylan, peroxy, toadjaune, grinapo, m-rey, and YorikSar for documentation fixes.

Breaking changes

None.

Improvements

Extended TLS support for 1.3. (#8505) by thiagoftsm
Switched to the React dashboard code as the default dashboard. (#8363) by Ferroin

Collectors

Added a new Pulsar collector. (#8364) by ilyam8
Added a new VerneMQ collector. (#8236) by ilyam8
Added high precision timer support for plugins such as idlejitter. (#8441) by mfundul
Added an alarm to the dns_query collector that detects DNS query failure. (#8434) by ilyam8
Added the ability to get the pod name from cgroup with kubectl in bare-metal deployments. (#7416) by WoozyMasta
Added the ability to connect to non-admin user IDs for a Ceph storage cluster. (#8276) by paulmezz
Added connections (backend) usage to Postgres monitoring. (#8126) by ManuelPombo
eBPF: Added support for additional Linux kernels found in Debian 10.2 and Ubuntu 18.04. (#8192) by thiagoftsm

Packaging/installation

Added missing override for Ubuntu Eoan. (#8547) by prologic
Added Docker build arguments to pass extra options to Netdata installer. (#8472) by Ferroin
Added deferred error message handling to the installer. (#8381) by Ferroin
Fixed cosmetic error checking for CentOS 8 version in install-required-packages.sh. (#8339) by prologic
Added various fixes and improvements to the installers. (#8315) by Ferroin
Migrated to installing only Python 3 packages during installation. (#8318) by Ferroin
Improved support for RHEL by not installing the CUPS plugin when v1.7 of CUPS cannot be installed. (#7216) by kevenwyld
Added support for Clear Linux in install-required-packages.sh. (#8154) by Ferroin
Removed Fedora 29 from CI and packaging. (#8100) by Ferroin
Removed Ubuntu 19.04 from CI and packaging. (#8040) by Ferroin
Removed OpenSUSE Leap 15.0 from CI. (#7990) by Ferroin

Exporting

Added a MongoDB connector to the exporting engine. (#8416) by vlvkobal
Added a Prometheus Remote Write connector to the exporting engine. (#8292) by vlvkobal
Added an AWS Kinesis connector to the exporting engine. (#8145) by vlvkobal

Documentation

Fixed typo in main README.md. (#8547) by bceylan
Updated the update instructions with per-method details. (#8394) by joelhans
Updated paragraph on install-required-packages.sh. (#8347) by prologic
Added Patti’s dashboard video to the documentation. (#8385) by joelhans
Fixed go.d modules in the COLLECTORS.md. (#8380) by ilyam8
Added frontmatter to all documentation in bulk. (#8354) and (#8372) by joelhans
Fixed MDX parsing in installation guide. (#8362) by joelhans
Fixed typo in eBPF documentation. (#8360) by ilyam8
Fixed links in packaging/installer to work on GitHub and docs. (#8319) by joelhans
Fixed typo in main README.md. (#8335) by peroxy
Removed mention saying that .deb packages are experimental. (#8250) by toadjaune
Added standards for abbreviations/acronyms to docs style guide. (#8313) by joelhans
Tweaked eBPF documentation, and added performance data. (#8261) by joelhans
Added requirements for the exim collector. (#8096) by petarkozic
Fixed misspelling of openSUSE and SUSE. (#8233) by m-rey
Added OpenGraph tags to documentation pages. (#8224) by joelhans
Fixed typo in custom dashboard documentation. (#8213) by shortpatti
Removed extra asterisks in main README. (#8193) by grinapo
Added eBPF README to documentation navigation and improved page title. (#8191) by joelhans
Fixed figure+image without closing tag in new documentation. (#8177) by joelhans
Corrected instructions for running Netdata behind Apache. (#8169) by cakrit
Added PR title guidelines to the contribution guidelines to make CHANGELOG.md more meaningful. (#8150) by cakrit
Fixed formatting in Custom dashboards documentation. (#8102) by YorikSar
Updated the manual install documentation with better information about CentOS 6. (#8088) by Ferroin
Added tutorials to support v1.20 release (#7943) by joelhans

CI/CD

Added logic to bail early on LWS build if cmake is not present. (#8559) by Ferroin
Added python.d configuration files to YAML linting CI process and increase line limit to 120 characters. (#8541) and (#8542) by ilyam8
Cleaned up GitHub Actions workflows. (#8383) by Ferroin
Migrated tests from Travis CI to Github Workflows. (#8331) by prologic
Covered install-required-packages.sh with Coverity scan. (#8388) by prologic
Added support for cross-host docker-compose builds. (#7754) by amoss
Reconfigured Travis CI to retry transient failures on lifecycle tests. (#8203) by prologic
Switched to checkout@v2 in GitHub Actions. (#8170) by ilyam8

Other

Added lsyncd to the backup group in apps.plugin. (#8159) by Default

Netdata Cloud

Fixed compiler warnings in the claiming code. (#8567) by vlvkobal
Fixed regressions in cloud functionality (build, CI, claiming). (#8568) by underhood
Switched over to soft feature flag. (#8545) by amoss
Improved claiming behavior to run as netdata user by default, or override if necessary. (#8516) by amoss
Updated the info endpoint for Cloud notifications. (#8519) by amoss
Added correct error logging for ACLK challenge/response. (#8538) by stelfrag
Cleaned up Cloud configuration files to move [agent_cloud_link] settings to [cloud]. (#8501) by underhood
Enhanced ACLK header payload to include timestamp-offset-usec. (#8499) by stelfrag
Added ACLK build failures to anonymous statistics. (#8429) by underhood
Added ACLK connection failures to anonymous statistics. (#8456) by underhood
Added HTTP proxy support to ACLK. (#8406)/(#8418) by underhood
Improved ownership of the claim.d directory. (#8475) by amoss
Fixed the ACLK response payload to match the new specification. (#8420) by stelfrag
Added the new cloud info in the info endpoint. (#8430) by amoss
Implemented ACLK Last Will and Testament. (#8410) by stelfrag
Fixed JSON parsing in ACLK. (#8426) by stelfrag
Fixed outstanding problems in claiming and add SOCKS5 support. (#8406)/(#8404) by amoss and underhood
Fixed the type value for alarm updates in the ACLK. (#8403) by stelfrag
Improved performance of ACLK. (#8399)/(#8401) by amoss
Improved the ACLK’s agent “pop-corning” phase. (#8398) by stelfrag
Improved ACLK according to results of the smoke-test. (#8358) by amoss and underhood
Added code to bundle LWS in binary packages. (#8255) by Ferroin
Added libwebsockets files to make dist. (#8275) by Ferroin
Adapted the claiming script to new API responses. (#8245) by hmoragrega
Fixed claiming script to reflect Netdata Cloud API changes. (#8220) by cosmix
Added libwebsockets bundling code to netdata-installer.sh. (#8144) by Ferroin

Bug fixes

Removed notifications from the dashboard and fixed the /default.html route. (#8599 by jacekkolasa
Fixed help-tooltips styling, private registry node deletion, and the right-hand sidebar “jumping” on document clicks. (#8553 by jacekkolasa
Fixed errors reported by Coverity. (#8593) by thiagoftsm, (#8579) by amoss, and (#8586) by thiagoftsm
Added netdata.service.* to .gitignore to hide system/netdata.service.v235 file. (#8556) by vlvkobal
Fixed Debian 8 (Jessie) support. (#8590) and (#8593) by prologic
Fixed broken Fedora 30/31 RPM builds. (#8572) by prologic
Fixed broken pipe ignoring in apps.plugin. (#8554) by vlvkobal
Fixed the bytespersec chart context in the Python Apache collector. (#8550) by ilyam8
Fixed charts.d.plugin to exit properly during Netdata service restart. (#8529) by ilyam8
Fixed minimist dependency vulnerability. (#8537) by jacekkolasa
Fixed our Debian/Ubuntu packages to package the expected systemd unit files. (#8468) by prologic
Fixed auto-updates for static (kickstart-static64.sh) installs. (#8507) by prologic
Fixed openSUSE 15.1 RPM package builds. (#8494) by prologic
Fixed how SimpleService truncates Python module names. (#8492) by ilyam8
Removed erroneous \n in uninstaller output. (#8446) by prologic
Fixed install-required-packages script to self-update apt. (#8491) by prologic
Added proper prefix to Python module names during loading. (#8474) by ilyam8
Fixed how the Netdata updater script cleans up after being run. (#8414) by prologic
Fixed the flushing error threshold with the database engine. (#8425) by mfundul
Fixed memory leak for host labels streaming from slaves to master. (#8460) by thiagoftsm
Fixed support for uninstalling the eBPF collector in the uninstaller. (#8444) by prologic
Fixed a bug involving stop_all_netdata uv_pipe_connect() in the installer. (#8444) by prologic
Fixed installer output regarding newlines. (#8447) by prologic
Fixed broken dependencies for Ubuntu 19.10. (#8397) by prologic
Fixed streaming scaling. (#8375) by mfundul
Fixed missing characters in kernel version field by encoding slave fields. (#8216) by thiagoftsm
Fixed installation for Ubuntu 14.04 (#7690) by Ehekatl
Fixed dependencies for Debian Jessie. (#8290) by Ferroin
Fixed dependency names for Arch Linux. (#8334) by Ferroin
Removed extra printed \n in various installers. (#8324)/(#8325)/(#8326) by Jiab77
Fixed missing folders in /var/ for .deb packages. (#8314) by SamK
Fixed Ceph collector to get osd_perf_infos in versions 14.2 and higher. (#8248) by ilyam8
Fixed RHEL / CentOS 8.x dependencies for Judy-devel and others.(#8202) by prologic
Removed extraneous commas from chart information in dashboard. (#8266) by FlyingSixtySix
Removed tmem collection from xenstat_plugin to allow Netdata on Xen 4.13 to compile successfully. (#7951) by rushikeshjadhav
Fixed get_latest_version for nightly channel update script. (#8172) by ilyam8
Restricted messages to Google Analytics. (#8161) by thiagoftsm
Fixed Python 3 dict access in OpenLDAP collector module. (#8162) by Mic92