Difference between revisions of "Netdata"
(add same rules to the IPv6 networking stack) |
(→HAProxy: need SSL cert) |
||
| Line 23: | Line 23: | ||
<source lang="python"> | <source lang="python"> | ||
frontend netdata | frontend netdata | ||
| − | bind *:20000 | + | bind *:20000 ssl crt /etc/haproxy/certs/wiki.freephile.org.pem |
mode http | mode http | ||
default_backend netdata-back | default_backend netdata-back | ||
| Line 29: | Line 29: | ||
backend netdata-back | backend netdata-back | ||
server nd1 127.0.0.1:19999 | server nd1 127.0.0.1:19999 | ||
| − | </source> | + | </source> |
=== Kernel === | === Kernel === | ||
Revision as of 09:16, 17 January 2019
Netdata will be part of the QualityBox dashboard.
See http://wiki.freephile.org:20000/
Contents
System Locations[edit | edit source]
- the daemon at /usr/sbin/netdata - config files in /etc/netdata - web files in /usr/share/netdata - plugins in /usr/libexec/netdata - cache files in /var/cache/netdata - db files in /var/lib/netdata - log files in /var/log/netdata - pid file at /var/run/netdata.pid - logrotate file at /etc/logrotate.d/netdata
Host Modifications[edit | edit source]
HAProxy[edit | edit source]
frontend netdata
bind *:20000 ssl crt /etc/haproxy/certs/wiki.freephile.org.pem
mode http
default_backend netdata-back
backend netdata-back
server nd1 127.0.0.1:19999
Kernel[edit | edit source]
You have kernel memory de-duper (called Kernel Same-page Merging, or KSM) available, but it is not currently enabled.
Memory de-duplication instructions
To enable it run:
echo 1 >/sys/kernel/mm/ksm/run echo 1000 >/sys/kernel/mm/ksm/sleep_millisecs
If you enable it, you will save 40-60% of netdata memory.
Ports[edit | edit source]
netdata by default listens on all IPs on port 19999. We add a rule to firewalld to allow 20000 and then pass that port through to the backend in haproxy config.
http://this.machine.ip:20000/ => http://127.0.0.1:19999
Start/Stop[edit | edit source]
To stop netdata run:
systemctl stop netdata
To start netdata run:
systemctl start netdata
Installation extras[edit | edit source]
The configuration will send messages to 'root' so be sure to either edit the conf sudo vim /etc/netdata/health_alarm_notify.conf, or set vim /etc/aliases && newaliases
Issues[edit | edit source]
You'll probably receive alarms for 'tcp listen drops'. This is likely bot-related (sending INVALID packets) and NOT due to your application dropping legitimate packets. There is a good discussion on how to identify the source of the problem and how to mitigate or resolve it Issue #3234 Issue #3826 TLDR; increase the threshold to 1 (/etc/netdata/health.d/tcp_listen.conf) so you don't get bogus alerts.
Also, you should modify your firewall to drop invalid packets before they're either counted (by netstats) or dropped (by the kernel).
iptables -A INPUT -m conntrack --ctstate INVALID -j DROP
ip6tables -A INPUT -m conntrack --ctstate INVALID -j DROP
iptables -A INPUT -m tcp -p tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
ip6tables -A INPUT -m tcp -p tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
Following the advice from NASA at https://wiki.earthdata.nasa.gov/display/HDD/SOMAXCONN, I increased my somaxconn kernel parameter to 1024 from 128
cat /proc/sys/net/core/somaxconn
128
sysctl -w net.core.somaxconn=1024
Updates[edit | edit source]
Netdata will update itself, and puts a script into cron:
ln -s /root/netdata/netdata-updater.sh /etc/cron.daily/netdata-updater
