Check_MK plugin: MTR for pretty ping graphs

by BenV on Dec.31, 2015, under Check MK

Another day, another Check_MK plugin!
This one is inspired by smokeping, but different because it doesn’t need smokeping. It does need the tool formerly known as Matt’s TraceRoute, aka mtr. It’s installed on all my machines by default and easily available in all distro’s that are worthy. Even pokemon OS has it 😉

The reason I wanted to build this plugin was first of all because of pretty graphs (of course!). The second reason was that my girlfriend had some network issues to figure out, but only ping and DNS resolve times don’t paint a complete picture. This plugin makes some graphs that hopefully fill that void a bit 🙂

Now that you’ve skipped the last 2 paragraphs, here are some example graphs that I made while testing the plugin:

Check_MK MTR plugin’s perf-o-meter

This is the plugin status per host on the service overview page of Check_MK. As you can see I configured multiple hosts.

Configuration

Configuration is simple, you install the agent plugin on the machine where you want to run the pings from. Next you copy the example mtr.cfg to your Check MK agent’s configuration directory, which in my case ends up as /etc/check_mk/mtr.cfg. In this file you make a section per host you want to ping, so for example:
[mathias-kettner.de] # Options here, see the example for the default options interval = 2

Execution

Once mtr.cfg is in place the plugin will automatically start mtr for every host the next time the agent is called. Because MTR takes a relatively long time to run (default it ‘only’ does 10 pings) they’re started in the background. Once the process is complete the report will be parsed as soon as check_mk_agent is called again, after which a new MTR run is started (unless you set time=300, which will wait until 5 minutes have passed since the last run).

Results

When the results come in the alarms will be generated if necessary and pnp4nagios creates some pretty graphs:

Check_MK MTR Plugin: destination host

Above is the graph of the destination graph where you can see packet loss as black bars (scaled to match the height of your graph), and best, avg and worst ping times. I’ve tried to make it a bit ‘smokey’, but rrdgraph is quite a beast to time.
NOTE: I made these graphs for RRDTool 1.5 and higher since they use gradients, so if you have RRDTool 1.4 or older your graphs might look quite a bit less fancy or even fail completely. Sorry! 🙂
“But BenV! That graph looks like utter garbage!
Oh yeah?! Patches are welcome! 🙂

Check_MK MTR Plugin: intermediate Hop

Above graph is made for every hop in between the source and destination. They don’t have alarms, but sometimes it’s interesting to see that certain hops are slow all of a sudden.
Which is another interesting discussion: how relevant are these hops in between?
Well, since routing can give you different results for every packet they’re not THAT interesting, especially since often the hops will drop your icmp packets completely. On the other hand, if you see that all hops are suddenly slow you can draw certain conclusions from that about your own network/internet speed. (as opposed to only the destination being slow).
Just be aware that “hop 4” can be router A one time and router B the next time, just as the hopcount can vary.

Check_MK MTR Plugin – Hops Overview

This ‘hop overview’ shows all the average times for all hops including the destination host. Again, individual hops don’t mean much, but seeing all hops in a single graph together with the destination host can sometimes reveal interesting patterns. (such as a slow destination host)

WATO

Check_MK MTR Plugin: Wato settings

As with my Check MK fail2ban plugin I’ve added the part so you can edit the parameters in WATO. Here you see the default values for the plugin.
If you click on the ‘MTR’ label under “Type of check” you can create custom rules to change the parameters for a specific host, like this:

Check_MK MTR Plugin: WATO rule parameters

Note that you can disable these alarms by setting them to 0.

Download

Of course now you want my plugin, so without wasting more bytes:
V0.5.2: mtr-0.5.2.mkp (55283 downloads)      SHA1: 0e6ea842de0965612fff1e9f37b3462c55ae7ae1  MD5: c1fba27991564523ad3381b41fbd5109 Agent updated to stop failing when mtr ran twice. Will try to fix this better in a future update, but at least it doesn’t crash anymore and will after 1 failed report return to properly parsing the specific host.
V0.5.1: — fsckup on my side, I patched an older version. Use 0.5.2 instead 😉
V0.5: mtr-0.5.mkp (2274 downloads)      SHA1: 75075fd2f4cfd0c74e04d492a03abd8bf344a6d2  MD5: faad8a2f0faf66ec1c07ed280fdea09b Alarms can now actually be disabled by setting levels to 0, better compatibility with older mtr / unpexpected mtr output and copes better with mtr returning multiple lines for a hop. (only first line is parsed)
V0.4: mtr-0.4.mkp (2202 downloads)      SHA1: efc201f28163678b9ab885f81cb94749f8469066  MD5: a3e42e5ba7d01365ce9e76c13917742e Initial release

:Check_MK, mtr, ping, plugin

25 Comments for this entry

fooc
July 13th, 2016 on 10:27
Hi,

I followed your insctructions and installed the mkp, i put the config and plugin file in the right agent dirs on the server but when i rescan the localhost i cannot see the mtr check, can u help me?

Log in to Reply
BenV
July 13th, 2016 on 10:35
@fooc: when you run the check_mk_agent, can you see the <<>> section in the output? Do you have the mtr package installed on your machine? (i.e. can you run ‘mtr http://www.google.com‘ on your machine?)

The mtr check will show up per configured host, so if no correct host sections are found it won’t show up 🙂

Log in to Reply
fooc
July 13th, 2016 on 11:39
mtr is working from command line.

mtr.cfg:

# Mtr Check_MK configuration

# NOTE: your MTR report shouldn’t take longer than 15 minutes

# [DEFAULTS]
# type=icmp # icmp, tcp or udp
# count=10 # number of pings per mtr report
# force_ipv4=0 # force ipv4, exclusive with force_ipv6
# force_ipv6=0 # force ipv6, exclusive with force_ipv4
# size=64 # packet size
# time=0 # minimum time between runs, 0 / default means run if mtr doesn’t run anymore
# port=80 # UDP/TCP port to connect to
# dns=0 # Use DNS resolution to lookup addresses
# address= # Bind to source address
# interval= # time MTR waits between sending pings
# timeout= # ping Timeout, see mtr man page

[www.google.com]
type = icmp
force_ipv4 = true

[ipv6.google.com]
type = icmp
force_ipv6 = true

this is agent output:

<<>>
Version: 1.2.6p16
AgentOS: linux
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local
OnlyFrom:
<<>>
/dev/mapper/centos-root xfs 40456908 3208476 37248432 8% /
devtmpfs devtmpfs 494956 0 494956 0% /dev
tmpfs tmpfs 504684 0 504684 0% /dev/shm
tmpfs tmpfs 504684 51044 453640 11% /run
tmpfs tmpfs 504684 0 504684 0% /sys/fs/cgroup
/dev/sda1 xfs 505580 167184 338396 34% /boot
/dev/mapper/centos-home xfs 19748864 32976 19715888 1% /home
tmpfs tmpfs 504684 3000 501684 1% /opt/omd/sites/oibsite01/tmp
tmpfs tmpfs 100940 0 100940 0% /run/user/996
tmpfs tmpfs 100940 0 100940 0% /run/user/0
<<>>
[df_inodes_start]
/dev/mapper/centos-root xfs 40476672 61786 40414886 1% /
devtmpfs devtmpfs 123739 343 123396 1% /dev
tmpfs tmpfs 126171 1 126170 1% /dev/shm
tmpfs tmpfs 126171 478 125693 1% /run
tmpfs tmpfs 126171 13 126158 1% /sys/fs/cgroup
/dev/sda1 xfs 512000 337 511663 1% /boot
/dev/mapper/centos-home xfs 19759104 3 19759101 1% /home
tmpfs tmpfs 126171 531 125640 1% /opt/omd/sites/oibsite01/tmp
tmpfs tmpfs 126171 1 126170 1% /run/user/996
tmpfs tmpfs 126171 1 126170 1% /run/user/0
[df_inodes_end]
<<>>
<<>>
<<>>
/dev/mapper/centos-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
/dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
/dev/mapper/centos-home /home xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
<<>>
(root,46300,6744,00:01:13,1) /usr/lib/systemd/systemd –switched-root –system –deserialize 21
(root,0,0,00:00:00,2) [kthreadd]
(root,0,0,00:00:02,3) [ksoftirqd/0]
(root,0,0,00:00:16,7) [migration/0]
(root,0,0,00:00:00,8) [rcu_bh]
(root,0,0,00:00:00,9) [rcuob/0]
(root,0,0,00:00:00,10) [rcuob/1]
(root,0,0,00:00:00,11) [rcuob/2]
(root,0,0,00:00:00,12) [rcuob/3]
(root,0,0,00:00:00,13) [rcuob/4]
(root,0,0,00:00:00,14) [rcuob/5]
(root,0,0,00:00:00,15) [rcuob/6]
(root,0,0,00:00:00,16) [rcuob/7]
(root,0,0,00:00:00,17) [rcuob/8]
(root,0,0,00:00:00,18) [rcuob/9]
(root,0,0,00:00:00,19) [rcuob/10]
(root,0,0,00:00:00,20) [rcuob/11]
(root,0,0,00:00:00,21) [rcuob/12]
(root,0,0,00:00:00,22) [rcuob/13]
(root,0,0,00:00:00,23) [rcuob/14]
(root,0,0,00:00:00,24) [rcuob/15]
(root,0,0,00:00:00,25) [rcuob/16]
(root,0,0,00:00:00,26) [rcuob/17]
(root,0,0,00:00:00,27) [rcuob/18]
(root,0,0,00:00:00,28) [rcuob/19]
(root,0,0,00:00:00,29) [rcuob/20]
(root,0,0,00:00:00,30) [rcuob/21]
(root,0,0,00:00:00,31) [rcuob/22]
(root,0,0,00:00:00,32) [rcuob/23]
(root,0,0,00:00:00,33) [rcuob/24]
(root,0,0,00:00:00,34) [rcuob/25]
(root,0,0,00:00:00,35) [rcuob/26]
(root,0,0,00:00:00,36) [rcuob/27]
(root,0,0,00:00:00,37) [rcuob/28]
(root,0,0,00:00:00,38) [rcuob/29]
(root,0,0,00:00:00,39) [rcuob/30]
(root,0,0,00:00:00,40) [rcuob/31]
(root,0,0,00:00:00,41) [rcuob/32]
(root,0,0,00:00:00,42) [rcuob/33]
(root,0,0,00:00:00,43) [rcuob/34]
(root,0,0,00:00:00,44) [rcuob/35]
(root,0,0,00:00:00,45) [rcuob/36]
(root,0,0,00:00:00,46) [rcuob/37]
(root,0,0,00:00:00,47) [rcuob/38]
(root,0,0,00:00:00,48) [rcuob/39]
(root,0,0,00:00:00,49) [rcuob/40]
(root,0,0,00:00:00,50) [rcuob/41]
(root,0,0,00:00:00,51) [rcuob/42]
(root,0,0,00:00:00,52) [rcuob/43]
(root,0,0,00:00:00,53) [rcuob/44]
(root,0,0,00:00:00,54) [rcuob/45]
(root,0,0,00:00:00,55) [rcuob/46]
(root,0,0,00:00:00,56) [rcuob/47]
(root,0,0,00:00:00,57) [rcuob/48]
(root,0,0,00:00:00,58) [rcuob/49]
(root,0,0,00:00:00,59) [rcuob/50]
(root,0,0,00:00:00,60) [rcuob/51]
(root,0,0,00:00:00,61) [rcuob/52]
(root,0,0,00:00:00,62) [rcuob/53]
(root,0,0,00:00:00,63) [rcuob/54]
(root,0,0,00:00:00,64) [rcuob/55]
(root,0,0,00:00:00,65) [rcuob/56]
(root,0,0,00:00:00,66) [rcuob/57]
(root,0,0,00:00:00,67) [rcuob/58]
(root,0,0,00:00:00,68) [rcuob/59]
(root,0,0,00:00:00,69) [rcuob/60]
(root,0,0,00:00:00,70) [rcuob/61]
(root,0,0,00:00:00,71) [rcuob/62]
(root,0,0,00:00:00,72) [rcuob/63]
(root,0,0,00:01:50,73) [rcu_sched]
(root,0,0,00:01:08,74) [rcuos/0]
(root,0,0,00:01:09,75) [rcuos/1]
(root,0,0,00:00:00,76) [rcuos/2]
(root,0,0,00:00:00,77) [rcuos/3]
(root,0,0,00:00:00,78) [rcuos/4]
(root,0,0,00:00:00,79) [rcuos/5]
(root,0,0,00:00:00,80) [rcuos/6]
(root,0,0,00:00:00,81) [rcuos/7]
(root,0,0,00:00:00,82) [rcuos/8]
(root,0,0,00:00:00,83) [rcuos/9]
(root,0,0,00:00:00,84) [rcuos/10]
(root,0,0,00:00:00,85) [rcuos/11]
(root,0,0,00:00:00,86) [rcuos/12]
(root,0,0,00:00:00,87) [rcuos/13]
(root,0,0,00:00:00,88) [rcuos/14]
(root,0,0,00:00:00,89) [rcuos/15]
(root,0,0,00:00:00,90) [rcuos/16]
(root,0,0,00:00:00,91) [rcuos/17]
(root,0,0,00:00:00,92) [rcuos/18]
(root,0,0,00:00:00,93) [rcuos/19]
(root,0,0,00:00:00,94) [rcuos/20]
(root,0,0,00:00:00,95) [rcuos/21]
(root,0,0,00:00:00,96) [rcuos/22]
(root,0,0,00:00:00,97) [rcuos/23]
(root,0,0,00:00:00,98) [rcuos/24]
(root,0,0,00:00:00,99) [rcuos/25]
(root,0,0,00:00:00,100) [rcuos/26]
(root,0,0,00:00:00,101) [rcuos/27]
(root,0,0,00:00:00,102) [rcuos/28]
(root,0,0,00:00:00,103) [rcuos/29]
(root,0,0,00:00:00,104) [rcuos/30]
(root,0,0,00:00:00,105) [rcuos/31]
(root,0,0,00:00:00,106) [rcuos/32]
(root,0,0,00:00:00,107) [rcuos/33]
(root,0,0,00:00:00,108) [rcuos/34]
(root,0,0,00:00:00,109) [rcuos/35]
(root,0,0,00:00:00,110) [rcuos/36]
(root,0,0,00:00:00,111) [rcuos/37]
(root,0,0,00:00:00,112) [rcuos/38]
(root,0,0,00:00:00,113) [rcuos/39]
(root,0,0,00:00:00,114) [rcuos/40]
(root,0,0,00:00:00,115) [rcuos/41]
(root,0,0,00:00:00,116) [rcuos/42]
(root,0,0,00:00:00,117) [rcuos/43]
(root,0,0,00:00:00,118) [rcuos/44]
(root,0,0,00:00:00,119) [rcuos/45]
(root,0,0,00:00:00,120) [rcuos/46]
(root,0,0,00:00:00,121) [rcuos/47]
(root,0,0,00:00:00,122) [rcuos/48]
(root,0,0,00:00:00,123) [rcuos/49]
(root,0,0,00:00:00,124) [rcuos/50]
(root,0,0,00:00:00,125) [rcuos/51]
(root,0,0,00:00:00,126) [rcuos/52]
(root,0,0,00:00:00,127) [rcuos/53]
(root,0,0,00:00:00,128) [rcuos/54]
(root,0,0,00:00:00,129) [rcuos/55]
(root,0,0,00:00:00,130) [rcuos/56]
(root,0,0,00:00:00,131) [rcuos/57]
(root,0,0,00:00:00,132) [rcuos/58]
(root,0,0,00:00:00,133) [rcuos/59]
(root,0,0,00:00:00,134) [rcuos/60]
(root,0,0,00:00:00,135) [rcuos/61]
(root,0,0,00:00:00,136) [rcuos/62]
(root,0,0,00:00:00,137) [rcuos/63]
(root,0,0,00:00:00,138) [watchdog/0]
(root,0,0,00:00:00,139) [watchdog/1]
(root,0,0,00:00:16,140) [migration/1]
(root,0,0,00:00:01,141) [ksoftirqd/1]
(root,0,0,00:00:00,143) [kworker/1:0H]
(root,0,0,00:00:00,144) [khelper]
(root,0,0,00:00:00,145) [kdevtmpfs]
(root,0,0,00:00:00,146) [netns]
(root,0,0,00:00:00,147) [perf]
(root,0,0,00:00:00,148) [writeback]
(root,0,0,00:00:00,149) [kintegrityd]
(root,0,0,00:00:00,150) [bioset]
(root,0,0,00:00:00,151) [kblockd]
(root,0,0,00:00:00,152) [md]
(root,0,0,00:00:00,157) [khungtaskd]
(root,0,0,00:00:02,158) [kswapd0]
(root,0,0,00:00:00,159) [ksmd]
(root,0,0,00:00:01,160) [khugepaged]
(root,0,0,00:00:00,161) [fsnotify_mark]
(root,0,0,00:00:00,162) [crypto]
(root,0,0,00:00:00,170) [kthrotld]
(root,0,0,00:00:00,172) [kmpath_rdacd]
(root,0,0,00:00:00,173) [kpsmoused]
(root,0,0,00:00:00,175) [ipv6_addrconf]
(root,0,0,00:00:00,195) [deferwq]
(root,0,0,00:00:00,226) [kauditd]
(root,0,0,00:00:00,393) [hv_vmbus_con]
(root,0,0,00:00:00,403) [ata_sff]
(root,0,0,00:00:00,404) [scsi_eh_0]
(root,0,0,00:00:00,405) [scsi_tmf_0]
(root,0,0,00:00:00,406) [scsi_eh_1]
(root,0,0,00:00:00,407) [scsi_tmf_1]
(root,0,0,00:00:00,412) [scsi_eh_2]
(root,0,0,00:00:00,415) [scsi_tmf_2]
(root,0,0,00:00:00,419) [scsi_eh_3]
(root,0,0,00:00:00,421) [scsi_tmf_3]
(root,0,0,00:00:03,444) [kworker/1:1H]
(root,0,0,00:00:00,496) [kdmflush]
(root,0,0,00:00:00,497) [bioset]
(root,0,0,00:00:00,507) [kdmflush]
(root,0,0,00:00:00,508) [bioset]
(root,0,0,00:00:00,522) [xfsalloc]
(root,0,0,00:00:00,523) [xfs_mru_cache]
(root,0,0,00:00:00,524) [xfs-buf/dm-0]
(root,0,0,00:00:00,525) [xfs-data/dm-0]
(root,0,0,00:00:00,526) [xfs-conv/dm-0]
(root,0,0,00:00:00,527) [xfs-cil/dm-0]
(root,0,0,00:00:51,528) [xfsaild/dm-0]
(root,34936,2820,00:00:10,599) /usr/lib/systemd/systemd-journald
(root,129128,696,00:00:00,617) /usr/sbin/lvmetad -f
(root,46000,884,00:00:00,627) /usr/lib/systemd/systemd-udevd
(root,0,0,00:00:00,663) [xfs-buf/sda1]
(root,0,0,00:00:00,666) [xfs-data/sda1]
(root,0,0,00:00:00,667) [kworker/0:1H]
(root,0,0,00:00:00,668) [xfs-conv/sda1]
(root,0,0,00:00:00,672) [xfs-cil/sda1]
(root,0,0,00:00:00,677) [xfsaild/sda1]
(root,0,0,00:00:00,707) [kdmflush]
(root,0,0,00:00:00,708) [bioset]
(root,0,0,00:00:00,715) [xfs-buf/dm-2]
(root,0,0,00:00:00,716) [xfs-data/dm-2]
(root,0,0,00:00:00,717) [xfs-conv/dm-2]
(root,0,0,00:00:00,718) [xfs-cil/dm-2]
(root,0,0,00:00:00,719) [xfsaild/dm-2]
(root,116720,1348,00:00:02,734) /sbin/auditd -n
(root,26396,1444,00:00:16,756) /usr/lib/systemd/systemd-logind
(root,262056,3264,00:00:03,757) /usr/sbin/rsyslogd -n
(dbus,34956,1496,00:00:41,759) /bin/dbus-daemon –system –address=systemd: –nofork –nopidfile –systemd-activation
(root,508700,2672,00:00:13,768) /usr/sbin/NetworkManager –no-daemon
(root,19176,860,00:00:10,769) /usr/sbin/irqbalance –foreground
(root,126364,1012,00:00:03,773) /usr/sbin/crond -n
(root,110032,644,00:00:00,775) /sbin/agetty –noclear tty1 linux
(polkitd,527448,1812,00:00:08,782) /usr/lib/polkit-1/polkitd –no-debug
(root,53060,1532,00:00:00,784) /usr/sbin/wpa_supplicant -u -f /var/log/wpa_supplicant.log -c /etc/wpa_supplicant/wpa_supplicant.conf -u -f /var/log/wpa_supplicant.log -P /var/run/wpa_supplicant.pid
(root,110520,1716,00:00:00,785) /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eth0.pid -lf /var/lib/NetworkManager/dhclient-55a03586-dc83-40c2-9d88-eac8c76f3ed6-eth0.lease -cf /var/lib/NetworkManager/dhclient-eth0.conf eth0
(root,553160,4144,00:00:20,984) /usr/bin/python -Es /usr/sbin/tuned -l -P
(root,29300,724,00:00:01,986) /usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid
(root,345492,4772,00:00:10,988) /usr/sbin/httpd -DFOREGROUND
(root,82556,1820,00:00:00,989) /usr/sbin/sshd -D
(root,90280,1112,00:00:05,1057) sendmail: accepting connections
(smmsp,85720,560,00:00:00,1100) sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue
(apache,262452,1408,00:00:02,1151) /usr/sbin/httpd -DFOREGROUND
(oibsite+,71688,1516,00:01:18,1865) /usr/bin/python /omd/sites/oibsite01/bin/liveproxyd
(oibsite+,795460,20376,00:04:55,2010) /omd/sites/oibsite01/bin/rrdcached -w 3600 -z 1800 -f 7200 -s oibsite01 -m 660 -l unix:/omd/sites/oibsite01/tmp/run/rrdcached.sock -p /omd/sites/oibsite01/tmp/rrdcached.pid -j /omd/sites/oibsite01/var/rrdcached
(oibsite+,156072,784,00:00:05,2098) /omd/sites/oibsite01/bin/npcd -d -f /omd/sites/oibsite01/etc/pnp4nagios/npcd.cfg
(oibsite+,109068,8488,00:00:07,2253) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(apache,345628,2812,00:00:00,2322) /usr/sbin/httpd -DFOREGROUND
(root,0,0,00:00:00,2450) [kworker/0:2]
(apache,345628,2984,00:00:00,2901) /usr/sbin/httpd -DFOREGROUND
(oibsite+,176024,6548,00:00:00,3237) /usr/bin/python /omd/sites/oibsite01/bin/liveproxyd 0
(apache,345628,2984,00:00:00,3739) /usr/sbin/httpd -DFOREGROUND
(oibsite+,264952,13792,00:00:00,3931) /usr/bin/php-cgi -c /omd/sites/oibsite01/etc/apache/php.ini -d session.save_handler=files -d session.save_path=/omd/sites/oibsite01/tmp/php/session -d upload_tmp_dir=/omd/sites/oibsite01/tmp/php/upload -d soap.wsdl_cache_dir=/omd/sites/oibsite01/tmp/php/wsdl-cache -d safe_mode=Off -d mysql.default_socket=/omd/sites/oibsite01/tmp/run/mysqld/mysqld.sock
(apache,345628,3472,00:00:00,3944) /usr/sbin/httpd -DFOREGROUND
(oibsite+,263284,56572,00:00:01,3963) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,263608,12412,00:00:00,3964) /usr/bin/php-cgi -c /omd/sites/oibsite01/etc/apache/php.ini -d session.save_handler=files -d session.save_path=/omd/sites/oibsite01/tmp/php/session -d upload_tmp_dir=/omd/sites/oibsite01/tmp/php/upload -d soap.wsdl_cache_dir=/omd/sites/oibsite01/tmp/php/wsdl-cache -d safe_mode=Off -d mysql.default_socket=/omd/sites/oibsite01/tmp/run/mysqld/mysqld.sock
(root,0,0,00:00:00,3987) [kworker/1:1]
(oibsite+,1047048,4784,00:00:00,3995) /omd/sites/oibsite01/bin/nagios -ud /omd/sites/oibsite01/tmp/nagios/nagios.cfg
(oibsite+,1047048,4784,00:00:00,3996) /omd/sites/oibsite01/bin/nagios -ud /omd/sites/oibsite01/tmp/nagios/nagios.cfg
(oibsite+,1047048,4784,00:00:00,3997) /omd/sites/oibsite01/bin/nagios -ud /omd/sites/oibsite01/tmp/nagios/nagios.cfg
(oibsite+,1047048,4784,00:00:00,3998) /omd/sites/oibsite01/bin/nagios -ud /omd/sites/oibsite01/tmp/nagios/nagios.cfg
(oibsite+,1047048,4784,00:00:00,3999) /omd/sites/oibsite01/bin/nagios -ud /omd/sites/oibsite01/tmp/nagios/nagios.cfg
(oibsite+,82628,10832,00:00:00,4000) python /omd/sites/oibsite01/var/check_mk/precompiled/hypervsrv03
(oibsite+,81312,9592,00:00:00,4002) python /omd/sites/oibsite01/var/check_mk/precompiled/Hypervsrv04testsnmp
(oibsite+,81292,9376,00:00:00,4003) python /omd/sites/oibsite01/var/check_mk/precompiled/EIND-VEEAM01
(oibsite+,82924,10188,00:00:00,4006) python /omd/sites/oibsite01/var/check_mk/precompiled/ILOHYPERV5
(oibsite+,80932,9204,00:00:00,4009) python /omd/sites/oibsite01/var/check_mk/precompiled/ZYWALL110-01
(oibsite+,102580,18652,00:00:00,4272) python /omd/sites/oibsite01/share/check_mk/modules/check_mk.py –defaults /omd/sites/oibsite01/etc/check_mk/defaults –automation diag-host — localhost ping 6556 1 5
(oibsite+,102576,18632,00:00:00,4274) python /omd/sites/oibsite01/share/check_mk/modules/check_mk.py –defaults /omd/sites/oibsite01/etc/check_mk/defaults –automation diag-host — localhost agent 6556 1 5
(oibsite+,102572,18652,00:00:00,4275) python /omd/sites/oibsite01/share/check_mk/modules/check_mk.py –defaults /omd/sites/oibsite01/etc/check_mk/defaults –automation diag-host — localhost snmpv1 6556 1 5
(oibsite+,102700,18656,00:00:00,4276) python /omd/sites/oibsite01/share/check_mk/modules/check_mk.py –defaults /omd/sites/oibsite01/etc/check_mk/defaults –automation diag-host — localhost snmpv2 6556 1 5
(oibsite+,102576,18652,00:00:00,4277) python /omd/sites/oibsite01/share/check_mk/modules/check_mk.py –defaults /omd/sites/oibsite01/etc/check_mk/defaults –automation diag-host — localhost snmpv2_nobulk 6556 1 5
(apache,345628,2728,00:00:00,4288) /usr/sbin/httpd -DFOREGROUND
(oibsite+,109068,5128,00:00:00,4296) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,18512,2024,00:00:00,4304) snmpbulkwalk -v2c -c -m -M -Cc -OQ -OU -On -Ot 10.255.190.22 .1.3.6.1.4.1.232.11.1.3.0
(oibsite+,109068,5128,00:00:00,4332) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,109068,5128,00:00:00,4354) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,18520,2024,00:00:00,4365) snmpwalk -v1 -c -m -M -Cc -OQ -OU -On -Ot 10.55.255.4 .1.3.6.1.2.1.2.2.1.8
(oibsite+,11636,1364,00:00:00,4366) /bin/sh -c ping -A -i 0.2 -c 2 -W 5 127.0.0.1 2>&1
(oibsite+,18516,2024,00:00:00,4368) snmpwalk -v1 -c -m -M -t 1.00 -r 5 -Cc -OQ -OU -On -Ot 127.0.0.1 .1.3.6.1.2.1.1.1.0
(oibsite+,10800,688,00:00:00,4369) ping -A -i 0.2 -c 2 -W 5 127.0.0.1
(oibsite+,18516,2024,00:00:00,4370) snmpbulkwalk -v2c -c -m -M -Cc -OQ -OU -On -Ot 10.255.255.216 .1.3.6.1.2.1.2.2.1.19
(oibsite+,18516,2020,00:00:00,4371) snmpwalk -v2c -c -m -M -t 1.00 -r 5 -Cc -OQ -OU -On -Ot 127.0.0.1 .1.3.6.1.2.1.1.1.0
(root,115248,1624,00:00:00,4372) /bin/bash /usr/bin/check_mk_agent
(oibsite+,18512,2024,00:00:00,4373) snmpbulkwalk -v2c -c -m -M -t 1.00 -r 5 -Cc -OQ -OU -On -Ot 127.0.0.1 .1.3.6.1.2.1.1.1.0
(root,47420,1624,00:00:00,4393) ps ax -o user,vsz,rss,cputime,pid,command –columns 10000
(root,15460,744,00:00:00,4394) sed -e 1d -e s/ *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) */(\1,\2,\3,\4,\5) /
(oibsite+,18520,2024,00:00:00,4395) snmpwalk -v1 -c -m -M -Cc -OQ -OU -On -Ot 10.255.190.23 .1.3.6.1.2.1.2.2.1.10
(apache,345628,3336,00:00:00,17654) /usr/sbin/httpd -DFOREGROUND
(apache,345628,2960,00:00:00,18356) /usr/sbin/httpd -DFOREGROUND
(apache,345628,2848,00:00:00,18949) /usr/sbin/httpd -DFOREGROUND
(oibsite+,1047040,5868,00:02:25,18999) /omd/sites/oibsite01/bin/nagios -ud /omd/sites/oibsite01/tmp/nagios/nagios.cfg
(oibsite+,109068,4908,00:00:00,21505) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,263284,56044,00:00:03,21506) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,263284,56024,00:00:03,21507) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,263284,55984,00:00:04,21508) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,263284,55988,00:00:03,21509) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(oibsite+,263284,56188,00:00:03,21510) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(root,0,0,00:00:29,21649) [kworker/u128:1]
(oibsite+,263284,56040,00:00:03,22123) /usr/sbin/httpd -f /omd/sites/oibsite01/etc/apache/apache.conf
(root,140972,3612,00:00:01,27231) sshd: root@notty
(root,52696,1568,00:00:00,27235) /usr/libexec/openssh/sftp-server
(root,0,0,00:00:00,30081) [kworker/0:2H]
(root,0,0,00:00:00,50965) [kworker/1:0]
(root,140944,5456,00:00:00,55988) sshd: root@pts/0
(root,115380,2004,00:00:00,56151) -bash
(root,185728,2412,00:00:00,56561) su oibsite01
(oibsite+,115504,2164,00:00:00,56562) bash
(root,0,0,00:00:10,57298) [kworker/u128:2]
(root,0,0,00:00:00,58016) [kworker/1:2]
(apache,345628,2996,00:00:00,58018) /usr/sbin/httpd -DFOREGROUND
(apache,345628,2996,00:00:00,58029) /usr/sbin/httpd -DFOREGROUND
(root,0,0,00:00:00,61130) [kworker/0:0]
(apache,345628,2996,00:00:00,62652) /usr/sbin/httpd -DFOREGROUND
(root,0,0,00:00:00,64438) [kworker/0:1]
<<>>
MemTotal: 1009368 kB
MemFree: 93940 kB
MemAvailable: 168520 kB
Buffers: 0 kB
Cached: 222096 kB
SwapCached: 10032 kB
Active: 374956 kB
Inactive: 377744 kB
Active(anon): 289472 kB
Inactive(anon): 294576 kB
Active(file): 85484 kB
Inactive(file): 83168 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 2097148 kB
SwapFree: 2039676 kB
Dirty: 2304 kB
Writeback: 0 kB
AnonPages: 520704 kB
Mapped: 25024 kB
Shmem: 53444 kB
Slab: 75412 kB
SReclaimable: 37104 kB
SUnreclaim: 38308 kB
KernelStack: 6256 kB
PageTables: 21716 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2601832 kB
Committed_AS: 1202252 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 53912 kB
VmallocChunk: 34359682048 kB
HardwareCorrupted: 0 kB
AnonHugePages: 241664 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 44992 kB
DirectMap2M: 1003520 kB
<<>>
2.02 0.71 0.34 3/328 4399 2
<<>>
153266.20 285985.00
<<>>
eth0: 1248867570 8539574 0 0 0 0 0 0 765057458 8138431 0 0 0 0 0 0
lo: 131774799 430234 0 0 0 0 0 0 131774799 430234 0 0 0 0 0 0
[lo]
Link detected: yes
Address: 00:00:00:00:00:00

[eth0]
Link detected: yes
Address: 00:15:5d:fe:19:0f

<<>>
01 18
0A 6
06 64
<<>>
1468402568
8 0 sda 815569 2587 9141787 1076033 2298951 126913 33475984 2181403 0 839887 3255923
253 0 dm-0 815017 0 9101945 1077715 2370855 0 33332664 2328137 0 837785 3406369
253 1 dm-1 3070 0 25672 3935 16891 0 135128 279754 0 2895 283691
253 2 dm-2 96 0 1441 78 16 0 4096 9 0 79 87
[dmsetup_info]
centos-home 253:2 centos home
centos-swap 253:1 centos swap
centos-root 253:0 centos root
<<>>
1468402568
nr_free_pages 27276
nr_alloc_batch 1099
nr_inactive_anon 73644
nr_active_anon 68625
nr_inactive_file 20583
nr_active_file 21580
nr_unevictable 0
nr_mlock 0
nr_anon_pages 65998
nr_mapped 6231
nr_file_pages 58057
nr_dirty 576
nr_writeback 0
nr_slab_reclaimable 9276
nr_slab_unreclaimable 9577
nr_page_table_pages 5361
nr_kernel_stack 391
nr_unstable 0
nr_bounce 0
nr_vmscan_write 16891
nr_vmscan_immediate_reclaim 269
nr_writeback_temp 0
nr_isolated_anon 0
nr_isolated_file 0
nr_shmem 13361
nr_dirtied 3409195
nr_written 3327541
numa_hit 594809413
numa_miss 0
numa_foreign 0
numa_interleave 12323
numa_local 594809413
numa_other 0
workingset_refault 982782
workingset_activate 141673
workingset_nodereclaim 0
nr_anon_transparent_hugepages 118
nr_free_cma 0
nr_dirty_threshold 12094
nr_dirty_background_threshold 4031
pgpgin 4570893
pgpgout 16737992
pswpin 2942
pswpout 16891
pgalloc_dma 7212350
pgalloc_dma32 594260777
pgalloc_normal 0
pgalloc_movable 0
pgfree 601518834
pgactivate 533147
pgdeactivate 574185
pgfault 1886758020
pgmajfault 705806
pgrefill_dma 7760
pgrefill_dma32 474431
pgrefill_normal 0
pgrefill_movable 0
pgsteal_kswapd_dma 19311
pgsteal_kswapd_dma32 1223400
pgsteal_kswapd_normal 0
pgsteal_kswapd_movable 0
pgsteal_direct_dma 0
pgsteal_direct_dma32 34455
pgsteal_direct_normal 0
pgsteal_direct_movable 0
pgscan_kswapd_dma 20017
pgscan_kswapd_dma32 1253522
pgscan_kswapd_normal 0
pgscan_kswapd_movable 0
pgscan_direct_dma 0
pgscan_direct_dma32 36382
pgscan_direct_normal 0
pgscan_direct_movable 0
pgscan_direct_throttle 0
zone_reclaim_failed 0
pginodesteal 0
slabs_scanned 176384
kswapd_inodesteal 0
kswapd_low_wmark_hit_quickly 16
kswapd_high_wmark_hit_quickly 35
pageoutrun 314
allocstall 645
pgrotated 16918
drop_pagecache 0
drop_slab 0
numa_pte_updates 0
numa_huge_pte_updates 0
numa_hint_faults 0
numa_hint_faults_local 0
numa_pages_migrated 0
pgmigrate_success 15536
pgmigrate_fail 0
compact_migrate_scanned 61595
compact_free_scanned 262382
compact_isolated 35555
compact_stall 72
compact_fail 44
compact_success 26
htlb_buddy_alloc_success 0
htlb_buddy_alloc_fail 0
unevictable_pgs_culled 6815
unevictable_pgs_scanned 0
unevictable_pgs_rescued 5741
unevictable_pgs_mlocked 8463
unevictable_pgs_munlocked 8463
unevictable_pgs_cleared 0
unevictable_pgs_stranded 0
thp_fault_alloc 1527
thp_fault_fallback 57
thp_collapse_alloc 385
thp_collapse_alloc_failed 0
thp_split 56
thp_zero_page_alloc 1
thp_zero_page_alloc_failed 0
cpu 1102530 19 853210 28469630 60121 0 26720 0 0 0
cpu0 544198 10 429478 14228156 31774 0 13552 0 0 0
cpu1 558332 9 423732 14241474 28346 0 13167 0 0 0
intr 48668347 2790508 9 0 0 0 0 3 0 0 0 0 0 161 0 0 149767 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ctxt 75336276
btime 1468249302
processes 1632214
procs_running 2
procs_blocked 0
softirq 75384302 0 26751531 1 8817552 1570898 0 17547857 7695307 0 13001156
<<>>
Personalities :
unused devices:
<<>>
<<>>
Mail queue is empty
<<>>
[oibsite01]
liveproxyd 0
rrdcached 0
npcd 0
nagios 0
apache 0
crontab 0
OVERALL 0
<<>>
[oibsite01]
accept_passive_host_checks;accept_passive_service_checks;cached_log_messages;check_external_commands;check_host_freshness;check_service_freshness;connections;connections_rate;enable_event_handlers;enable_flap_detection;enable_notifications;execute_host_checks;execute_service_checks;external_command_buffer_max;external_command_buffer_slots;external_command_buffer_usage;external_commands;external_commands_rate;forks;forks_rate;host_checks;host_checks_rate;interval_length;last_command_check;last_log_rotation;livecheck_overflows;livecheck_overflows_rate;livechecks;livechecks_rate;livestatus_active_connections;livestatus_queued_connections;livestatus_threads;livestatus_version;log_messages;log_messages_rate;mk_inventory_last;nagios_pid;neb_callbacks;neb_callbacks_rate;num_hosts;num_services;obsess_over_hosts;obsess_over_services;process_performance_data;program_start;program_version;requests;requests_rate;service_checks;service_checks_rate
1;1;60;1;0;1;2329;1.2067419162e-01;1;1;1;1;1;1;32768;0;1;6.3281250000e-02;42674;9.4375605149e-01;21058;3.5845328272e-01;60;1468402568;1468360800;0;0.0000000000e+00;0;0.0000000000e+00;1;0;20;1.2.6p16;743;2.0228296726e-20;0;18999;741516;8.6155502748e+00;14;563;0;0;1;1468310847;3.5.0;4275;3.4787955077e-01;843703;1.5852275571e+00
<<>>
<<>>

Log in to Reply
BenV
July 13th, 2016 on 11:53
@fooc: too bad the html ate half your comment, but I assume there was no mtr section.
Where is your mtr.cfg, and what happens when you run the plugin manually? Does it give an error?
Try running it from the plugin dir like this: ./mtr –debug

Log in to Reply
BenV
July 13th, 2016 on 11:55
@fooc: I forgot you also need to set your config dir:
export MK_CONFDIR=”/etc/check_mk”
and then the ‘./mtr –debug’

Log in to Reply
fooc
July 13th, 2016 on 12:15
cfg file is located in:
/usr/lib/check_mk_agent
/usr/lib/check_mk_agent/local
/usr/lib/check_mk_agent/plugins

I did the export as you asked. When i give the ./mtr –debug command it says:
bash: ./mtr: Permission denied

Under both the root and site user.

Thanx for your effort

Log in to Reply
fooc
July 13th, 2016 on 13:37
I think i got the permissions problem:

[root@localhost plugins]# chmod 755 mtr
[root@localhost plugins]# ./mtr –debug
<<>>

Log in to Reply
fooc
July 13th, 2016 on 13:50
Ah yes its working now!!!! thanx it was the permissions

Log in to Reply
mlist
October 24th, 2016 on 12:00
Hi BenV

I’m writing “Check_MK beginner guide” and, as example of adding custom mkp, I chose to include your great plugin
(obviously mentioning you and your website).
I have been able to make it working on localhost (where check_MK is running) with the following steps:

– mkp install /tmp/mtr-0.5.2.mkp
– cp -pi /opt/omd/sites/mysite/local/share/check_mk/agents/cfg_examples/mtr.cfg /etc/check_mk/
– cmk -II localhost

After that I tried to make it working on another centos 7.2 64bit using the same mtr.cfg with:

– scp /opt/omd/sites/mysite/local/share/check_mk/agents/mtr root@centos7tst1:/usr/lib/check_mk_agent/plugins/
– scp /opt/omd/sites/mysite/local/share/check_mk/agents/cfg_examples/mtr.cfg centos7tst1:/etc/check_mk/

The problem is that it doesn’t work and the check_MK gui display the following error:

UNKNOWN – check failed – please submit a crash report!

On the check_MK host, running “cmk -d centos7tst1”, the output is:

<<>>
**ERROR** Report file /var/lib/check_mk_agent/mtr.report.www-mysite-it has less than 3 lines, expecting at least 1 hop! Throwing away invalid report
http://www.mysite.it|0|0

and, indeed, on the centos7tst1 the file contains just one line:

[root@centos7tst1 ~]# cat /var/lib/check_mk_agent/mtr.report.www-mysite-it
1477302404

while on localhost:

[root@checkmktst1 ~]# cat /var/lib/check_mk_agent/mtr.report.www-mysite-it
1477302399
Start: Mon Oct 24 11:46:39 2016
HOST: checkmktst1 Loss% Snt Last Avg Best Wrst StDev
1.|– 10.79.160.3 0.0% 10 1.1 1.1 0.6 3.3 0.7
2.|– 10.72.254.214 0.0% 10 0.8 1.1 0.7 2.9 0.5
3.|– 10.72.254.229 0.0% 10 1.0 6.4 1.0 47.2 14.4
4.|– 10.255.248.49 0.0% 10 0.7 1.1 0.6 3.0 0.6
5.|– ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
6.|– 10.39.11.138 0.0% 10 11.8 15.7 11.8 32.4 6.3
7.|– mysite_PUB_IP 0.0% 10 11.9 12.1 11.8 12.7 0.0

But manually running “mtr mysite_PUB_IP”, the output is the same on both machine.

Do you have any suggestion?

Best
Marco
Italy

Log in to Reply
BenV
October 24th, 2016 on 19:12
@mlist: Good luck with your guide, sounds like a good thing to have around for people new to Check_MK 🙂

As to your problem, on the centos7tst1 machine try running the plugin manually:
cd /etc/check_mk
/usr/lib/check_mk_agent/plugins/mtr

Maybe it’s a python error, but without output it’s hard for me to predict where the problem lies 🙂

Log in to Reply
mlist
October 25th, 2016 on 08:59
Hi BenV thanks for your kind reply.

This is the required output:

On localhost:
[root@checkmktst1 check_mk]# cd /etc/check_mk/
[root@checkmktst1 check_mk]# /usr/lib/check_mk_agent/plugins/mtr
<<>>
http://www.mysite.com|0|0

On centos7tst1:
[root@centos7tst1 ~]# cd /etc/check_mk
[root@centos7tst1 check_mk]# /usr/lib/check_mk_agent/plugins/mtr
<<>>
http://www.mysite.com|0|0

As you can see the output is the same.

Log in to Reply
BenV
October 25th, 2016 on 09:05
That output look good. First time it runs it’ll return zeroes because the run hasn’t finished yet.
Run it again a few minutes later and you should see the zeroes turn into ping times and packet loss percentages etc.

Can you confirm that this happens? 🙂

Log in to Reply
mlist
October 25th, 2016 on 11:25
I just found the problem.
The problem was that check_mk_agent was running under systemd instead of xinetd. I know that the documentation recommend to use xinetd in fact I usually do in this way but because others checks were working correctly, I didn’t think that this could cause problems to some plugins.
Reinstalling the agent solved the problem.

Just a curiosity: why in your opinion it didn’t work using systemd?

Log in to Reply
BenV
October 25th, 2016 on 11:35
@mlist: I’m glad you found the problem.

It’s hard to diagnose the problem when I can’t look around the system, so no idea why the systemd path didn’t work.
In my opinion you don’t want to have any port open for check_mk to work, both systemd and xinetd are not necessary.

The way I always have it running is through ssh.
You can configure CheckMK to use specific “datasource programs”, which in my case is simply an ssh command with a specific key that’s in the authorized_keys file on the client with a forced command that always executes check_mk_agent.

This way you don’t have any unecessary ports open and get encrypted data transfer for free without having to muck around with systemd/xinetd 🙂

Log in to Reply
mlist
October 25th, 2016 on 15:04
This is an interesting point of view since check_mk says that the choise to have an agent running checks locally, has also been done for security reasons and to avoid the use of nrpe or ssh:
https://mathias-kettner.de/checkmk_security.html
🙂

Anyway I really thank you because I think your plugin is really useful. I searched for something like that in the past (using other monitoring solutions) but never found.
I also thank you for your kindness, I will mention you in my guide; if you are registered in the check_mk english mailing list, you will receive updates about it anyway I’ll write you as soon as it will be available

bye
Marco
Italy

Log in to Reply
BenV
October 25th, 2016 on 15:32
@mlist: As far as I’m concerned logging in on your clients with a monitoring specific ssh-key that can and will only execute the check_mk_agent program upon login while giving the results back over an encrypted channel is a lot safer than an open port that random people on your network can connect to without authentication to get that data. But maybe that’s just me :p

Also, most linux systems run an ssh deamon by default anyway, so this attack vector is already present on most systems, suffering mostly from bots that scan for weak passwords.
If you properly close down your system (i.e. set “AllowUsers” and “PermitRootLogin without-password” aka key-only and while you’re at it set PasswordAuthentication to “no” in sshd_config), you shouldn’t worry about this 🙂

Will look forward to your guide, and yes I’m subscribed to that list 🙂

Log in to Reply
mlist
October 25th, 2016 on 17:17
I’m not a security expert but I think that your thought makes perfect sense, even for me.
I know that this is “out of topic” but, because it is really interesting, I still kindly ask you to reply to these questions:

a) Apart security aspects, check_MK asserts that, speaking about peformance, the agent based mechanism is its great strength. Using the the ssh method you just described, couldn’t lead to performance problems, in particular in large environments?

b) Would you available to post some screenshot and steps about the ssh configuration? I could add it to my guide that just reached about 100 pages 🙂

Log in to Reply
BenV
October 25th, 2016 on 17:32
@mlist:
a) While I agree that there’s some overhead for setting up the connection, it isn’t much. If you care about microsecond latency on checks that run every X minutes you’re probably looking in the wrong direction 😉
At my current job we run a reasonably big Check_MK (OMD based) monitoring setup which uses gearman to distribute jobs to slave machines within our clients’ networks, those slaves use SSH to gather the check_mk_agent results. This setup currently has almost 700 hosts and over 10000 services, all based on ssh. Most of the overhead on the main Check_MK machine is due to Apache and Nagvis.

b) I could do that, but I don’t have much time for this right now, I’ll see if I can make a post on this coming thurdsay.
However, it’s not rocket science as I said 😉
I assume you’re mainly interested in the ssh configuration through WATO.
Short version:
– WATO — Host and Service Parameters — DataSource Programs — Individual Program Call instead of agent access — Create Rule — Command line to execute: “ssh -l root -i /etc/check_mk/keys/yourmonitoringkey <IP>”

Hope this helps 🙂

Log in to Reply
mlist
October 26th, 2016 on 09:09
a)
Ok BenV, it sounds like you are a really skilled person, surely much more than me; I love to speak with people like you because I always learn something.
Morover I just thought that this kind of configuration (ssh instead of tcp 6556) could help in situations where is difficult or even impossibile to open ports different that 22. You just did us a nice tip.

b)
Yes, I’m interested in the ssh configuration through WATO and a step by step post would be appreciated but you already driven me in the right direction thus if you have not time to write it, I’ll try by myself. Anyway I’ll keep my eyes on your website.

Thanks again BenV!

ciao
Marco
Italy

Log in to Reply
balroq
January 10th, 2017 on 11:53
You wouldn’t happen to have the script running without check_mk. I mean as simple command line script?

Log in to Reply
tintamar
February 14th, 2017 on 12:08
Hi BenV,

I have just tested your plugin with success.

Do you have some advice on where i should look to modify the display of the hop overview, since i am also interested in the IP of the hop number

Thks

Log in to Reply
BenV
February 14th, 2017 on 14:08
@tintamar: Since those addresses often differ from one run to the next (different hop, hop with multiple ips, etc) I chose not to implement it that way.
If you want to alter the graphs you can try editing check_mk-mtr.php in the pnp-templates, but you’ll also need to update the mtr check itself to put those IP addresses somewhere as performance data. IIRC it doesn’t do anything with this data right now, although the agent does provide the IPs in its output.

BTW: If you get something fancy working I’m always interested in patches, feel free to email me 🙂

Log in to Reply
dheerugudi
July 3rd, 2017 on 19:36
I have installed the plugin and placed the config file as mentioned above, but I get this error on checkMk dashboard
“Invalid check parameter: Undefined key ‘rta’ in the dictionary. Allowed are avg, stddev, loss.! The parameter is: {‘loss’: (10, 25), ‘rta’: (150, 250), ‘rtstddev’: (150, 250), ‘stddev’: (150, 250), ‘avg’: (150, 250), ‘pl’: (10, 25)} The variable name is: checkgroup_parameters:mtr” did I miss anything?

Log in to Reply
BenV
July 4th, 2017 on 09:48
@dheerugudi:
That’s fascinating, where the heck did those keys come from?

Are you by chance running the enterprise version of Check_MK?
If so: I know they reworked my plugin and included it (with my permission), but it’s a somewhat different version from the one available here, so it’s possible that the information on this page is not applicable to that version.
If not: can you show your configuration? The keys ‘rta’ and ‘rtstddev’ and completely unknown to me, the plugin and this page never mention those — where do they come from?
The only parameters my plugin supports are ‘loss’, ‘avg’, and ‘stddev’.

Log in to Reply
dheerugudi
July 5th, 2017 on 21:06
@BenV

I am using free edition.
Let me explain you the steps I followed to setup mtr plugin
On checkmk agent
1) Downloaded and installed V0.5.2: mtr-0.5.2.mkp from your website
2) Created mtr folder and copied mtr.cfg under /usr/lib/check_mk_agent/plugins/mtr/mtr.cfg
3) Placed mtr.cfg under /etc/check_mk/mtr.cfg

On the check_mk dashboard for mtr config

Hosts–>Manual Checks–>Traceroute with MTR–>(here I have created the rule) MTR destination=google.com saved it

After that the gives that error.
Let me know If I am missing anything.. Thanks

Log in to Reply