Check_MK plugin: lmsensors
by BenV on Jun.07, 2011, under Software
On the subject of Pretty Graphs (see my earlier post), I decided to write a plugin (a.k.a. ‘Package’) for Check_MK in order to monitor (and make pretty graphs!) of the sensor output of lmsensors.
Most machines support this out of the box these days, and it’s always interesting to see the conditions of your machine. In case you don’t know, it gives the temperature and voltage of your CPU and mainboard. Thus it’s a good source for making pretty graphs 🙂
Since these plugins are very easy to install (once you’ve got check_mk up and running that is) and still nobody had written one for lmsensors, I decided to do it myself. Writing python isn’t my strongest point (yet), but these are good opportunities to learn.
One of the issues I ran into while writing the plugin was that PNP4Nagios fails on service names that have a plus character in them. For instance, I had Sensor +12V. This created the files Sensor_+12V.rrd and corresponding xml, but when one would go to the PNP4Nagios graph of that sensor it would request a file called Sensor__12V.rrd, which obviously failed.
Therefore I molested the names a bit, so your sensor might now be simply called MB12V instead of M/B+12V.
Configuration of LM-Sensors
For my plugin to work you need to make sure that you have the ‘sensors’ tool. This normally comes in a package called “lmsensors” or “lm-sensors”. Note that you obviously need some kind of hardware sensor on your machine that’s supported by lm-sensors for it to work, including the required kernel module. Fortunately this will often work out of the box.
After making sure you have sensors and that running ‘sensors’ will give output like this:
benv@localhost:~$ sensors
it8720-isa-0228
Adapter: ISA adapter
in0: +1.04 V (min = +0.00 V, max = +4.08 V)
in1: +1.66 V (min = +0.00 V, max = +4.08 V)
in2: +3.39 V (min = +0.00 V, max = +4.08 V)
+5V: +3.04 V (min = +0.00 V, max = +4.08 V)
in4: +3.10 V (min = +0.00 V, max = +4.08 V)
in5: +1.90 V (min = +0.00 V, max = +4.08 V)
in6: +4.08 V (min = +0.00 V, max = +4.08 V)
5VSB: +3.04 V (min = +0.00 V, max = +4.08 V)
Vbat: +3.30 V
fan1: 2235 RPM (min = 10 RPM)
fan2: 0 RPM (min = 0 RPM)
fan3: 2500 RPM (min = 0 RPM)
fan5: 0 RPM (min = 0 RPM)
temp1: +41.0°C (low = +127.0°C, high = +127.0°C) sensor = thermistor
temp2: +36.0°C (low = +127.0°C, high = +90.0°C) sensor = thermal diode
temp3: +38.0°C (low = +127.0°C, high = +127.0°C) sensor = thermistor
cpu0_vid: +1.250 V
k10temp-pci-00c3
Adapter: PCI adapter
temp1: +32.0°C (high = +70.0°C)
You might see things like ‘in0’ instead of ‘CPU Voltage’. Don’t ask me what voltage corresponds with what sensor on your mainboard, but you can rename the sensor output by editing /etc/sensors.conf or /etc/sensors3.conf depending on your flavor of linux.
In order for the check_mk plugin templates to recognize the type you’ll need to make sure they have some kind of indication of the type of sensor. For instance label the temperature sensors with ‘temp’ or ‘temperature’. The default names like ‘in0’ will also work, but something like ‘Pizza sensor’ obviously won’t.
To change labels or ignore certain sensors because they give bogus data (not connected etc), first find your adapter type.
In the example above this is it8720-isa-0228. Now edit the sensors.conf file and add a section
for this adapter if it isn’t already there.
Here’s an example for renaming in0 to “CPU Voltage” and turning off the second fan since it’s not connected.
Also we’ll change the minimum and maximum voltage for the CPU Voltage — this determines when nagios will send out an alarm or not:
chip "it8720-isa-0228"
set in0_min 1.0
set in0_max 2.0
label in0 "CPU Voltage"
ignore fan2
After changing the sensors file you’ll need to make lmsensors aware of the configuration change by running ‘sensors -s’. (might need root).
benv@localhost:~$ sensors -s
benv@localhost:~$ sensors
it8720-isa-0228
Adapter: ISA adapter
CPU Voltage: +1.31 V (min = +1.00 V, max = +2.00 V)
fan1: 2235 RPM (min = 10 RPM)
fan3: 2500 RPM (min = 0 RPM)
# some stuff deleted to save space :)
Tada. Now repeat this process for all sensors 🙂
Installation:
There are two parts to installing a Check MK plugin. First on the host that actually runs check_mk we need to install the package. This is quickly done:
root@checkmk# wget http://notes.benv.junerules.com/wp-content/uploads/2011/12/lmsensors-1.4.mkp
root@checkmk# md5sum lmsensors-1.4.mkp
115bd50557d1db7e934baa64e172e506 lmsensors-1.4.mkp
root@checmk# check_mk -vP install lmsensors-1.4.mkp
Installing lmsensors version 1.4.
Checks:
lmsensors
Checks man pages:
lmsensors
Agents:
lmsensors
Multisite extensions:
plugins/perfometer/lmsensors.py
PNP4Nagios templates:
check_mk-lmsensors.php
check_mk-lmsensors.fan.php
check_mk-lmsensors.temp.php
check_mk-lmsensors.volt.php
root@checkmk# check_mk -II
lmsensors.fan 2 new checks
lmsensors.volt 4 new checks
root@checkmk# check_mk -O
Done. Soon there will be pretty graphs for this machine 🙂
Now for a remote machine you will need to put the agent in place. Since this is only a single file it’s trivial to do:
benv@checkmk$ scp /usr/share/check_mk/agents/lmsensors root@othermachine:/usr/share/check_mk/agents
Note that the place you want to put that thing in is the $MK_LIBDIR/plugins directory. In my case, this was /usr/lib/check_mk_agent/plugins, but it could very well be somewhere else on your system. You can find it in the check_mk_agent script if you don’t know:
benv@somemachine$ grep MK_LIBDIR `which check_mk_agent`
export MK_LIBDIR="/usr/lib/check_mk_agent"
PLUGINSDIR=$MK_LIBDIR/plugins
Let Check_MK do an inventory on your remote machine [check_mk -II $machine] and the rest goes automagically! 🙂
And now we have pretty graphs for my sensors.
Comments and/or suggestions are welcome.
LMSensors in Check_MK
Updates:
Version 1.1: now has pnp templates to put graphs of the same type together.
Here’s an example:
Now we have combined graphs (v1.1)
Version 1.2: changed sed to perl in agent plugin, sensornames with more than one space (among things) were giving issues. Thanks to Cyril Pawelko for finding the issue and helping with testing!
Version 1.3: minor change to PNP templates — Nico Weinreich informed me that his fan templates weren’t working correctly so I updated the regular expression used to match corresponding sensor types. If you didn’t have this issue this update won’t do anything useful for you 🙂
Version 1.4: Seems like I was a dumbass and didn’t check the 1.2 package properly. This package really makes it work with perl instead of sed.
Also updated the voltage pnp template to hopefully match more voltage sensors.
Note: if the pnp4nagios template doesn’t work for you, check your pnp4nagios perfdata dir, for example /var/lib/pnp4nagios/perfdata/ and see what .rrd files exist for your host. They are based on the sensor name, so if your sensor name is “St John”, it will not match the voltage template. These names come directly from your sensors.conf (if you don’t have it the default names for the sensors).
See above on how to rename your sensors.
Version 1.5: Another bug spotted by Cyril! The pnp4nagios temperature template had a botched variable name.
Downloads:
lmsensors-1.6.mkp (5243 downloads)
SHA1: 42f7f7eebf803fb3ed755107521761bfb4fbb6bc MD5: 199266bfd9d750243b9bc49773b6b4d6
[Download not found]
[Download not found]
[Download not found]
[Download not found]
[Download not found]
[Download not found]
33 Comments for this entry
1 Trackback or Pingback for this entry
-
Check_MK bleeding edge « runnane
June 4th, 2014 on 12:18[…] lm-sensors package, neatly packed by BenV: http://notes.benv.junerules.com/lmsensors/ […]
September 28th, 2011 on 22:56
The bar graphs on the right side are not working for me. I just get “invalid data: global name ‘savefloat’ is not defined”
I am running OMD on Ubuntu 10.04 LTS with check_mk version 1.1.9i8
-Daniel
October 1st, 2011 on 15:19
danielbair: I guess your version of check_mk doesn’t include the definition of saveint/savefloat yet. If upgrading to a newer version isn’t an option you could try to remove the savefloat function in perfometer/lmsensors.py (lines 9 and 10), it should still work if the check reports good values.
November 22nd, 2011 on 10:46
Installation on check_mk Linux server has been successful (following your instructions)
“check_mk -II localhost” doesn’t reports any lmsensors data.
November 22nd, 2011 on 13:12
loreto: can you post/mail me the output of ‘sensors’ and ‘check_mk –debug –checks=lmsensors.temp -II localhost’ (if you think the temperature sensors should report something, otherwise ‘lmsensors.fan’ or ‘lmsensors.volt’)?
November 23rd, 2011 on 09:36
Ciao BenV, I didn’t find you email so I’m posting the output.
————————————————
– host588 – Check_mk server – Virtual Machine
————————————————
[root@host588]# check_mk -vP install /root/Check_MK/Plugins/lmsensors-1.0.mkp
Installing lmsensors version 1.0.
Checks:
lmsensors
Agents:
lmsensors
Multisite extensions:
plugins/perfometer/lmsensors.py
[root@host588]# check_mk -II localhost
cpu.loads 1 new checks
cpu.threads 1 new checks
df 8 new checks
diskstat 1 new checks
kernel 3 new checks
kernel.util 1 new checks
lnx_if 1 new checks
local 1 new checks
mem.used 1 new checks
ntp.time 1 new checks
postfix_mailq 1 new checks
tcp_conn_stats 1 new checks
uptime 1 new checks
*****
** lmsensors is not listed (may be it’s a VM)
*****
[root@host588]# check_mk -debug –checks=lmsensors.temp -II localhost
Network error: [Errno -2] Name or service not known
*****
** Now copying lmsensors to PM
*****
————————————-
– host755 – Physical machine
————————————-
root@host755# grep MK_LIBDIR $(which check_mk_agent)
export MK_LIBDIR=”/usr/lib/check_mk_agent”
PLUGINSDIR=$MK_LIBDIR/plugins
LOCALDIR=$MK_LIBDIR/local
*****
** Because of it didn’t work, I copied lmsensors on all check_mk directories
*****
scp -p /usr/share/check_mk/agents/lmsensors root@host755:/usr/lib/check_mk_agent/
scp -p /usr/share/check_mk/agents/lmsensors root@host755:/usr/lib/check_mk_agent/plugins/
scp -p /usr/share/check_mk/agents/lmsensors root@host755:/usr/lib/check_mk_agent/local/
scp -p /usr/share/check_mk/agents/lmsensors root@host755:/usr/share/check_mk/agents/
chmod 755 lmsensors
[root@host588]# check_mk -II host755
cpu.loads 1 new checks
cpu.threads 1 new checks
df 8 new checks
diskstat 1 new checks
kernel 3 new checks
kernel.util 1 new checks
mem.used 1 new checks
ntp.time 1 new checks
tcp_conn_stats 1 new checks
uptime 1 new checks
*****
** lmsensors is not listed
*****
[root@host588]# check_mk -debug –checks=lmsensors.temp -II host755
Network error: [Errno -2] Name or service not known
Ciao
November 23rd, 2011 on 11:07
My email is benv (at) junerules dot com 🙂
And you’re right: lmsensors doesn’t work on virtual machines, or at least, that’s my experience so far.
On the physical machine it should work however, depending on the sensors in your machine and whether or not they are supported by lmsensors (and in your kernel).
The check_mk lmsensors plugin expects ‘/usr/bin/sensors’ to exist and to provide output. For example, my server here at home outputs this:
benv@server:~$ sensors
lm85-i2c-0-2e
Adapter: SMBus I801 adapter at 2000
in0: +1.56 V (min = +0.00 V, max = +3.32 V)
Vcore: +1.34 V (min = +0.00 V, max = +2.99 V)
+3.3V: +3.33 V (min = +2.97 V, max = +3.63 V)
+5V: +5.10 V (min = +4.50 V, max = +5.50 V)
+12V: +12.00 V (min = +0.00 V, max = +15.94 V)
fan1: 1361 RPM (min = 0 RPM)
fan3: 952 RPM (min = 0 RPM)
fan4: 1251 RPM (min = 0 RPM)
temp1: +63.0 C (low = +0.0 C, high = +75.0 C)
M/B Temp: +49.0 C (low = +0.0 C, high = +65.0 C)
temp3: +39.0 C (low = +0.0 C, high = +55.0 C)
cpu0_vid: +1.088 V
Can you check if you have the tool and post the output here, and/or email it to me? 🙂
November 23rd, 2011 on 15:02
Note that I just updated the package, this version also includes a PNP4Nagios template and a little man page for the check.
December 5th, 2011 on 12:51
Ciao BenV,
Sorry for the delay of my answer, but since the command “sensor” did not include any output on my machine, I tried to install the drivers of the manufacturer. Unfortunately, the “sensor” command still does not give any result. We are investigating.
Ciao
Loreto
February 16th, 2013 on 06:35
hey 😉
i have installed the plugin on my check_mk instance. the problem i have is, that check_mk does not recognise the plugin when i reinventarize the check_mk on the local machine. on a remote server this works very fine. any ideas? 😉
February 16th, 2013 on 10:33
Stefan927: I’m guessing it’s either the output of sensors that’s weird or your check_mk can’t locate the agent plugin locally.
You can try running: check_mk_agent | grep PluginsDirectory
and check the dir it returns, there should be a file called ‘lmsensors’ there.
Next try running that file and see if it gives proper output (a section <<<lmsensors>>> followed by sensordata).
Also you can see if this command gives any weird errors:
check_mk –debug –checks lmsensors.temp localhost
Good luck hunting for the problem 🙂
February 17th, 2013 on 01:23
thanks for your quick reply 😉
exatly that was the problem.
i have installed the latest package (lmsensors-1.5.mkp) and the lmsensors file was extrated to the wrong directory “/usr/share/check_mk/agents/” but the default path when you install check_mk is “/usr/lib/check_mk_agent/plugins”. Is this a package error? 😉
February 17th, 2013 on 11:39
Stefan927: as far as I know that’s not a package error — that is, I only tell check_mk to make a package and check_mk (and I hope therefore the package)’knows’ what parts are agent plugins and libraries etc. Then again, I don’t know what the official word on this is from Mr Check_MK himself, the documentation is not very detailed on this 🙂
March 26th, 2013 on 03:00
Hi there and nice agent you have build. Unfortunately I’m a bit stupid or I have missed some part because I have this kind of situation:
root@blah:/opt# telnet foo 6556
… …
<<>>
coretemp-isa-0000
Core_0 +48.0 C high +73.0 C crit +85.0 C
Core_1 +41.0 C high +73.0 C crit +85.0 C
Core_2 +45.0 C high +73.0 C crit +85.0 C
Core_3 +43.0 C high +73.0 C crit +85.0 C
coretemp-isa-0001
Core_0 +48.0 C high +73.0 C crit +85.0 C
Core_1 +39.0 C high +73.0 C crit +85.0 C
Core_2 +40.0 C high +73.0 C crit +85.0 C
Core_3 +40.0 C high +73.0 C crit +85.0 C
Like You see I have two CPU system with 8 cores. So check_mk can see just that last one CPU because it’s actually same core numbers I think. Is there some easy way to fix this? I suppose CPUx_Core_y could solve this problem or is there some clever solution? 🙂
March 26th, 2013 on 10:44
Hello sippe,
That’s a bug you got there, or at least, I never ran into this situation myself and wasn’t smart enough to realize beforehand that this might occur 🙂
I’ll have to look into fixing this for you, but you can try to work around the problem by renaming the sensors in /etc/sensors.conf like this:
chip “coretemp-isa-0000”
label in0 “CPU1_Core0”
label in1 “CPU1_Core1”
…
chip “coretemp-isa-0001”
label in0 “CPU2_Core0”
label in1 “CPU2_Core1”
…
(don’t forget to run sensors -s afterwards to apply these changes)
I’ll try to fix it in the sensor agent soon, but I’m a bit short on time at the moment so give me a few days 🙂
March 26th, 2013 on 13:02
Hi Ben!
Great thanks for you. I really like your tool and I suppose it’s getting even better all the time. 🙂
Nice work! 🙂
April 20th, 2013 on 14:59
Hi Ben,
another “bug” in the pnp4nagios template spottet.
when i go to the pnp4nagios directly i have the following error:
Template /var/www/virtual/xxx/monitoring/htdocs/pnp4nagios//templates/check_mk-lmsensors.volt.php does not provide array $def[].
any ideas? 😉
April 22nd, 2013 on 13:06
Stefan927: Sounds like the template isn’t able to find your sensordata .rrd files. It expects something like ‘Sensor_core0.rrd’, can you provide the name of your .rrd file?
April 22nd, 2013 on 15:17
these sensor_*.rrd’s i have.
Sensor_12.0V.rrd
Sensor_3.3V.rrd
Sensor_3.3V_Standby.rrd
Sensor_5.0V.rrd
Sensor_5.0V_Standby.rrd
Sensor_Core_0.rrd
Sensor_Core_0_Temperature.rrd
Sensor_Core_1.rrd
Sensor_Core_1_Temperature.rrd
Sensor_Memory_Temperature.rrd
Sensor_Memory_Vcc.rrd
Sensor_Processor_Vcc.rrd
Sensor_System_Fan.rrd
Sensor_Vbat.rrd
Sensor_VR_Temperature.rrd
April 30th, 2013 on 12:55
Stefan927: sorry for the slow reply, busy week.
The rrd files look fine as far as their names are concerned, so that’s not the issue.
Maybe you can email me the details of your setup (check_mk / pnp4nagios versions etc) so we can try to figure out the problem that way, might be easier 🙂
My email is benv (at) junerules dot com.
January 9th, 2014 on 01:35
Hi Ben,
i have a Problem… 🙂
————————–
root@noc:/opt/omd/sites/monitoring# check_mk -vP install lmsensors-1.5.mkp
-bash: check_mk: command not found
root@noc:/opt/omd/sites/monitoring# su monitoring
OMD[monitoring]:~$ check_mk -vP install lmsensors-1.5.mkp
Traceback (most recent call last):
File “/omd/sites/monitoring/share/check_mk/modules/check_mk.py”, line 4845, in
do_packaging(args)
File “/omd/sites/monitoring/share/check_mk/modules/packaging.py”, line 98, in do_packaging
f(args)
File “/omd/sites/monitoring/share/check_mk/modules/packaging.py”, line 334, in package_install
tar = tarfile.open(path, “r:gz”)
File “/usr/lib/python2.7/tarfile.py”, line 1678, in open
return func(name, filemode, fileobj, **kwargs)
File “/usr/lib/python2.7/tarfile.py”, line 1729, in gzopen
raise ReadError(“not a gzip file”)
tarfile.ReadError: not a gzip file
OMD[monitoring]:~$
—————————–
January 9th, 2014 on 03:16
Hi Ben,
i can solve the Problem… redownload the File…but now i have a new Problem. I use OMD with Check-MK. I cant find the Agent File to copy to the remote Server…
January 9th, 2014 on 09:02
Is it not in /usr/share/check_mk/agents/lmsensors? If you have the locate command available you can try running ‘locate lmsensors’ and see what that returns, possibly after running updatedb 🙂
January 23rd, 2014 on 15:44
Ben,
you did some amazing job here.
I’ve got it working, but I have one aesthetic issue – rrd filenames are saved with sensor name duplicated eg. Sensor_in3_in3.rrd or Sensor_fan1_fan1.rrd. You can figure out that it will get ugly on pnp graphs, as names are taken from rrd filenames. Here is and example
http://fonoc.net/snap/Check_MK_gv_18916224.png
Do you have an idea where can I change that behavior or why it happens that way?
Thanks and keep on great work.
January 23rd, 2014 on 15:53
Hm, good question. Over here I don’t see that behaviour, but most of the names come from the output of lmsensors.
See what ‘sensors’ says, or check the agent plugin out. Are the names double there as well?
On my machines I simply get rrd files like ‘Sensor_fan1.rrd’.
January 23rd, 2014 on 22:08
sensors output, both from sensors and from check_mk_agent through your plugin, look pretty normal:
<<>>
coretemp-isa-0000
CPU_Physical_Tempetature +35.0 C high +80.0 C crit +100.0 C
Core_0_Temperature +35.0 C high +80.0 C crit +100.0 C
Core_1_Temperature +31.0 C high +80.0 C crit +100.0 C
Core_2_Temperature +32.0 C high +80.0 C crit +100.0 C
Core_3_Temperature +32.0 C high +80.0 C crit +100.0 C
it8728-isa-0a30
in0 +0.02 V min +0.00 V max +3.06 V
in1 +2.09 V min +0.00 V max +3.06 V
in2 +2.03 V min +0.00 V max +3.06 V
in3 +2.05 V min +0.00 V max +3.06 V
in4 +0.01 V min +0.00 V max +3.06 V
in5 +1.76 V min +0.00 V max +3.06 V
in6 +1.54 V min +0.00 V max +3.06 V
3VSB +3.38 V min +0.00 V max +6.12 V
All other rrd filenames (that are not ‘Sensor..’ are normal.
What is creating those files check_mk or pnp4nagios?
I was trying to figure out in code where they come from, but I couldn’t figure it out.
My best bet is that they are generated from ‘Service Description’ + ‘Item’
or maybe check_mk version I’m running is causing that behavior (1.2.2p3 (omd 1.10 package))?
January 24th, 2014 on 10:42
The rrd files are created by pnp4nagios (which in turn gets called by nagios through process_perfdata.pl), maybe it’s a configuration/version thing there. I’m running check_mk version 1.2.2p1 here without problems, pnp4nagios version 0.6.21.
From process_perfdata.pl it seems it uses the Service Description as filename, if the rrd storage type is set to ‘SINGLE’, but when it’s set to ‘MULTILPE’ it will use the Service Description AND ‘name’ of the item. Maybe there’s where it goes wrong. I have the storage type set to SINGLE here – so maybe this is the problem.
The graph template then takes the name of the rrd file as title for the legend as you guessed.
Seems like I’ll have to test with a pnp4nagios setup with the storage type set to multiple, but I won’t have time to test this today. I’ll come back on this 🙂
January 24th, 2014 on 14:31
Correct.
OMD comes with pnp4nagios configured with this:
RRD_STORAGE_TYPE = MULTIPLE
November 15th, 2015 on 11:30
Hello,
First thank you for the script, but i have a small issue, I’m getting the following output in check_mk:
CRIT – CRITICAL – Sensor value 35.0 C, which is smaller than +127.0
and it’s not right, the output in sensors is:
temp1: +35.0°C (low = +127.0°C, high = +127.0°C) sensor = thermistor
Any idea how could i fix it ?
November 15th, 2015 on 14:53
@adrian: in order to set the low and high alarms you need to create/edit your /etc/sensors.conf (or whatever file your distro uses, it can also be sensors3.conf) and then run sensors -s.
For example for my it8728-isa-0a30 adapter I can adjust the temperature by adding a section like this:
chip “it8728-*”
set temp1_min -20
set temp1_max 127
After editing/adjusting the file run ‘sensors -s’ and see if it’s any better 🙂
Wouter.
March 29th, 2017 on 09:12
You can fix the issue in “check_lmsensors” using this patch:
— lmsensors.org 2011-06-07 16:01:01.000000000 +0200 +++ lmsensors 2017-03-29 09:10:26.184876477 +0200 @@ -59,8 +59,8 @@ break if values == None: return (3, “UNKNOWN – sensor status not found in agent output”) – value = float(line[1]) – stype = line[2] + value = float(values[1]) + stype = values[2] perfdata = [ ( item, value, “”, savefloat(smax) ) ] if smax != None and value > float(smax): return (2, “CRITICAL – Sensor value %s %s, which is bigger than %s” % (value, stype, smax), perfdata)
March 29th, 2017 on 10:11
@brightdroid: thanks for the patch, updated the package as v1.6! 🙂
July 23rd, 2018 on 18:25
Is there any way to find the cpu frequency and its alerts thresold values suppose my cpu will operate 4.2GH suppoe if it down with 3.6hz i need to get alert.
July 23rd, 2018 on 21:38
This sounds more like a lmsensors question than related to the check_mk plugin if you ask me.
There are several tools available to get the CPU speeds, your kernel will usually have a governor setup to handle these speeds. These governors are interchangable, you could for example force it to always run at 4.6GHz.
I don’t really understand your question I’m afraid 🙂