[Check_mk (english)] Predictive monitoring not working on 1.4* (no reference for prediction (Got no historic metrics))

Paul Dunkler paul at uplex.de
Wed Sep 27 11:44:51 CEST 2017


I just digged a bit deeper.
Seems like Matthias Kettner actually introduced the new way of querying the historic RRD data in the following commit back in 2016

> commit 6dc8ddc89f60b3764a91c5b9204a5ef2caf4fc64
> Author: Mathias Kettner <mk at mathias-kettner.de>
> Date:   Wed Feb 3 11:30:15 2016 +0100
> 
>     Fix predictive monitoring when using single RRDs

But with that commit, nothing new was added to livestatus....

And i just found some "evidence" of that it seems like Enterprise Edition of Livestatus is able to handle the dynamic column named "rrddata":

> diff --git a/.werks/8580 b/.werks/8580
> new file mode 100644
> index 0000000..d47068a
> --- /dev/null
> +++ b/.werks/8580
> @@ -0,0 +1,19 @@
> +Title: Removed space leaks related to dynamic columns.
> +Level: 1
> +Edition: cee
> +Component: livestatus
> +Compatible: compat
> +Version: 1.4.0i3
> +Date: 1480928352
> +Class: fix
> +
> +When some special columns were queried via Livestatus, the monitoring core
> +leaked memory. This affected the following columns:
> +
> +<ul>
> +<li><tt>mk_logwatch_file</tt> in table <tt>hosts</tt></li>
> +<li><tt>value</tt> in table <tt>eventconsolereplication</tt></li>
> +<li><tt>rrddata</tt> in tables <tt>hosts</tt> and <tt>services</tt></li>
> +</ul>
> +
> +


Best,
Paul

> Good to know that i'm not the only one fighting with this.
> I started to think being stupid...
> 
>> This actually has not been working since 1.2.8 (beta) – I sent a bug report in back then and did get some answers – yet I was told it was working, so I thought it was just me – I’m glad it is more than just me.
>> 
>> This is a feature that we rely on a lot – so it would be nice to get some “official” answer on what is wrong with it.
>> 
>> While this snippet is from beta version, it still the same output today.
>> 
>> 2016-04-04 17:18:37 Replacing non-existing column 'rrddata:m1:memory.max:1459746000:1459832399:1' with null column
>> 2016-04-04 17:18:37 Query: OutputFormat: python
>> 2016-04-04 17:18:37 Query: Filter: host_name = {host}
>> 2016-04-04 17:18:37 Query: Filter: description = Memory and pagefile
>> 2016-04-04 17:18:37 Time to process request: 28 us. Size of answer: 9 bytes
>> 2016-04-04 17:18:37 Query: GET services
>> 2016-04-04 17:18:37 Query: Columns: rrddata:m1:pagefile.max:1459746000:1459832399:1
>> 2016-04-04 17:18:37 Replacing non-existing column 'rrddata:m1:pagefile.max:1459746000:1459832399:1' with null column
>> 2016-04-04 17:18:37 Query: OutputFormat: python
>> 2016-04-04 17:18:37 Query: Filter: host_name =  {host}
>> 2016-04-04 17:18:37 Query: Filter: description = Memory and pagefile
>> 2016-04-04 17:18:37 Time to process request: 13 us. Size of answer: 9 bytes
>> 
>> 
>> It seems the change to using LiveStatus to get the prediction broke something – broke in the case data NEVER seems to get writing out to live status, I find no evidence in the code that it does, before this change there was a section to write the data to the xml files.  I might be wrong, but nevertheless, this is still really broken.
> 
> Yes i checked the code from earlier versions.
> 
> It seems to be the function "get_rrd_data" in
> ./share/check_mk/web/htdocs/prediction.py
> 
> ...which has recently been move to a library in
> ./lib/python/cmk_base/prediction.py
> 
> That was using rrdtool to extract the data for prediction from rrd files.
> Now it tries to query that information via livestatus.
> But it doesn't seem like livestatus in the RAW edition got this column included.
> 
> I can't find anything when searching for the word "rrddata" in the livestatus folder of the current check_mk git checkout.
> Maybe it is an option to reimplement the function by reverting and adopting it to the newer code.
> But for sure it would be much nicer to get a comment on this and maybe just get the missing code parts merged back into Livestatus RAW?
> 
> Best,
> Paul
> 
> 
>> From: checkmk-en [mailto:checkmk-en-bounces at lists.mathias-kettner.de <mailto:checkmk-en-bounces at lists.mathias-kettner.de>] On Behalf Of Gavin Hill
>> Sent: Tuesday, September 26, 2017 12:10 PM
>> To: Paul Dunkler <paul at uplex.de <mailto:paul at uplex.de>>; checkmk-en at lists.mathias-kettner.de <mailto:checkmk-en at lists.mathias-kettner.de>
>> Subject: Re: [Check_mk (english)] Predictive monitoring not working on 1.4* (no reference for prediction (Got no historic metrics))
>> 
>> Paul I can’t help you but I can confirm I’m seeing the same behaviour on 1.4.0p9
>> 
>> Gavin
>> 
>> From: checkmk-en [mailto:checkmk-en-bounces at lists.mathias-kettner.de <mailto:checkmk-en-bounces at lists.mathias-kettner.de>] On Behalf Of Paul Dunkler
>> Sent: Tuesday, 26 September 2017 17:37
>> To: checkmk-en at lists.mathias-kettner.de <mailto:checkmk-en at lists.mathias-kettner.de>
>> Subject: [Check_mk (english)] Predictive monitoring not working on 1.4* (no reference for prediction (Got no historic metrics))
>> 
>> Hi there,
>> 
>> i just realized that there is the "predictive monitoring"-Feature for some check types and tried it out.
>> Sadly it doesn't seem working on 1.4*
>> 
>> So far i tried the following versions:
>>> 1.4.0p12.cre
>>> 1.4.0p6.cre
>>> 1.4.0p9.cre
>> 
>> What i can see in the interface:
>>> Service description states that it couldn't find any historic metrics
>>> https://imgur.com/23n1yW4 <https://imgur.com/23n1yW4>
>> 
>> Errors from livestatus.log:
>>> 2017-09-26 13:01:24 [client 18] Unknown dynamic column 'rrddata'
>> 
>> 
>> Digging into the code:
>> 
>> Livestatus is queried for getting past values like the following example:
>>> GET services
>>> Columns: perf_data:m1:load15.max:1506376800:1506463199:1
>>> OutputFormat: python
>>> Filter: host_name = some.hostname
>>> Filter: description = CPU load
>> 
>> That doesn't work because there is no column named "rrddata" in livestatus (checked livestatus code as well).
>> Only thing i could find in livestatus is the column "perf_data". But this doesn't seem to be a dynamic column.
>> 
>> Could it be that the dynamic column "rrddata" is indeed a part of the check_mk enterprise editions new graphing system?
>> Is there any chance that we get this fixed soon? I would really like to use this helpful feature.
>> 
>> If somebody could point me into the right direction i am willing to help too (maybe contributing a patch)
>> 
>>>> Paul Dunkler
> 
>> Paul Dunkler
> 
> ** * * UPLEX - Nils Goroll Systemoptimierung
> 
> Scheffelstraße 32
> 22301 Hamburg
> 
> tel +49 40 288 057 31
> mob +49 151 252 228 42
> fax +49 40 429 497 53
> 
> xmpp://pauldunkler@jabber.ccc <xmpp://pauldunkler@jabber.ccc>.de
> 
> http://uplex.de/ <http://uplex.de/>
> _______________________________________________
> checkmk-en mailing list
> checkmk-en at lists.mathias-kettner.de <mailto:checkmk-en at lists.mathias-kettner.de>
> http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en <http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en>
—
Paul Dunkler

** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

tel +49 40 288 057 31
mob +49 151 252 228 42
fax +49 40 429 497 53

xmpp://pauldunkler@jabber.ccc <xmpp://pauldunkler@jabber.ccc>.de

http://uplex.de/ <http://uplex.de/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mathias-kettner.de/pipermail/checkmk-en/attachments/20170927/e409b21a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 529 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.mathias-kettner.de/pipermail/checkmk-en/attachments/20170927/e409b21a/attachment-0001.sig>


More information about the checkmk-en mailing list