[checkmk-commits] [tribe29/checkmk] 247b86: Reduce multiprocessing overhead

Lars noreply at github.com
Tue Jul 9 23:30:54 CEST 2019

  Branch: refs/heads/1.6.0
  Home:   https://github.com/tribe29/checkmk
  Commit: 247b86e675c7fa9e19a20273a6e9cf46d95a3f6d
  Author: Lars Michelsen <lm at tribe29.com>
  Date:   2019-07-09 (Tue, 09 Jul 2019)

  Changed paths:
    M tests/testlib/__init__.py

  Log Message:
  Reduce multiprocessing overhead

Each site is managed by an own site process in liveproxyd. Each of
these processes reported it's state to the master process through
as multiprocessing queue which needed synchronization of the processes
together with pickling and unpickling of the transported data.

The queue size was unlimited which could lead to to the situation where
the sites push data to the queue faster than the manager could process.
The master would then always process outdated state information until it
reaches the end of the queued data. This could block the manager process
for a too long time.

All this was way too complicated, because the state information from the
site processes is an already formatted state dump which was just
concatenated together to write it to the state file

The mechanism has now been simplified: Each site writes it's state to
a dedicated file tmp/liveproxyd/states/[site_id]. The manager process
simply reads all of those files and writes them to the general state file.

Change-Id: Ie227bda33541e914dd6eed9e38cee4583b2b92f5

More information about the checkmk-commits mailing list