Mailing List Archive

action with action.execOnlyWhenPreviousIsSuspended="on" after asynchronous omelasticsearch action won't run
Hi. I've noticed that an action with
*action.execOnlyWhenPreviousIsSuspended="on"* after *omelasticsearch*
action with *queue.type="linkedlist"* won't be executed when
*omelasticsearch* is suspended. I'm not sure if this is a bug or
intended behavior. May be related to
https://github.com/rsyslog/rsyslog/issues/1110

How to reproduce:

- build 8.1901.0 from tarball with *--enable-elasticsearch*
*--enable-impstats*
- configure rsyslog:

cat > rsyslog.conf <<EOF
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

global(
        action.reportSuspension="on"
        workDirectory="/var/spool/rsyslog"
        umask="0022"
        processInternalMessages="on"
)

module(load="imudp")
module(load="omelasticsearch")
module(
        load="impstats"
        interval="60"
        log.syslog="off"
        log.file="/var/log/rsyslog_stats.log"
)

input(type="imudp" port="5514" ruleset="el")

ruleset(name="el") {
        action(
                type="omelasticsearch"
                server="localhost:9200"
                searchIndex="test"
                bulkmode="on"
                maxbytes="100m"
                queue.type="linkedlist"
                queue.size="1000"
                queue.dequeueBatchSize="500"
                errorFile="/var/log/rsyslog_to_es_fails.log"
                action.resumeRetryCount="1"
        )
        action(
                type="omfile"
                File="/var/log/rsyslog_failover.log"
                action.execOnlyWhenPreviousIsSuspended="on"
        )
        stop
}


*.* -/var/log/catchall.log
EOF

- run rsyslog, and send sample data to it. e.g. using python script

cat > syslog.py <<EOF
import logging
import time
from logging.handlers import SysLogHandler

l = logging.getLogger(__name__)
h = SysLogHandler(address=('localhost', 5514))
l.addHandler(h)

while True:
    l.error("It's {}".format(time.time()))
    time.sleep(1)
EOF

- Stop elasticsearch node
- Watch rsyslog suspending omelasticsearch queue

Jan 31 13:23:18 rsyslog-dbg rsyslogd-2007: action
'action-0-omelasticsearch' suspended (module 'omelasticsearch'), retry
0. There should be messages before this one giving the reason for
suspension. [v8.1901.0 try https://www.rsyslog.com/e/2007 ]
Jan 31 13:23:19 rsyslog-dbg rsyslogd-2007: omelasticsearch: checkConn
failed after 1 attempts. [v8.1901.0 try https://www.rsyslog.com/e/2007 ]
Jan 31 13:23:19 rsyslog-dbg rsyslogd-2007: action
'action-0-omelasticsearch' suspended (module 'omelasticsearch'), next
retry is Thu Jan 31 13:23:49 2019, retry nbr 0. There should be messages
before this one giving the reason for suspension. [v8.1901.0 try
https://www.rsyslog.com/e/2007 ]

- Notice absence of */var/log/rsyslog_failover.log*
- However, when elasticsearch action is in "Direct" mode, failover works
fine

Debug log in attachment. Shut down elasticsearch node approx on line 44035
Re: action with action.execOnlyWhenPreviousIsSuspended="on" after asynchronous omelasticsearch action won't run [ In reply to ]
El jue., 31 ene. 2019 a las 16:48, Kirill Shestakov via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hi. I've noticed that an action with
> *action.execOnlyWhenPreviousIsSuspended="on"* after *omelasticsearch*
> action with *queue.type="linkedlist"* won't be executed when
> *omelasticsearch* is suspended. I'm not sure if this is a bug or
> intended behavior.

That's intended, because the action actually does not fail when it is
run on a queue.

Rainer
> May be related to
> https://github.com/rsyslog/rsyslog/issues/1110
>
> How to reproduce:
>
> - build 8.1901.0 from tarball with *--enable-elasticsearch*
> *--enable-impstats*
> - configure rsyslog:
>
> cat > rsyslog.conf <<EOF
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>
> global(
> action.reportSuspension="on"
> workDirectory="/var/spool/rsyslog"
> umask="0022"
> processInternalMessages="on"
> )
>
> module(load="imudp")
> module(load="omelasticsearch")
> module(
> load="impstats"
> interval="60"
> log.syslog="off"
> log.file="/var/log/rsyslog_stats.log"
> )
>
> input(type="imudp" port="5514" ruleset="el")
>
> ruleset(name="el") {
> action(
> type="omelasticsearch"
> server="localhost:9200"
> searchIndex="test"
> bulkmode="on"
> maxbytes="100m"
> queue.type="linkedlist"
> queue.size="1000"
> queue.dequeueBatchSize="500"
> errorFile="/var/log/rsyslog_to_es_fails.log"
> action.resumeRetryCount="1"
> )
> action(
> type="omfile"
> File="/var/log/rsyslog_failover.log"
> action.execOnlyWhenPreviousIsSuspended="on"
> )
> stop
> }
>
>
> *.* -/var/log/catchall.log
> EOF
>
> - run rsyslog, and send sample data to it. e.g. using python script
>
> cat > syslog.py <<EOF
> import logging
> import time
> from logging.handlers import SysLogHandler
>
> l = logging.getLogger(__name__)
> h = SysLogHandler(address=('localhost', 5514))
> l.addHandler(h)
>
> while True:
> l.error("It's {}".format(time.time()))
> time.sleep(1)
> EOF
>
> - Stop elasticsearch node
> - Watch rsyslog suspending omelasticsearch queue
>
> Jan 31 13:23:18 rsyslog-dbg rsyslogd-2007: action
> 'action-0-omelasticsearch' suspended (module 'omelasticsearch'), retry
> 0. There should be messages before this one giving the reason for
> suspension. [v8.1901.0 try https://www.rsyslog.com/e/2007 ]
> Jan 31 13:23:19 rsyslog-dbg rsyslogd-2007: omelasticsearch: checkConn
> failed after 1 attempts. [v8.1901.0 try https://www.rsyslog.com/e/2007 ]
> Jan 31 13:23:19 rsyslog-dbg rsyslogd-2007: action
> 'action-0-omelasticsearch' suspended (module 'omelasticsearch'), next
> retry is Thu Jan 31 13:23:49 2019, retry nbr 0. There should be messages
> before this one giving the reason for suspension. [v8.1901.0 try
> https://www.rsyslog.com/e/2007 ]
>
> - Notice absence of */var/log/rsyslog_failover.log*
> - However, when elasticsearch action is in "Direct" mode, failover works
> fine
>
> Debug log in attachment. Shut down elasticsearch node approx on line 44035
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.