Mailing List Archive

Making sure I understand execOnlyWhenPreviousIsSuspended correctly,
Hi,


I'm sure this question has been answered multiple times, I do however just want to confirm/make sure I'm not misunderstanding anything. I actually can't find the exact documentation for this.

We recently configured two rsyslog-servers (aka. central log servers), they use the same shared storage.
Each client is configured to send messages to log server A, and if that fails, it should send to log server B.
If log server B also is unavailable, client will "cache its logs" according to its queue configuration.
Log server B is configured with a different template for its actions when receiving logs, thus avoiding writing to the same file on the shared filesystem.

That works well, and as intended. With the following caveat,
If a client send logs (while true ; do echo $(date) | logger -t foo && sleep 1; done) to log server A, and I stop rsyslog on log server A, it will take about a minute or two before logserver B starts writing logs to disk.
Those messages that are generated during that minute/two minutes, are now in the client's memory only. They do not show up in the logfile when logserver B starts writing.
If I now start rsyslog on log server A, those messages (the ones in the memory of the client), are now being written down to disk by logserver A.

So that means, if I manually take down logserver A, and a client stops rsyslog/crashes or similar, its messages (that were written 'while the failover/detection of failover') will be discarded.

Is that correctly understood based on the following client configuration,

action(
port = "20515"
target = "log server a"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
)

action(
queue.type = "LinkedList"
queue.filename = "queue_pelslog2.trioptima.net"
action.resumeRetryCount = "-1"
queue.saveonshutdown = "on"
queue.maxdiskspace = "5g"
port = "20515"
target = "log server b"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
action.execOnlyWhenPreviousIsSuspended = "on"
)

And if so, is there a better way of doing it ? Or is the only way to not end up in a scenario like this to always send my logs to multiple log servers ?

With the risk of misunderstanding something, I just don't understand why "the logs that are being generated during the failover cant be flushed to action 2 - why are they only getting written when log server A gets available again).
If I set action 1 to be a queue of linkedlist, action 2 never seems to get triggered. As I understand it, only "direct queues" can be used to trigger "execOnlyWhenPreviousIsSuspended", is that correct ?
Or. at least in the scenario above, direct queue is the only option, because setting action 1 to linkedlist will not make it fail just because the target gets unavailable, since it will write the message to the queue, right ?

Hope I didn't make it to confusing.

Best regards,

Patrik Martinsson?-?Linux System Administrator M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden ?E?patrik.martinsson@trioptima.com?
W?www.trioptima.com???
YT?Genuine Happiness !??
Please see the disclaimer:?www.trioptima.com/email

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
what I would do is
ruleset(name="forwarding"
queue.type = "LinkedList"
queue.filename = "queue_pelslog2.trioptima.net"
action.resumeRetryCount = "-1"
queue.saveonshutdown = "on"
queue.maxdiskspace = "5g")
{
action(
port = "20515"
target = "log server a"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
)

action(
port = "20515"
target = "log server b"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
action.execOnlyWhenPreviousIsSuspended = "on"
)
}

so that the queue is separate from the two actions

I suspect that the omrelp module is keeping some messages in it's memory before
it suspends, but I'd need Rainer to comment on that and what can be done there.

David Lang

On Wed, 4 Sep 2019, Patrik Martinsson via rsyslog wrote:

> Date: Wed, 4 Sep 2019 14:47:34 +0000
> From: Patrik Martinsson via rsyslog <rsyslog@lists.adiscon.com>
> To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
> Cc: Patrik Martinsson <patrik.martinsson@trioptima.com>
> Subject: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended
> correctly,
>
> Hi,
>
>
> I'm sure this question has been answered multiple times, I do however just want to confirm/make sure I'm not misunderstanding anything. I actually can't find the exact documentation for this.
>
> We recently configured two rsyslog-servers (aka. central log servers), they use the same shared storage.
> Each client is configured to send messages to log server A, and if that fails, it should send to log server B.
> If log server B also is unavailable, client will "cache its logs" according to its queue configuration.
> Log server B is configured with a different template for its actions when receiving logs, thus avoiding writing to the same file on the shared filesystem.
>
> That works well, and as intended. With the following caveat,
> If a client send logs (while true ; do echo $(date) | logger -t foo && sleep 1; done) to log server A, and I stop rsyslog on log server A, it will take about a minute or two before logserver B starts writing logs to disk.
> Those messages that are generated during that minute/two minutes, are now in the client's memory only. They do not show up in the logfile when logserver B starts writing.
> If I now start rsyslog on log server A, those messages (the ones in the memory of the client), are now being written down to disk by logserver A.
>
> So that means, if I manually take down logserver A, and a client stops rsyslog/crashes or similar, its messages (that were written 'while the failover/detection of failover') will be discarded.
>
> Is that correctly understood based on the following client configuration,
>
> action(
> port = "20515"
> target = "log server a"
> type = "omrelp"
> template = "RSYSLOG_ForwardFormat"
> )
>
> action(
> queue.type = "LinkedList"
> queue.filename = "queue_pelslog2.trioptima.net"
> action.resumeRetryCount = "-1"
> queue.saveonshutdown = "on"
> queue.maxdiskspace = "5g"
> port = "20515"
> target = "log server b"
> type = "omrelp"
> template = "RSYSLOG_ForwardFormat"
> action.execOnlyWhenPreviousIsSuspended = "on"
> )
>
> And if so, is there a better way of doing it ? Or is the only way to not end up in a scenario like this to always send my logs to multiple log servers ?
>
> With the risk of misunderstanding something, I just don't understand why "the logs that are being generated during the failover cant be flushed to action 2 - why are they only getting written when log server A gets available again).
> If I set action 1 to be a queue of linkedlist, action 2 never seems to get triggered. As I understand it, only "direct queues" can be used to trigger "execOnlyWhenPreviousIsSuspended", is that correct ?
> Or. at least in the scenario above, direct queue is the only option, because setting action 1 to linkedlist will not make it fail just because the target gets unavailable, since it will write the message to the queue, right ?
>
> Hope I didn't make it to confusing.
>
> Best regards,
>
> Patrik Martinsson?-?Linux System Administrator M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden ?E?patrik.martinsson@trioptima.com?
> W?www.trioptima.com???
> YT?Genuine Happiness !??
> Please see the disclaimer:?www.trioptima.com/email
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
Hi David,

Thanks for your input and suggestions.
I tried it, but experienced the same behavior, I attach my full config, and I can also attach full debug from the client if that is needed.

$ > cat /etc/rsyslog.conf

# Basic rsyslog configuration that will be used on all servers.
#
# Templates, rulesets, actions, extra inputs, etc. will be included
# from the /etc/rsyslog.d/ directory.
#

# Global directives,
global(
PreserveFQDN = "on"
WorkDirectory = "/var/lib/rsyslog"
)

# Load modules,
module(load = "imklog")
module(load = "omrelp")
module(load = "imuxsock")

# Use the old way of including files, since the new syntax is introduced
# version in 8.33,
$IncludeConfig /etc/rsyslog.d/*.conf

$ > cat /etc/rsyslog.d/00-actions-relp.conf

ruleset(
name="remote-test"
queue.type = "LinkedList"
queue.filename = "queue_forwarding"
queue.saveonshutdown = "on"
queue.maxdiskspace = "5g"
queue.dequeuebatchsize ="1" # I've tried both with this and without, no difference
){
action(port = "20515"
target = "log server A"
type = "omrelp"
template = "RSYSLOG_ForwardFormat")

action(port = "20515"
target = "log server B"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
action.execOnlyWhenPreviousIsSuspended = "on"
action.resumeRetryCount = "-1")
}

$ > /etc/rsyslog.d/10-actions.conf

call remote-test
*.info;mail.none;authpriv.none;cron.none action(type = "omfile" dynafile = "messages")
authpriv.* action(type = "omfile" dynafile = "secure")
mail.* action(type = "omfile" dynafile = "maillog")
cron.* action(type = "omfile" dynafile = "cron")
uucp,news.crit action(type = "omfile" dynafile = "spooler")
local7.* action(type = "omfile" dynafile = "boot")
kern.* action(type = "omfile" dynafile = "kernel")

# Handle separately,
*.emerg action(type = "omfile" file=":omusrmsg:*")


Forgot to mention our rsyslog-client version,
- rsyslog-8.24.0-2.el6.x86_64

On the log server we have,
- rsyslog-8.24.0-41.el7_7.x86_64


Just to make sure I'm explaining my self,

Experienced behavior,
1. Client sends logs
2. Log server A ("primary log server"), receives them and writes them to disk.
3. I stop rsyslog on Log server A.
4. While its shutting down (a minute or two), the client seems to "buffer messages".
5. When rsyslog-server on log server A is fully shut down, Log server B starts to write messages.
6. However, messages that were produced on the client during the time Log server A was shutting down, seems to only be in the "memory of the client".
7. Now, if I start rsyslog-server on Log server A again, the client seems to "flush" its messages and they get written down to disk by Log server A.
8. If I shutdown rsyslog on the client / or the client crashes, before I have started Log server A, those messages gets lost.


What I want,
1. The behavior described above, except the part where messages gets lost if Log server A is down. I would be more than happy if the client could write those messages to a disk-queue instead of discarding them.

What I mean is, I don't want the messages that gets produced while Log server A is shutting down to ever get lost.

I do not however know if that is possible, but it doesn't feel right that we possible can loose a minute or two worth of messages - or at least, I would like to know if that is the case and if we can do something about it.

Best regards,
Patrik Martinsson





Patrik Martinsson - Linux System Administrator
M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden
E patrik.martinsson@trioptima.com
W www.trioptima.com
YT Genuine Happiness !

Please see the disclaimer: www.trioptima.com/email


________________________________________
From: David Lang <david@lang.hm>
Sent: Wednesday, September 4, 2019 8:39 PM
To: Patrik Martinsson via rsyslog
Cc: Patrik Martinsson
Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,

what I would do is
ruleset(name="forwarding"
queue.type = "LinkedList"
queue.filename = "queue_pelslog2.trioptima.net"
action.resumeRetryCount = "-1"
queue.saveonshutdown = "on"
queue.maxdiskspace = "5g")
{
action(
port = "20515"
target = "log server a"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
)

action(
port = "20515"
target = "log server b"
type = "omrelp"
template = "RSYSLOG_ForwardFormat"
action.execOnlyWhenPreviousIsSuspended = "on"
)
}

so that the queue is separate from the two actions

I suspect that the omrelp module is keeping some messages in it's memory before
it suspends, but I'd need Rainer to comment on that and what can be done there.

David Lang

On Wed, 4 Sep 2019, Patrik Martinsson via rsyslog wrote:

> Date: Wed, 4 Sep 2019 14:47:34 +0000
> From: Patrik Martinsson via rsyslog <rsyslog@lists.adiscon.com>
> To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
> Cc: Patrik Martinsson <patrik.martinsson@trioptima.com>
> Subject: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended
> correctly,
>
> Hi,
>
>
> I'm sure this question has been answered multiple times, I do however just want to confirm/make sure I'm not misunderstanding anything. I actually can't find the exact documentation for this.
>
> We recently configured two rsyslog-servers (aka. central log servers), they use the same shared storage.
> Each client is configured to send messages to log server A, and if that fails, it should send to log server B.
> If log server B also is unavailable, client will "cache its logs" according to its queue configuration.
> Log server B is configured with a different template for its actions when receiving logs, thus avoiding writing to the same file on the shared filesystem.
>
> That works well, and as intended. With the following caveat,
> If a client send logs (while true ; do echo $(date) | logger -t foo && sleep 1; done) to log server A, and I stop rsyslog on log server A, it will take about a minute or two before logserver B starts writing logs to disk.
> Those messages that are generated during that minute/two minutes, are now in the client's memory only. They do not show up in the logfile when logserver B starts writing.
> If I now start rsyslog on log server A, those messages (the ones in the memory of the client), are now being written down to disk by logserver A.
>
> So that means, if I manually take down logserver A, and a client stops rsyslog/crashes or similar, its messages (that were written 'while the failover/detection of failover') will be discarded.
>
> Is that correctly understood based on the following client configuration,
>
> action(
> port = "20515"
> target = "log server a"
> type = "omrelp"
> template = "RSYSLOG_ForwardFormat"
> )
>
> action(
> queue.type = "LinkedList"
> queue.filename = "queue_pelslog2.trioptima.net"
> action.resumeRetryCount = "-1"
> queue.saveonshutdown = "on"
> queue.maxdiskspace = "5g"
> port = "20515"
> target = "log server b"
> type = "omrelp"
> template = "RSYSLOG_ForwardFormat"
> action.execOnlyWhenPreviousIsSuspended = "on"
> )
>
> And if so, is there a better way of doing it ? Or is the only way to not end up in a scenario like this to always send my logs to multiple log servers ?
>
> With the risk of misunderstanding something, I just don't understand why "the logs that are being generated during the failover cant be flushed to action 2 - why are they only getting written when log server A gets available again).
> If I set action 1 to be a queue of linkedlist, action 2 never seems to get triggered. As I understand it, only "direct queues" can be used to trigger "execOnlyWhenPreviousIsSuspended", is that correct ?
> Or. at least in the scenario above, direct queue is the only option, because setting action 1 to linkedlist will not make it fail just because the target gets unavailable, since it will write the message to the queue, right ?
>
> Hope I didn't make it to confusing.
>
> Best regards,
>
> Patrik Martinsson - Linux System Administrator M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden E patrik.martinsson@trioptima.com
> W www.trioptima.com
> YT Genuine Happiness !
> Please see the disclaimer: www.trioptima.com/email
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
El mié., 4 sept. 2019 a las 20:39, David Lang via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> I suspect that the omrelp module is keeping some messages in it's memory before
> it suspends, but I'd need Rainer to comment on that and what can be done there.

Depends on config and when the OS notifies us. There is the RELP
window size (basically the same as TCP window). This may fill if we do
not get an OS error quickly enough. If detecting broken session
immediately is ultra-vital, the RELP window can be set to 1, but be
aware that it has performance consequences. Depending on network, they
may be severe.

doc: https://www.rsyslog.com/doc/v8-stable/configuration/modules/omrelp.html#windowsize

HTH
Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
On Thu, 5 Sep 2019, Rainer Gerhards wrote:

>> I suspect that the omrelp module is keeping some messages in it's memory before
>> it suspends, but I'd need Rainer to comment on that and what can be done there.
>
> Depends on config and when the OS notifies us. There is the RELP
> window size (basically the same as TCP window). This may fill if we do
> not get an OS error quickly enough. If detecting broken session
> immediately is ultra-vital, the RELP window can be set to 1, but be
> aware that it has performance consequences. Depending on network, they
> may be severe.
>
> doc: https://www.rsyslog.com/doc/v8-stable/configuration/modules/omrelp.html#windowsize

rephrasing to be sure I understand what's happening here.

RELP takes the messages off of the queue and keeps track of them internally
(the window is essentially a mini queue inside RELP that is a configuable size).

When the connection is lost and the RELP output is suspended, these messages are
stuck inside RELP. If the connection is re-established, they will be sent, but
there is currently no way to get them saved to disk or otherwise back out of
RELP at this time.

I'm assuming that it would take fairly significant sugery to change this (as a
brainstorming idea, could the internal window queue be replaced by calls to the
'normal' queue code, enabling disk assisted queues, save on shutdown, etc)

I suspect that this is a common problem with all the output modules that
support delayed delivery (elasticsearch, kafka, probably not the database
outputs since they do atomic commmits), am I right?

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
Ah ok, well that makes sense then - the behavior I'm experiencing is exactly like that.

Without knowing the internals of rsyslog, I was just surprised that I could loose messages when having two actions configured with RELP.
I was sure that the failover and 'action.execOnlyWhenPreviousIsSuspended' would take care of it.

But I understand I was wrong in that assumption.

So, then I fall back to, the only way to not loose messages when Log Server A reboots/goes down/etc. is to *always* send my logs to both Log Servers at the same time ?

Ps. I actually tried setting windowSitze to 1 just to see if fewer messages were "lost", but I didn't notice any difference.
The setting suppose to be on the first action right ?

Thanks for the deep-diving of rsyslog internals and RELP.

Patrik Martinsson - Linux System Administrator
M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden
E patrik.martinsson@trioptima.com
W www.trioptima.com
YT Genuine Happiness !

Please see the disclaimer: www.trioptima.com/email


________________________________________
From: rsyslog <rsyslog-bounces@lists.adiscon.com> on behalf of David Lang via rsyslog <rsyslog@lists.adiscon.com>
Sent: Thursday, September 5, 2019 12:23 PM
To: Rainer Gerhards
Cc: David Lang; rsyslog-users
Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,

On Thu, 5 Sep 2019, Rainer Gerhards wrote:

>> I suspect that the omrelp module is keeping some messages in it's memory before
>> it suspends, but I'd need Rainer to comment on that and what can be done there.
>
> Depends on config and when the OS notifies us. There is the RELP
> window size (basically the same as TCP window). This may fill if we do
> not get an OS error quickly enough. If detecting broken session
> immediately is ultra-vital, the RELP window can be set to 1, but be
> aware that it has performance consequences. Depending on network, they
> may be severe.
>
> doc: https://www.rsyslog.com/doc/v8-stable/configuration/modules/omrelp.html#windowsize

rephrasing to be sure I understand what's happening here.

RELP takes the messages off of the queue and keeps track of them internally
(the window is essentially a mini queue inside RELP that is a configuable size).

When the connection is lost and the RELP output is suspended, these messages are
stuck inside RELP. If the connection is re-established, they will be sent, but
there is currently no way to get them saved to disk or otherwise back out of
RELP at this time.

I'm assuming that it would take fairly significant sugery to change this (as a
brainstorming idea, could the internal window queue be replaced by calls to the
'normal' queue code, enabling disk assisted queues, save on shutdown, etc)

I suspect that this is a common problem with all the output modules that
support delayed delivery (elasticsearch, kafka, probably not the database
outputs since they do atomic commmits), am I right?

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
> Ps. I actually tried setting windowSitze to 1 just to see if fewer messages were "lost", but I didn't notice any difference.
> The setting suppose to be on the first action right ?

yes - if you change it on the first action and all remains the same,
pls open a github issue together with sample conf. I won't be able to
really look at it before october, but that would smell very much like
a bug.

Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
Thanks!

I adjusted it to "30", and noticing a huge difference in messages being "stored in the internal RELP queue on the client".
IE. with 30 in windowSize, I loose about 15 seconds of messages, which correponds well with 2 messages/sec.

Again, thanks for all the useful insights!

PS. I still think it would be a good idea to be able to "save" those messages stored temporarily in the "internal RELP-queue". In that way, one wouldn't need to worry about "loosing messages" if the client reboots/crashes before the primary logserver comes back up again.

Just my two cents.


Patrik Martinsson - Linux System Administrator
M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden
E patrik.martinsson@trioptima.com
W www.trioptima.com
YT Genuine Happiness !

Please see the disclaimer: www.trioptima.com/email


________________________________________
From: Rainer Gerhards <rgerhards@hq.adiscon.com>
Sent: Thursday, September 5, 2019 12:39 PM
To: Patrik Martinsson
Cc: rsyslog-users; David Lang
Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,

> Ps. I actually tried setting windowSitze to 1 just to see if fewer messages were "lost", but I didn't notice any difference.
> The setting suppose to be on the first action right ?

yes - if you change it on the first action and all remains the same,
pls open a github issue together with sample conf. I won't be able to
really look at it before october, but that would smell very much like
a bug.

Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
El jue., 5 sept. 2019 a las 12:55, Patrik Martinsson
(<patrik.martinsson@trioptima.com>) escribió:
>
> Thanks!
>
> I adjusted it to "30", and noticing a huge difference in messages being "stored in the internal RELP queue on the client".
> IE. with 30 in windowSize, I loose about 15 seconds of messages, which correponds well with 2 messages/sec.
>
> Again, thanks for all the useful insights!
>
> PS. I still think it would be a good idea to be able to "save" those messages stored temporarily in the "internal RELP-queue". In that way, one wouldn't need to worry about "loosing messages" if the client reboots/crashes before the primary logserver comes back up again.

Just as an exercise: ask you company if they are willing to fund ~4000
Euros to make this happen? I guess the answer is no - and that says a
bit on priorities

Rainer
>
> Just my two cents.
>
>
> Patrik Martinsson - Linux System Administrator
> Mäster Samuelsgatan 17, 111 44 Stockholm, Sweden
> E patrik.martinsson@trioptima.com
> W www.trioptima.com
> YT Genuine Happiness !
>
> Please see the disclaimer: www.trioptima.com/email
>
>
> ________________________________________
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Thursday, September 5, 2019 12:39 PM
> To: Patrik Martinsson
> Cc: rsyslog-users; David Lang
> Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,
>
> > Ps. I actually tried setting windowSitze to 1 just to see if fewer messages were "lost", but I didn't notice any difference.
> > The setting suppose to be on the first action right ?
>
> yes - if you change it on the first action and all remains the same,
> pls open a github issue together with sample conf. I won't be able to
> really look at it before october, but that would smell very much like
> a bug.
>
> Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
I've actually recently quit my job, and would not be in position to argue for such thing.
But, one would think that someone surely could fund that.

However, I do get your point - it was merely wishful thinking from my side.

Anyways, great job with rsyslog and thanks again for great support and software.

Patrik Martinsson - Linux System Administrator
M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden
E patrik.martinsson@trioptima.com
W www.trioptima.com
YT Genuine Happiness !

Please see the disclaimer: www.trioptima.com/email


________________________________________
From: Rainer Gerhards <rgerhards@hq.adiscon.com>
Sent: Thursday, September 5, 2019 12:58 PM
To: Patrik Martinsson
Cc: rsyslog-users; David Lang
Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,

El jue., 5 sept. 2019 a las 12:55, Patrik Martinsson
(<patrik.martinsson@trioptima.com>) escribi?:
>
> Thanks!
>
> I adjusted it to "30", and noticing a huge difference in messages being "stored in the internal RELP queue on the client".
> IE. with 30 in windowSize, I loose about 15 seconds of messages, which correponds well with 2 messages/sec.
>
> Again, thanks for all the useful insights!
>
> PS. I still think it would be a good idea to be able to "save" those messages stored temporarily in the "internal RELP-queue". In that way, one wouldn't need to worry about "loosing messages" if the client reboots/crashes before the primary logserver comes back up again.

Just as an exercise: ask you company if they are willing to fund ~4000
Euros to make this happen? I guess the answer is no - and that says a
bit on priorities

Rainer
>
> Just my two cents.
>
>
> Patrik Martinsson - Linux System Administrator
> M?ster Samuelsgatan 17, 111 44 Stockholm, Sweden
> E patrik.martinsson@trioptima.com
> W www.trioptima.com
> YT Genuine Happiness !
>
> Please see the disclaimer: www.trioptima.com/email
>
>
> ________________________________________
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Thursday, September 5, 2019 12:39 PM
> To: Patrik Martinsson
> Cc: rsyslog-users; David Lang
> Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,
>
> > Ps. I actually tried setting windowSitze to 1 just to see if fewer messages were "lost", but I didn't notice any difference.
> > The setting suppose to be on the first action right ?
>
> yes - if you change it on the first action and all remains the same,
> pls open a github issue together with sample conf. I won't be able to
> really look at it before october, but that would smell very much like
> a bug.
>
> Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, [ In reply to ]
El jue., 5 sept. 2019 a las 13:03, Patrik Martinsson
(<patrik.martinsson@trioptima.com>) escribió:
>
> I've actually recently quit my job, and would not be in position to argue for such thing.
> But, one would think that someone surely could fund that.

so thought I
>
> However, I do get your point - it was merely wishful thinking from my side.

we have a very large TODO list. I tend to put those items very low
where we had discussions with companies, provided a no-brainer quote
for a big enterprise but it was not accepted because after some
thinking feature wasn't worth it. Even more so if that happened
multiple times. So I came to the conclusion that time here is not well
spend. Technically, it would make a lot of sense, though...

Rainer

>
> Anyways, great job with rsyslog and thanks again for great support and software.
>
> Patrik Martinsson - Linux System Administrator
> Mäster Samuelsgatan 17, 111 44 Stockholm, Sweden
> E patrik.martinsson@trioptima.com
> W www.trioptima.com
> YT Genuine Happiness !
>
> Please see the disclaimer: www.trioptima.com/email
>
>
> ________________________________________
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Thursday, September 5, 2019 12:58 PM
> To: Patrik Martinsson
> Cc: rsyslog-users; David Lang
> Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,
>
> El jue., 5 sept. 2019 a las 12:55, Patrik Martinsson
> (<patrik.martinsson@trioptima.com>) escribió:
> >
> > Thanks!
> >
> > I adjusted it to "30", and noticing a huge difference in messages being "stored in the internal RELP queue on the client".
> > IE. with 30 in windowSize, I loose about 15 seconds of messages, which correponds well with 2 messages/sec.
> >
> > Again, thanks for all the useful insights!
> >
> > PS. I still think it would be a good idea to be able to "save" those messages stored temporarily in the "internal RELP-queue". In that way, one wouldn't need to worry about "loosing messages" if the client reboots/crashes before the primary logserver comes back up again.
>
> Just as an exercise: ask you company if they are willing to fund ~4000
> Euros to make this happen? I guess the answer is no - and that says a
> bit on priorities
>
> Rainer
> >
> > Just my two cents.
> >
> >
> > Patrik Martinsson - Linux System Administrator
> > Mäster Samuelsgatan 17, 111 44 Stockholm, Sweden
> > E patrik.martinsson@trioptima.com
> > W www.trioptima.com
> > YT Genuine Happiness !
> >
> > Please see the disclaimer: www.trioptima.com/email
> >
> >
> > ________________________________________
> > From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Sent: Thursday, September 5, 2019 12:39 PM
> > To: Patrik Martinsson
> > Cc: rsyslog-users; David Lang
> > Subject: Re: [rsyslog] Making sure I understand execOnlyWhenPreviousIsSuspended correctly,
> >
> > > Ps. I actually tried setting windowSitze to 1 just to see if fewer messages were "lost", but I didn't notice any difference.
> > > The setting suppose to be on the first action right ?
> >
> > yes - if you change it on the first action and all remains the same,
> > pls open a github issue together with sample conf. I won't be able to
> > really look at it before october, but that would smell very much like
> > a bug.
> >
> > Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.