Mailing List Archive

omfile: Force instant write
Hi,

currently, while playing with lookup table, mmnormalize and templates, I
need to test things.

Therefore I created some omfile actions to write the current message.

However, this doesn't really happen: I have to send HUP signal to
rsyslogd to get anything written to that file. :/

My action:

action(
type="omfile"
file="/tmp/debug.log"
template="my-debug-template"
)

The message I am sending via logger is very short, something like

$ logger -t mytest "[21/Jan/2019:12:20:41 +0100] foo"

I guess I am not filling the buffer enough. Is there a way to force a
write on every message? Thought setting 'asyncWriting="on"' would
trigger a write every second due to 'flushInterval="1"' but that's not
the case. Also, setting 'ioBufferSize="1"' has not the effect I am
looking for.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
Hi,

mhh... looks like sending HUP signal isn't reloading complete
configuration like expect. After restarting rsyslogd I'll get immediate
output like expected.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
El mié., 23 ene. 2019 a las 7:26, Thomas Deutschmann via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hi,
>
> mhh... looks like sending HUP signal isn't reloading complete
> configuration like expect. After restarting rsyslogd I'll get immediate
> output like expected.

HUP does NOT reload configuration. That works as expected and documented.

Rainer
>
>
> --
> Regards,
> Thomas Deutschmann / Gentoo Linux Developer
> C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
El mié., 23 ene. 2019 a las 7:10, Thomas Deutschmann via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hi,
>
> currently, while playing with lookup table, mmnormalize and templates, I
> need to test things.
>
> Therefore I created some omfile actions to write the current message.
>
> However, this doesn't really happen: I have to send HUP signal to
> rsyslogd to get anything written to that file. :/
>
> My action:
>
> action(
> type="omfile"
> file="/tmp/debug.log"
> template="my-debug-template"
> )

That should and, in all cases I see, does work. Could you send me a debug log?

>
> The message I am sending via logger is very short, something like
>
> $ logger -t mytest "[21/Jan/2019:12:20:41 +0100] foo"
>
> I guess I am not filling the buffer enough. Is there a way to force a
> write on every message? Thought setting 'asyncWriting="on"' would
> trigger a write every second due to 'flushInterval="1"' but that's not
> the case. Also, setting 'ioBufferSize="1"' has not the effect I am
> looking for.

If you enable asyncWriting, there may be a bug. Note that it is highly
discouraged to use asyncWriting for trvial file write - it makes only
sense for time-consuming operations, like zipping or encrypting. For
regular writes, it strongly increases the overhead.

Rainer
>
>
> --
> Regards,
> Thomas Deutschmann / Gentoo Linux Developer
> C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
On Wed, 23 Jan 2019, Thomas Deutschmann via rsyslog wrote:

> I guess I am not filling the buffer enough. Is there a way to force a
> write on every message?

FlushOnTXEnd should do this, but I've also noticed that rsyslog does not write
out to the filesystem until there is 'enough' data or a HUP arrives.

Is it possible that the default has changed?

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
El mié., 23 ene. 2019 a las 9:28, David Lang (<david@lang.hm>) escribió:
>
> On Wed, 23 Jan 2019, Thomas Deutschmann via rsyslog wrote:
>
> > I guess I am not filling the buffer enough. Is there a way to force a
> > write on every message?
>
> FlushOnTXEnd should do this, but I've also noticed that rsyslog does not write
> out to the filesystem until there is 'enough' data or a HUP arrives.
>
> Is it possible that the default has changed?

IIRC there was a bug that I recently fixed (8.39+ ?). I think the
testbench also has test for this. So I am pretty sure by default log
files are written as soon as a batch is done.

Rainer

>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
On Wed, 23 Jan 2019, Rainer Gerhards wrote:

> El mié., 23 ene. 2019 a las 9:28, David Lang (<david@lang.hm>) escribió:
>>
>> On Wed, 23 Jan 2019, Thomas Deutschmann via rsyslog wrote:
>>
>> > I guess I am not filling the buffer enough. Is there a way to force a
>> > write on every message?
>>
>> FlushOnTXEnd should do this, but I've also noticed that rsyslog does not write
>> out to the filesystem until there is 'enough' data or a HUP arrives.
>>
>> Is it possible that the default has changed?
>
> IIRC there was a bug that I recently fixed (8.39+ ?). I think the
> testbench also has test for this. So I am pretty sure by default log
> files are written as soon as a batch is done.

Ok, I haven't tested that recent a version.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
Hi,

the main problem was that for some reason I thought sending HUP would be
enough to activate the configuration changes (maybe I was fooled by the
message "lookup table ... reloaded from file" after sending HUP and by
the fact that for some reason restarting rsyslogd will take ~3 minutes
-- I think this is caused by ommysql but I still need to investigate).

Once I really restarted rsyslogd logs where written like expected and I
stopped investigating.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: omfile: Force instant write [ In reply to ]
On Thu, 24 Jan 2019, Thomas Deutschmann via rsyslog wrote:

> the main problem was that for some reason I thought sending HUP would be
> enough to activate the configuration changes

loading the config file loads libraries, and that could mean that internal data
structures could be different between two different configs. So you have to do a
full stop/start to change the config, and that can loose log messages, so a HUP
closes outputs (needed for log rotation), but doesn't re-parse the config file.

David Lang
k
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.