Mailing List Archive

MISSING_SUBJECT rule on email with subject
Hi,

We're currently seeing the rule MISSING_SUBJECT sporadically hitting on
emails that have a subject. This issue seems to have started during last
week, which is when clients started complaining about false positive
detections. Please see example headers at the following link:

https://pastebin.com/raw/GtnV67Hj

Has anyone seen the same or similar issue recently? If not, can anyone
offer some advice or guidance?

Thanks!
Stephan
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
On 3 Jun 2019, at 2:20, Stephan Fourie wrote:

> Hi,
>
> We're currently seeing the rule MISSING_SUBJECT sporadically hitting
> on emails that have a subject. This issue seems to have started during
> last week, which is when clients started complaining about false
> positive detections. Please see example headers at the following link:
>
> https://pastebin.com/raw/GtnV67Hj

The headers are all missing the traditional space between the colon and
the header content. This is formally allowable (see
https://tools.ietf.org/html/rfc5322#appendix-A.5,) but it may be
breaking the parsing of the message. More significantly, there are what
appear to be continuation parts of folded headers which have no leading
whitespace, which is NOT allowable and will definitely break parsing.

Is this an artifact of how you copied the message or is it really that
way? If the misformatting is being done by something before SpamAssassin
sees it, SA will parse the headers incorrectly.
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
On Mon, 03 Jun 2019 11:43:44 -0400
Bill Cole wrote:

> On 3 Jun 2019, at 2:20, Stephan Fourie wrote:
>
> > Hi,
> >
> > We're currently seeing the rule MISSING_SUBJECT sporadically
> > hitting on emails that have a subject. This issue seems to have
> > started during last week, which is when clients started complaining
> > about false positive detections. Please see example headers at the
> > following link:
> >
> > https://pastebin.com/raw/GtnV67Hj
>
> The headers are all missing the traditional space between the colon
> and the header content.

And this include google headers, so presumably the spaces have been
stripped locally.
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
>> On 3 Jun 2019, at 2:20, Stephan Fourie wrote:
>> > We're currently seeing the rule MISSING_SUBJECT sporadically
>> > hitting on emails that have a subject. This issue seems to have
>> > started during last week, which is when clients started complaining
>> > about false positive detections. Please see example headers at the
>> > following link:
>> >
>> > https://pastebin.com/raw/GtnV67Hj

>On Mon, 03 Jun 2019 11:43:44 -0400 Bill Cole wrote:
>> The headers are all missing the traditional space between the colon
>> and the header content.

On 03.06.19 19:11, RW wrote:
>And this include google headers, so presumably the spaces have been
>stripped locally.

now one question is,
if the spaces have been stripped prior to spam checking,
another is,
if SA does/should expect whitespaces after header fields.

if the first answer is true, then SA can't do much about misformatted
e-mail.

But since FROM_AND_TO_IS_SAME_DOMAIN was hit, I don't think the spaces were
stripped, so

- we need to see the original message as it was scanned. Anything else,
reformated by anyone (e.g. outlook or exchange use to reformat mail),
can't help us much finding the issue.

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux IS user friendly, it's just selective who its friends are...
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
Hi,

My apologies, seems something went wrong with the formatting when it was
pasted to the pastebin. Here's a new example with spacing intact:
https://pastebin.com/raw/tQtSMQPs

In this example some of the other headers were also not 'seen'.

Thanks!
Stephan

On 2019/06/04 10:55, Matus UHLAR - fantomas wrote:
>>> On 3 Jun 2019, at 2:20, Stephan Fourie wrote:
>>> > We're currently seeing the rule MISSING_SUBJECT sporadically
>>> > hitting on emails that have a subject. This issue seems to have
>>> > started during last week, which is when clients started complaining
>>> > about false positive detections. Please see example headers at the
>>> > following link:
>>> >
>>> > https://pastebin.com/raw/GtnV67Hj
>
>> On Mon, 03 Jun 2019 11:43:44 -0400 Bill Cole wrote:
>>> The headers are all missing the traditional space between the colon
>>> and the header content.
>
> On 03.06.19 19:11, RW wrote:
>> And this include google headers, so presumably the spaces have been
>> stripped locally.
>
> now one question is,
> if the spaces have been stripped prior to spam checking,
> another is,
> if SA does/should expect whitespaces after header fields.
>
> if the first answer is true, then SA can't do much about misformatted
> e-mail.
>
> But since FROM_AND_TO_IS_SAME_DOMAIN was hit, I don't think the spaces
> were
> stripped, so
>
> - we need to see the original message as it was scanned. Anything else,
>  reformated by anyone (e.g. outlook or exchange use to reformat mail),
> can't help us much finding the issue.
>
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
On 04.06.19 16:29, Stephan Fourie wrote:
>My apologies, seems something went wrong with the formatting when it
>was pasted to the pastebin. Here's a new example with spacing intact:
>https://pastebin.com/raw/tQtSMQPs
>
>In this example some of the other headers were also not 'seen'.

there's something strange:

1.0 HK_RANDOM_FROM From username looks random
0.5 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (xxxxxxxxxxxxx[at]gmail.com)


1.0 MISSING_FROM Missing From: header
1.8 MISSING_SUBJECT Missing Subject: header

so the spam scanner both did and did not see the From: header.

What do you use for mail scanning?

>On 2019/06/04 10:55, Matus UHLAR - fantomas wrote:
>>>>On 3 Jun 2019, at 2:20, Stephan Fourie wrote:
>>>>> We're currently seeing the rule MISSING_SUBJECT sporadically
>>>>> hitting on emails that have a subject. This issue seems to have
>>>>> started during last week, which is when clients started complaining
>>>>> about false positive detections. Please see example headers at the
>>>>> following link:
>>>>>
>>>>> https://pastebin.com/raw/GtnV67Hj
>>
>>>On Mon, 03 Jun 2019 11:43:44 -0400 Bill Cole wrote:
>>>>The headers are all missing the traditional space between the colon
>>>>and the header content.
>>
>>On 03.06.19 19:11, RW wrote:
>>>And this include google headers, so presumably the spaces have been
>>>stripped locally.
>>
>>now one question is,
>>if the spaces have been stripped prior to spam checking,
>>another is,
>>if SA does/should expect whitespaces after header fields.
>>
>>if the first answer is true, then SA can't do much about misformatted
>>e-mail.
>>
>>But since FROM_AND_TO_IS_SAME_DOMAIN was hit, I don't think the
>>spaces were
>>stripped, so
>>
>>- we need to see the original message as it was scanned. Anything else,
>>?reformated by anyone (e.g. outlook or exchange use to reformat mail),
>>can't help us much finding the issue.
>>
>

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Microsoft dick is soft to do no harm
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
Hi,

my guess is that for some reason an empty line is inserted in the email
somewhere above the headers and before the message is processed by
spamassassin. This will cause all headers below this empty line to be
treated as the actual body of the message, so all missing header tests
will hit and will result in what you actually see. This could be a bug
in the software you use for email content filtering...

Regards,

Savvas Karagiannidis


On 04/06/2019 17:29, Stephan Fourie wrote:
> Hi,
>
> My apologies, seems something went wrong with the formatting when it
> was pasted to the pastebin. Here's a new example with spacing intact:
> https://pastebin.com/raw/tQtSMQPs
>
> In this example some of the other headers were also not 'seen'.
>
> Thanks!
> Stephan
>
> On 2019/06/04 10:55, Matus UHLAR - fantomas wrote:
>>>> On 3 Jun 2019, at 2:20, Stephan Fourie wrote:
>>>> > We're currently seeing the rule MISSING_SUBJECT sporadically
>>>> > hitting on emails that have a subject. This issue seems to have
>>>> > started during last week, which is when clients started complaining
>>>> > about false positive detections. Please see example headers at the
>>>> > following link:
>>>> >
>>>> > https://pastebin.com/raw/GtnV67Hj
>>
>>> On Mon, 03 Jun 2019 11:43:44 -0400 Bill Cole wrote:
>>>> The headers are all missing the traditional space between the colon
>>>> and the header content.
>>
>> On 03.06.19 19:11, RW wrote:
>>> And this include google headers, so presumably the spaces have been
>>> stripped locally.
>>
>> now one question is,
>> if the spaces have been stripped prior to spam checking,
>> another is,
>> if SA does/should expect whitespaces after header fields.
>>
>> if the first answer is true, then SA can't do much about misformatted
>> e-mail.
>>
>> But since FROM_AND_TO_IS_SAME_DOMAIN was hit, I don't think the
>> spaces were
>> stripped, so
>>
>> - we need to see the original message as it was scanned. Anything else,
>>  reformated by anyone (e.g. outlook or exchange use to reformat mail),
>> can't help us much finding the issue.
>>
>
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
On Tue, 4 Jun 2019 18:10:51 +0300
Savvas Karagiannidis wrote:

> Hi,
>
> my guess is that for some reason an empty line is inserted in the
> email somewhere above the headers and before the message is processed
> by spamassassin. This will cause all headers below this empty line to
> be treated as the actual body of the message, so all missing header
> tests will hit and will result in what you actually see.

But as has already been pointed out it has the combination of
MISSING_FROM and HK_RANDOM_FROM, and the latter is based on a From:addr
test.
RE: MISSING_SUBJECT rule on email with subject [ In reply to ]
> But as has already been pointed out it has the combination of
> MISSING_FROM and HK_RANDOM_FROM, and the latter is based on a
> From:addr test.

I saw this too, however, I thought I noticed a potentially bad regex (from another custom rule) breaking mine. I think this is the case as when I removed the rule, it stopped the missing_subject stopped hitting.
However, I'm still testing.
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
Hi Charles,

My apologies, I forgot to provide feedback to the mailing list. Bad
regex was the cause of this problem for us, too. As soon as the custom
rule was fixed, the problem went away.

Kind Regards,
Stephan

On 2019/06/24 15:58, Charles Amstutz wrote:
>> But as has already been pointed out it has the combination of
>> MISSING_FROM and HK_RANDOM_FROM, and the latter is based on a
>> From:addr test.
> I saw this too, however, I thought I noticed a potentially bad regex (from another custom rule) breaking mine. I think this is the case as when I removed the rule, it stopped the missing_subject stopped hitting.
> However, I'm still testing.
RE: MISSING_SUBJECT rule on email with subject [ In reply to ]
> Hi Charles,
>
> My apologies, I forgot to provide feedback to the mailing list. Bad regex was
> the cause of this problem for us, too. As soon as the custom rule was fixed,
> the problem went away.


If I can ask, was it an incorrectly escaped special character? I think it is the @ symbol breaking mine.
Re: MISSING_SUBJECT rule on email with subject [ In reply to ]
Hi Charles,

Yes, it was an incorrectly escaped forward slash in a subject rule.

On 2019/06/24 16:12, Charles Amstutz wrote:
>> Hi Charles,
>>
>> My apologies, I forgot to provide feedback to the mailing list. Bad regex was
>> the cause of this problem for us, too. As soon as the custom rule was fixed,
>> the problem went away.
>
> If I can ask, was it an incorrectly escaped special character? I think it is the @ symbol breaking mine.