Mailing List Archive

Rules for invisible div and 0pt font?
Hi all,

In reviewing today's FNs I came across the following spample:
https://pastebin.com/9QQVwUY6

There is a div here with display:none, as well as font-size:0px. The spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule relating to a hidden div or tiny font.

Does LOW_CONTRAST include font-size too small, or just color too light? Is there a rule for matching display:none?

If not, may I propose that the following rules be sandboxed?

rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/

rawbody AC_HIDDEN_FONT /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/

The font one above could be modified for [0-3] or similar, if we want to catch tiny versus literally hidden fonts.

Cheers.

--- Amir
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Monday 17 June 2019 at 21:14:36, Amir Caspi wrote:

> Hi all,
>
> In reviewing today's FNs I came across the following spample:
> https://pastebin.com/9QQVwUY6
>
> There is a div here with display:none, as well as font-size:0px. The
> spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule
> relating to a hidden div or tiny font.
>
> Does LOW_CONTRAST include font-size too small, or just color too light? Is
> there a rule for matching display:none?

Is display:none ever used for instructiosn to screen readers for the blind /
visually impaired?

I have no idea whether it is, but it's a potentially legitimate use which
comes to mind. If not, what is "display:none" actually for?

> If not, may I propose that the following rules be sandboxed?
>
> rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/
>
> rawbody AC_HIDDEN_FONT /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>
> The font one above could be modified for [0-3] or similar, if we want to
> catch tiny versus literally hidden fonts.

If this feature *is* used for screenreaders, you could be creating a false
positive trap here...


Antony.

--
Wanted: telepath. You know where to apply.

Please reply to the list;
please *don't* CC me.
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 17, 2019, at 1:14 PM, Amir Caspi <cepheid@3phase.com> wrote:
> rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/

Since display:none is a pretty common method for showing and hiding elements depending on things like screen size, I would guess this is going to hit mostly ham.

--
It was easy to be a vegetarian by day. It was preventing yourself from
becoming a humanitarian at night that took the real effort.
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 17, 2019, at 1:14 PM, Amir Caspi <Cepheid@3phase.com> wrote:
>
> rawbody AC_HIDDEN_FONT /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>

Actually, based on another spample (https://pastebin.com/rrU2AsVT <https://pastebin.com/rrU2AsVT>), let's modify this one -- the em/pt/px/% isn't required:

rawbody AC_HIDDEN_FONT /font-size\s*:\s*0\s*(?:em|pt|px|%)?\s*;/

It might also be prudent to look for 0-height or 0-width line-height, max-height, max-width... so that would change the hidden-font to:

rawbody AC_HIDDEN_FONT /(?:font-size|line-height|max-height|max-width)\s*:\s*0\s*(?:em|pt|px|%)?\s*;/

And, looks like another rule might be useful:

rawbody AC_LARGE_NEG_INDENT /text-indent\s*:\s*-[0-9]{3,}(?:em|pt|px|%)\s*;/

This looks for a large negative text-indent, as is used in the spample linked above.

Cheers.

--- Amir
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 17, 2019, at 1:18 PM, Antony Stone <Antony.Stone@spamassassin.open.source.it> wrote:
>
> If this feature *is* used for screenreaders, you could be creating a false positive trap here...

You may well be right, hence the request to sandbox and see how it compares against masscheck.

On Jun 17, 2019, at 1:25 PM, @lbutlr <kremels@kreme.com> wrote:
>
> Since display:none is a pretty common method for showing and hiding elements depending on things like screen size, I would guess this is going to hit mostly ham.

Wouldn't that only be true for dynamic content that can actually evaluate the screensize, and hence would require javascript? Or is there a way of doing this with static email content? (I'm very well versed in HTML for web browsers, but not as much for MUAs...)

The font-size, line-height, max-height, max-width of would almost certainly be pretty spammy, I would imagine.

Anyway, that's the whole point of sandboxing...

Cheers!

--- Amir
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 17, 2019, at 1:30 PM, Amir Caspi <cepheid@3phase.com> wrote:
> Wouldn't that only be true for dynamic content that can actually evaluate the screensize, and hence would require javascript? Or is there a way of doing this with static email content? (I'm very well versed in HTML for web browsers, but not as much for MUAs...)

Pretty sure in css you can display based on screen size (well, not screen per se, but display size) without resorting to javascript, but I am not positive.

@media (max-width: 900px) {
… stuff
}

Would only be active if the width of the window is 900px or less. That can include setting a display property to hidden or not.

--
"Kill yourself and roll a rogue. We'll wait"
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 17, 2019, at 1:45 PM, @lbutlr <kremels@kreme.com> wrote:
>
> Would only be active if the width of the window is 900px or less. That can include setting a display property to hidden or not.

One way of working around that, then, would be to ensure this is only within a div/span tag...

Maybe something like:

rawbody AC_HIDDEN_ELEMENT /<(?:div|span|p)\s+[^>]+display\s*:\s*none\s*;/i

One could restrict this further by trying to capture the style= attribute but I don't think that's necessary. This formulation should capture a display:none only in an inline style for div, span, and p elements.

HOWEVER, this formulation also leaves open the very easy workaround of defining a CSS style for hidden elements (e.g., putting display:none within a CSS class definition) and then setting the div/span/p element to that class.

So a DIFFERENT workaround would be to ensure that the display:none doesn't occur within a @media {} block... but because we can't use variable-length lookbehind, the only way I can think of doing that is to check for @media blocks WITH display:none, and the total number of display:none, and if the latter is larger, consider the rule to be hit. I'm not sure how to compare the number of rule hits in SA... but if we wanted to do that, then:

rawbody AC_MEDIA_DISPLAYNONE /@media[^{]*{[^}]*display\s*:\s*none\s*;/i

Then create a meta that hits when AC_HIDDEN_ELEMENT > AC_MEDIA_DISPLAYNONE

Cheers.

--- Amir
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 17, 2019, at 2:17 PM, Amir Caspi <Cepheid@3phase.com> wrote:
>
> rawbody AC_MEDIA_DISPLAYNONE /@media[^{]*{[^}]*display\s*:\s*none\s*;/i
>

Well, urgh, this particular rule wouldn't work well since it wouldn't capture classes within the @media block. But something LIKE it.

--- Amir
Re: Rules for invisible div and 0pt font? [ In reply to ]
On 6/17/19 9:14 PM, Amir Caspi wrote:
> Hi all,
>
> In reviewing today's FNs I came across the following spample:
> https://pastebin.com/9QQVwUY6
>
> There is a div here with display:none, as well as font-size:0px. The spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule relating to a hidden div or tiny font.
>
> Does LOW_CONTRAST include font-size too small, or just color too light? Is there a rule for matching display:none?
>
> If not, may I propose that the following rules be sandboxed?
>
> rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/
>
> rawbody AC_HIDDEN_FONT /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>
> The font one above could be modified for [0-3] or similar, if we want to catch tiny versus literally hidden fonts.
>
> Cheers.
>
> --- Amir
>
There is T_HIDDEN_WORD on my sandbox (https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
I have just committed a more generic version.
Giovanni
Re: Rules for invisible div and 0pt font? [ In reply to ]
On 2019-06-17 21:26, Amir Caspi wrote:> On Jun 17, 2019, at 1:14 PM,
Amir Caspi <Cepheid@3phase.com
> <mailto:Cepheid@3phase.com>> wrote:
>>
>> rawbodyAC_HIDDEN_FONT/font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>>
>
> Actually, based on another spample (https://pastebin.com/rrU2AsVT),
> let's modify this one -- the em/pt/px/% isn't required:
>
> rawbodyAC_HIDDEN_FONT/font-size\s*:\s*0\s*(?:em|pt|px|%)?\s*;/
>
> It might also be prudent to look for 0-height or 0-width line-height,
> max-height, max-width... so that would change the hidden-font to:
>
>
rawbodyAC_HIDDEN_FONT/(?:font-size|line-height|max-height|max-width)\s*:\s*0\s*(?:em|pt|px|%)?\s*;/
>
> And, looks like another rule might be useful:
>
> rawbodyAC_LARGE_NEG_INDENT/text-indent\s*:\s*-[0-9]{3,}(?:em|pt|px|%)\s*;/
>
> This looks for a large negative text-indent, as is used in the spample
> linked above.
>
> Cheers.
>
> --- Amir
>

Don't forget that css also has an "!important" flag that comes before
the semicolon.

rawbodyAC_HIDDEN_FONT/(?:font-size|line-height|max-height|max-width)\s*:\s*0\s*(?:em|pt|px|%)?(?:\s*!important)?\s*;/

BR/Mvh. Dan Malm, Systems Engineer, One.com
Re: Rules for invisible div and 0pt font? [ In reply to ]
On 17 Jun 2019, at 15:25, @lbutlr wrote:

> On Jun 17, 2019, at 1:14 PM, Amir Caspi <cepheid@3phase.com> wrote:
>> rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/
>
> Since display:none is a pretty common method for showing and hiding
> elements depending on things like screen size, I would guess this is
> going to hit mostly ham.

Mail in my personal recent archives matching that includes ham from
USPS, Apple, Home Depot, Office Depot, Paypal, Fidelity, Subway, Kroger,
and others, all of it non-bulk requested and expected mail. In short:
valuable business to consumer transactional and/or account-specific
mail.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Re: Rules for invisible div and 0pt font? [ In reply to ]
Are the matches all within @media blocks like lbutlr suggested or do they occur inline within div/span/etc as well?

Thanks!

--- Amir
thumbed via iPhone

> On Jun 18, 2019, at 8:42 AM, Bill Cole <sausers-20150205@billmail.scconsult.com> wrote:
>
>> On 17 Jun 2019, at 15:25, @lbutlr wrote:
>>
>>> On Jun 17, 2019, at 1:14 PM, Amir Caspi <cepheid@3phase.com> wrote:
>>> rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/
>>
>> Since display:none is a pretty common method for showing and hiding elements depending on things like screen size, I would guess this is going to hit mostly ham.
>
> Mail in my personal recent archives matching that includes ham from USPS, Apple, Home Depot, Office Depot, Paypal, Fidelity, Subway, Kroger, and others, all of it non-bulk requested and expected mail. In short: valuable business to consumer transactional and/or account-specific mail.
>
> --
> Bill Cole
> bill@scconsult.com or billcole@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Re: Rules for invisible div and 0pt font? [ In reply to ]
On 18 Jun 2019, at 10:52, Amir Caspi wrote:

> Are the matches all within @media blocks like lbutlr suggested or do
> they occur inline within div/span/etc as well?

Looking at the 2 most recent (a USPS "Informed Delivery Daily Digest"
message and Office Depot order followup) I see display:none only in
inline style attributes of block elements. e.g.:


<p style=3D"display:none;visibility:hidden">
<br />
<img height=3D"0" width=3D"0" alt=3D""
src=3D"http://pixel.watch/REDACT" styl=
e=3D"display:none;visibility:hidden" />

And:

<!-- mobile - banner -->
<tr style="display:none;width: 0;max-height:
0;line-height:0;overflow: hidden;float:left" class="mobile-show">





>
> Thanks!
>
> --- Amir
> thumbed via iPhone
>
>> On Jun 18, 2019, at 8:42 AM, Bill Cole
>> <sausers-20150205@billmail.scconsult.com> wrote:
>>
>>> On 17 Jun 2019, at 15:25, @lbutlr wrote:
>>>
>>>> On Jun 17, 2019, at 1:14 PM, Amir Caspi <cepheid@3phase.com> wrote:
>>>> rawbody AC_HIDDEN_ELEMENT /display\s*:\s*none\s*;/
>>>
>>> Since display:none is a pretty common method for showing and hiding
>>> elements depending on things like screen size, I would guess this is
>>> going to hit mostly ham.
>>
>> Mail in my personal recent archives matching that includes ham from
>> USPS, Apple, Home Depot, Office Depot, Paypal, Fidelity, Subway,
>> Kroger, and others, all of it non-bulk requested and expected mail.
>> In short: valuable business to consumer transactional and/or
>> account-specific mail.
>>
>> --
>> Bill Cole
>> bill@scconsult.com or billcole@apache.org
>> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 18, 2019, at 10:55 AM, Bill Cole <sausers-20150205@billmail.scconsult.com> wrote:
>
> Looking at the 2 most recent (a USPS "Informed Delivery Daily Digest" message and Office Depot order followup) I see display:none only in inline style attributes of block elements. e.g.:

Looks like the first one is a web bug. The second one is more problematic, because it also includes things like width:0 and max-height:0, which the OTHER rule is intended to catch.

Ugh.

I guess those rules are likely to hit a lot of ham, but maybe there are some good metas...

--- Amir
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Tue, 18 Jun 2019, Giovanni Bechis wrote:

> On 6/17/19 9:14 PM, Amir Caspi wrote:
>> There is a div here with display:none, as well as font-size:0px. The spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule relating to a hidden div or tiny font.
>
> There is T_HIDDEN_WORD on my sandbox (https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
> I have just committed a more generic version.

You probably also want to add "tflags publish" if its performance is
acceptable to you.

--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Your mouse has moved. Your Windows Operating System must be
relicensed due to this hardware change. Please contact Microsoft
to obtain a new activation key. If this hardware change results in
added functionality you may be subject to additional license fees.
Your system will now shut down. Thank you for choosing Microsoft.
-----------------------------------------------------------------------
Today: SWMBO's Birthday
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Tue, 18 Jun 2019 at 19:14, John Hardin <jhardin@impsec.org> wrote:

> On Tue, 18 Jun 2019, Giovanni Bechis wrote:
>
> > On 6/17/19 9:14 PM, Amir Caspi wrote:
> >> There is a div here with display:none, as well as font-size:0px. The
> spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule
> relating to a hidden div or tiny font.
> >
> > There is T_HIDDEN_WORD on my sandbox (
> https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
> > I have just committed a more generic version.
>
> You probably also want to add "tflags publish" if its performance is
> acceptable to you.
>

Also rename from T_ otherwise it will be skipped. If you drop the T_ and
omit the publish it will let QA decide if performance is good enough :)


Paul
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Tue, 18 Jun 2019 at 20:23, Paul Stead <paul.stead@gmail.com> wrote:

> Also rename from T_ otherwise it will be skipped. If you drop the T_ and
> omit the publish it will let QA decide if performance is good enough :)
>

Although not looking so good today -
https://ruleqa.spamassassin.org/20190618-r1861562-n/T_HIDDEN_WORD/detail

Paul
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Tue, 18 Jun 2019, Paul Stead wrote:

> On Tue, 18 Jun 2019 at 19:14, John Hardin <jhardin@impsec.org> wrote:
>
>> On Tue, 18 Jun 2019, Giovanni Bechis wrote:
>>
>>> On 6/17/19 9:14 PM, Amir Caspi wrote:
>>>> There is a div here with display:none, as well as font-size:0px. The
>> spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule
>> relating to a hidden div or tiny font.
>>>
>>> There is T_HIDDEN_WORD on my sandbox (
>> https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
>>> I have just committed a more generic version.
>>
>> You probably also want to add "tflags publish" if its performance is
>> acceptable to you.
>>
>
> Also rename from T_ otherwise it will be skipped. If you drop the T_ and
> omit the publish it will let QA decide if performance is good enough :)

That's only if you explicitly named it with the T_ prefix. That prefix is
automatically added in some cases.

--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
All I could think about was this bear is so close to me I can
see its teeth. I could have kissed it. I wished I had a gun.
-- Alyson Jones-Robinson
-----------------------------------------------------------------------
Today: SWMBO's Birthday
Re: Rules for invisible div and 0pt font? [ In reply to ]
Re: Rules for invisible div and 0pt font? [ In reply to ]
On Jun 18, 2019, at 2:21 AM, Giovanni Bechis <giovanni@paclan.it> wrote:
>
>> rawbody AC_HIDDEN_FONT /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>>
> There is T_HIDDEN_WORD on my sandbox (https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
> I have just committed a more generic version.

Unfortunately I'm still seeing a bunch of spams with font-size: 0px that aren't hitting any sort of "hidden font" or "tiny font" rule.

The above suggested rule would catch those, in case someone can try sandboxing that. I had also suggested matching on line-height:0 and similar, but it appears that those might be used in hams as well... so we might want to limit it to just font-size.

But it looks like Giovanni's T_GB_HIDDEN_WORD isn't scoring so well lately... not sure how it compares to my suggestion above.

Cheers.

--- Amir