Mailing List Archive

#1045: Ban lurker doesn't work anymore
#1045: Ban lurker doesn't work anymore
------------------------+---------------------------------------------------
Reporter: Yvan | Type: defect
Status: new | Priority: high
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: major
Keywords: ban lurker |
------------------------+---------------------------------------------------
I've just upgraded from 3.0.1 to 3.0.2, using Varnish's Debian packages
(deb http://repo.varnish-cache.org/debian/ squeeze varnish-3.0).

It seems that the ban lurker isn't clearing out the ban list, or does it
too slowly.

I've just did a test (on a production server, very low traffic, about 5k
objects in cache):
- restarted varnish (to empty ban list)
- set the ban_lurker_sleep to 0.001s
- sent a ban rule, such as: obj.http.x-host == mydomain.com &&
obj.http.x-url == /index.html
- waited for the ban rule to disappear from "varnishadm ban.list" (using
"watch")

On 3.0.1, it takes about 4 seconds. On 3.0.2 it takes 168 seconds.

Until the upgrade, setting ban_lurker_sleep to 0.001s was enough to empty
the ban list in about 1h. Setting it to 0.000001s after the upgrade only
sees the ban list increase (about 100k rules in 24h, even if most of these
were in Gone state).

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
------------------------+---------------------------------------------------
Reporter: Yvan | Type: defect
Status: new | Priority: high
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: major
Keywords: ban lurker |
------------------------+---------------------------------------------------

Comment(by scoof):

Proposed fix attached, awaiting somebody to bless this before committing

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
------------------------+---------------------------------------------------
Reporter: Yvan | Type: defect
Status: new | Priority: high
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: major
Keywords: ban lurker |
------------------------+---------------------------------------------------

Comment(by kristian):

Hmm, I see where this went wrong. There is no indication in
ban_check_object() whether the check was OK or not, only if an object was
banned. We blindly assume that return 0 means sleep 1.0s.

In #1030 we corrected the basic issue, but didn't take into account what
happens when the ban list is perfectly ban-lurker-friendly but objects
simply can't be banned.

Scoof: I'm not too happy with the solution in your patch because it
negates large parts of the fix in #1030.

Phk: Seems like we need three return values from ban_check_object(). I
suppose it's dirty to "hide" that in BAN_CheckObject (e.g: ret =
ban_check_object(..); if (ret == 0 || ret == 1) return 0; else return 1;
)? The third return value would be "nothing banned, but bans updated",
which shouldn't be relevant outside of cache_ban.c.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
------------------------+---------------------------------------------------
Reporter: Yvan | Type: defect
Status: new | Priority: high
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: major
Keywords: ban lurker |
------------------------+---------------------------------------------------

Comment(by kristian):

Let me try that again:

Phk: Seems like we need three return values from ban_check_object(). I
suppose it's dirty to "hide" that in BAN_CheckObject (e.g:
{{{
int
BAN_CheckObject(...)
{
ret = ban_check_object(..);
if (ret == 0 || ret == 1)
return 0;
else
return 1;
}
}}}

The third return value would be "nothing banned, but bans updated", which
shouldn't be relevant outside of cache_ban.c.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
---------------------+------------------------------------------------------
Reporter: Yvan | Type: defect
Status: closed | Priority: high
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: major
Resolution: fixed | Keywords: ban lurker
---------------------+------------------------------------------------------
Changes (by phk):

* status: new => closed
* resolution: => fixed


Comment:

I just spent some quality time with the ban-lurker, it should work better
now.

Please report if not.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
---------------------+------------------------------------------------------
Reporter: Yvan | Type: defect
Status: closed | Priority: high
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: major
Resolution: fixed | Keywords: ban lurker
---------------------+------------------------------------------------------

Comment(by Yvan):

Shouldn't this bug, fixed 7 months ago, need a new release of varnish? The
ban list just keep increasing in 3.0.2, can I assume it's no problem and
go with it anyway? I've just set an alert trigger on list > 20k+ bans,
hence the alert never goes off. I reverted back to 3.0.1, but after new
servers installed, automatically installed 3.0.2.

Thanks for fixing this by the way :-)

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
------------------------+---------------------
Reporter: Yvan | Owner:
Type: defect | Status: closed
Priority: high | Milestone:
Component: varnishd | Version: 3.0.2
Severity: major | Resolution: fixed
Keywords: ban lurker |
------------------------+---------------------

Comment (by tgigli):

Guys,
I am using 3.0.2 version and have the same problem. When some ban

{{{
root@trestles:/etc/varnish# varnishd -V
varnishd (varnish-3.0.2 revision cbf1284)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2011 Varnish Software AS
}}}

It keep growing. Sometimes it seems to work cleaning 1 or 2 lines, but it
takes a lot of time and the speed that it is cleaned is really slow.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:6>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
------------------------+---------------------
Reporter: Yvan | Owner:
Type: defect | Status: closed
Priority: high | Milestone:
Component: varnishd | Version: 3.0.2
Severity: major | Resolution: fixed
Keywords: ban lurker |
------------------------+---------------------

Comment (by Yvan):

@tgigli: you should use the 3.0.3 release, which has the bugfix. In my
comment posted 5 months ago, the 3.0.3 wasn't released yet.

So use the 3.0.3, with the standard ban lurker parameters, that should
work like a charm.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:7>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1045: Ban lurker doesn't work anymore [ In reply to ]
#1045: Ban lurker doesn't work anymore
------------------------+---------------------
Reporter: Yvan | Owner:
Type: defect | Status: closed
Priority: high | Milestone:
Component: varnishd | Version: 3.0.2
Severity: major | Resolution: fixed
Keywords: ban lurker |
------------------------+---------------------

Comment (by keretamalam):

I am about to leave and cannot look into this further right now, but I do
want to mention that relying on pkg-config itself is not a problem (as you
said, we also use it later), the problem is that on some systems, the .pc
file for pkg-config is supplying wrong information (whereas on yours, it's
botan-config that is wrong), so that's why I'm trying to make it work with
both (while one of them is bad).
[http://maskodoq.blogspot.com/2014/03/unit-link-terbaik-di-indonesia.html
Unit Link Terbaik di Indonesia Commonwealth Life Investra Link]
[http://maskodoq.blogspot.com/2013/07/CiptoJunaedy.html Cipto Junaedy]
[http://etnisjawa.blogspot.com/2013/08/apakah-cipto-junaedy-bohong.html
Cipto Junaedy]

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1045#comment:8>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs