Mailing List Archive

Compression in Varnish
Hi, everyone

When looking through varnish code, I noticed, that Varnish uses it's own
libvgz to compress and uncompress data. As far as I understand, this libvgz
is actually a zlib. The question is, why zlib isn't used directly?
Also, I use zlib in a vmodule, and when this vmodule is loaded, functions,
which are really called, are ones from libvgz. After noticing this, I
switched zlib to libvgz, but inflate function reports Z_DATA_ERROR on
gzipped data. Other tools, which do gunzipping, successfully unzip this
data. Do I need to do something special to use libvgz functions, or it's
same, as zlib? I mean something like pass set up extra variables, or do
extra initialization. In case I do, what?

Thaks in advance, Stas
Re: Compression in Varnish [ In reply to ]
Hi Stas,

we probably need phk's attention for the second part of the question, but here's
at least a first attempt.

> The question is, why zlib isn't used directly?

Because some subtle changes to zlib were needed to allow for an efficient
integration of (un)compression into Varnish.

> Do I need to do something special to use libvgz functions

-> phk

At any rate, you should not assume the libvgz interface to be stable.

Nils

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: Compression in Varnish [ In reply to ]
In message <CAKJRfiKHxAoo0Wm+C4Kt0K6XEkzgCiwgPwogkiOTskAQduR5uQ@mail.gmail.com>
, Stas Semeniuta writes:

>After noticing this, I
>switched zlib to libvgz, but inflate function reports Z_DATA_ERROR on
>gzipped data.

Can you send me an example of the data which fails ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: Compression in Varnish [ In reply to ]
>Can you send me an example of the data which fails ?

The problem was in my code, libvgz decompresses this data correctly. Sorry
for hasty conclusions about libvgz not working.

>Because some subtle changes to zlib were needed to allow for an efficient
>integration of (un)compression into Varnish.

So in case you use zlib directly, varnish performance will degrade? If it's
so, how much will it get worse?

Thanks in advance, Stas