Mailing List Archive

Wish
The virus definition database update downloads a big file (ok, it´s no so big) to update a file that is almost the same.
I think it will be good to have a non compressed virus definition database accesible via rsync protocol.

--
Alfrenovsky
Re: Wish [ In reply to ]
On Friday 05 September 2003 00:04, Alfredo Rezinovsky wrote:
> The virus definition database update downloads a big file (ok, it´s no so
> big) to update a file that is almost the same. I think it will be good to
> have a non compressed virus definition database accesible via rsync
> protocol.

Just FYI (and I have no idea how good or whether at all it works): gzip
supports a an output format that should work well with rsync.

greets
-- vbi

--
People seem to enjoy things more when they know a lot of other people
have been left out on the pleasure.
-- Russell Baker
Re: Wish [ In reply to ]
On Friday 05 September 2003 10:34 am, Adrian von Bidder wrote:

> On Friday 05 September 2003 00:04, Alfredo Rezinovsky wrote:
> > The virus definition database update downloads a big file (ok, it´s no so
> > big) to update a file that is almost the same. I think it will be good to
> > have a non compressed virus definition database accesible via rsync
> > protocol.
>
> Just FYI (and I have no idea how good or whether at all it works): gzip
> supports a an output format that should work well with rsync.

rsync contains its own compression algorithm which does an excellent job on
its own.

Antony.

--

I love deadlines. I love the whooshing noise they make as they go by.

- Douglas Noel Adams
Re: Wish [ In reply to ]
El vie, 05-09-2003 a las 06:49, Antony Stone escribió:
> On Friday 05 September 2003 10:34 am, Adrian von Bidder wrote:
>
> > On Friday 05 September 2003 00:04, Alfredo Rezinovsky wrote:
> > > The virus definition database update downloads a big file (ok, it´s no so
> > > big) to update a file that is almost the same. I think it will be good to
> > > have a non compressed virus definition database accesible via rsync
> > > protocol.
> >
> > Just FYI (and I have no idea how good or whether at all it works): gzip
> > supports a an output format that should work well with rsync.
>
> rsync contains its own compression algorithm which does an excellent job on
> its own.
>

rsync transfers only the differences between two files compressing them
(optional)

(file2-file2).gz will be small

If the database are already compressed there are very low probabilities
on data coincidence, thus the differences will be almost the whole file.

file2.gz-file1.gz will be BIG

If you look at the kernel archive, for example, patch you will allways
see a compressed diff file, not a diff of the compressed files.

The only need is to have clamav to support non compressed virus
definition databse. And avaiability of rsync server with those files.

--
Alfrenovsky
Re: Wish [ In reply to ]
On Friday 05 September 2003 11:49, Antony Stone wrote:
> On Friday 05 September 2003 10:34 am, Adrian von Bidder wrote:
> > On Friday 05 September 2003 00:04, Alfredo Rezinovsky wrote:
> > > The virus definition database update downloads a big file (ok, it´s no
> > > so big) to update a file that is almost the same. I think it will be
> > > good to have a non compressed virus definition database accesible via
> > > rsync protocol.
> >
> > Just FYI (and I have no idea how good or whether at all it works): gzip
> > supports a an output format that should work well with rsync.
>
> rsync contains its own compression algorithm which does an excellent job on
> its own.

I just thought that having the same files available for HTTP-download and
rsync, it would be advantageous if they were compressed. (I don't know how
widespread support for x-gzip encoding in HTTP is - ignoring for now the
additional CPU load on the server).

greets
-- vbi

--
"It takes all sorts of in & out-door schooling to get adapted
to my kind of fooling"
- R. Frost
Re: Wish [ In reply to ]
Sorry. I didn´t saw that the db files are avaiable in both compressed and
non compresses versions. :-)
The only thing needed is to set up rsync servers.

Test this. (I have not a lot of bandwith, but the idea is that lot of
bandwidth is not needed with rsync).

----- cut here -----
$clamavdir=/usr/local/share/clamav
rsync -vz rsync://alfredo.rezinovsky.com.ar/clamav/* $clamavdir
----- cut here -----

Look at the "speedup" value, no matter what is the bandwith, this speedup
shows the ratio of the file divided by the real ammount of data
transfered.

When you are trying to sync the same files. you´ll need to write 65 bytes
and read 150 to get noticed that you have the latest, witch is almost the
same than downloading the md5.

Enabling rsync servers is VERY EASY, and can save bandwith on servers and
clients (very important with dial-up clients)

--
Alfrenovsky
Re: Wish [ In reply to ]
On Fri, 5 Sep 2003 12:15:57 +0200
Adrian von Bidder <avbidder@fortytwo.ch> wrote:

> (I don't know how
> widespread support for x-gzip encoding in HTTP is - ignoring for now the
>
> additional CPU load on the server).

I run mod_gzip on seveal heavily loaded sites and it prooved to be very
usefull ... minor cpu load increase vs. greatly decreased traffic.
Browsers nowdays support gzip well, there are some minor bugs in IE
related to refreshing in case of gzipped content ... Which should not
matter at all in case of clamav :)

--

Jure Pecar
Re: Wish [ In reply to ]
> see a compressed diff file, not a diff of the compressed files.
>
> The only need is to have clamav to support non compressed virus
> definition databse. And avaiability of rsync server with those files.

A new database format will be using by default the gzip compression (available
in libz).

Best regards,
Tomasz Kojm
--
oo ..... zolw@konarski.edu.pl
(\/)\......... http://www.konarski.edu.pl/~zolw
\..........._ I nie zapomnij kliknac w brzuszek...
//\ /\\ <- C. Amboinensis www.pajacyk.pl