Comments on: Bandwith optimization: squid, apt-cacher and jesred

By: www.porcheron.info

www.porcheron.info — Tue, 22 Mar 2011 11:41:23 +0000

Bandwith optimization squid apt cacher and jesred.. Keen

By: ehcache.net

ehcache.net — Wed, 23 Feb 2011 09:01:54 +0000

Bandwith optimization: squid, apt-cacher and jesred…

At work, I now have around 50 desktops running Ubuntu and around 40 servers (including customers machines) also running Ubuntu. As you can imagine, when you have a security update of X, this represents a lot of bandwith usage! Not to speak about Hardy …

By: Ike Mccaman

Ike Mccaman — Sun, 07 Nov 2010 19:24:31 +0000

Thanks so much for the article. Great.

By: Portablejim

Portablejim — Fri, 09 Jul 2010 07:32:37 +0000

Thanks Andras Korn. I have created a How-to based on the rules you gave on my website called “Using apt-cacher-ng to handle deb files instead of squid” (http://portablejim.site-hosts.net/tips/95-squidandaptcacherng.html)

By: Andras Korn

Andras Korn — Mon, 13 Jul 2009 09:57:56 +0000

The ulrpath_regex acl is really only intended for non-apt-clients, such as ordinary browsers, that browse repositories. You can of course remove the requirement for .debs to be under a pool/ directory to make the matching broader, for example.

APT and its ilk should be detected by the “aptget” browser acl and thus redirected to the apt proxy.

BTW, with the appropriate refresh_pattern (e.g. by matching Packages and .deb and giving them high expiry times) you may be able to dispense with a dedicated apt proxy entirely as squid will happily cache the package files itself.

By: lionel

lionel — Mon, 13 Jul 2009 08:39:04 +0000

Hi Andras,

Thanks for your comment. I’m not a squid expert, but the rules you have given does not make the job: you are catching all repositories (maybe we can check more than urlpath_regex). But looks promising.
For apt-cacher-ng, I’ll give a try. Lot of comments seems to indicate this is a good competitor. I have to admit that I did not had problem with apt-cacher on my Ubuntu hardy, and as it is used by the whole company, I haven’t touch it since.

By: Andras Korn

Andras Korn — Sun, 12 Jul 2009 22:38:59 +0000

You don’t need jesred.

Something like the following in squid.conf does the trick just fine (substitute the hostname of your apt cache for “aptcacher” everwhere):

cache_peer aptcacher parent 3142 7 proxy-only no-query no-netdb-exchange connect-timeout=15

cache_peer_access aptcacher allow aptget
cache_peer_access aptcacher allow deburl
cache_peer_access aptcacher deny all

never_direct allow aptget
never_direct allow deburl
never_direct deny all

The urlpath_regex may be too broad and/or too narrow and/or unnecessary, based on your exact needs.

FWIW, my experience with apt-cacher-ng has been a lot better than with apt-cacher (which used a lot of resources in addition to being slow and unstable).

By: Marc Tardif

Marc Tardif — Wed, 22 Oct 2008 18:12:10 +0000

It seems that one solution is the limit the number of redirector processes to spawn from squid with the following configuration:

redirect_children 1

By: Marc Tardif

Marc Tardif — Wed, 22 Oct 2008 15:53:34 +0000

After implementing the above configuration, I seem to be getting a lot of corruption errors:

Wed Oct 22 11:53:18 2008|127.0.0.1| ALARM! /var/cache/apt-cacher/packages/archive.ubuntu.com_ubuntu_dists_intrepid_universe_binary-i386_Packages.bz2 file size mismatch (found 212992, expected 4541620). Renaming to /var/cache/apt-cacher/packages/archive.ubuntu.com_ubuntu_dists_intrepid_universe_binary-i386_Packages.bz2.corrupted.