Bandwith optimization: squid, apt-cacher and jesred


At work, I now have around 50 desktops running Ubuntu and around 40 servers (including customers machines) also running Ubuntu. As you can imagine, when you have a security update of X, this represents a lot of bandwith usage! Not to speak about Hardy upgrade! We started to look at different solutions to optimize our precious bandwith.

Some search gave:

  • local mirror: huch… this is a bit too much for us :)
  • squid usage: good, but you need to tweak too much your squid installation to keep your .deb inside the pool. And squid can make .deb expires even they are still valid.
  • apt-proxy/apt-cacher/apt-cacher-ng: all looks good but… you have to modify your client configuration. As I am lazy, I don’t want to do that (and also, because I have mobile users who only want to use the cache when they are on the corporate network). Between the three, I chose apt-cacher, just based on some reading on the web… Other may be as good as apt-cacher!

We selected the association: squid + apt-cacher + jesred. Let’s have a look on each component:

  • apt-cacher: .deb and Packages/Source cache. You can also import data from another source (for exemple from a cd-rom).
  • squid: THE proxy, we use it as a transparent proxy in our case.
  • jesred: rewrite squid URL and redirect access to the Ubuntu archive to apt-cacher.

The installation described below was made on a Ubuntu 8.04. The machine is a Xen virtual machine (I’ll talk about Xen another time ;-)). All the softwares are taken from Ubuntu repositories: squid beeing in main, other packages are in universe (ensure universe is enabled). Installation and configuration is really easy!

squid installation

# apt-get install squid

squid configuration

Edit /etc/squid.conf and add in ACL definititions:
acl mylan src 10.0.0.0/255.255.0.0

Allow traffic from you network:
http_access allow mylan

You can now test your squid. It should be operational.

apt-cacher installation

# apt-get install apt-cacher

I just changed the admin_email value in /etc/apt-cacher/apt-cacher.conf

As a quick test, set http_proxy env value and try to use apt. Everything should go throught the cache (check the logs).

jesred installation

# apt-get install jesred

jesred configuration

Edit /etc/jesred.acl to authorize your network (just add you lan at the end of the file).

Edit /etc/jesred.rules and add:
regex ^http://((.*)archive.ubuntu.com/ubuntu/(dists|pool)/.*)$    http://localhost:3142/\1
regex ^http://(security.ubuntu.com/ubuntu/(dists|pool)/.*)$    http://localhost:3142/\1

I have also added two aborts in order to use upgrade-manager:
abort .gpg
abort ReleaseAnnouncement

Last but not least, the glue between all the elements:

Edit /etc/squid.conf and add:
redirect_program /usr/lib/squid/jesred

Finished ! Now your squid redirect all requests to *archive.ubuntu.com and security.ubuntu.com to apt-cacher. Happy installation / upgrades!

Information and Links

Join the fray by commenting, tracking what others have to say, or linking to it from your blog.


Other Posts
Better jesred rules
Mandatory Ubuntu 8.04 LTS release post

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

Be the first to leave a comment!