Archive for April, 2008

Bandwith optimization: squid, apt-cacher and jesred

At work, I now have around 50 desktops running Ubuntu and around 40 servers (including customers machines) also running Ubuntu. As you can imagine, when you have a security update of X, this represents a lot of bandwith usage! Not to speak about Hardy upgrade! We started to look at different solutions to optimize our precious bandwith.

Some search gave:

  • local mirror: huch… this is a bit too much for us :)
  • squid usage: good, but you need to tweak too much your squid installation to keep your .deb inside the pool. And squid can make .deb expires even they are still valid.
  • apt-proxy/apt-cacher/apt-cacher-ng: all looks good but… you have to modify your client configuration. As I am lazy, I don’t want to do that (and also, because I have mobile users who only want to use the cache when they are on the corporate network). Between the three, I chose apt-cacher, just based on some reading on the web… Other may be as good as apt-cacher!

We selected the association: squid + apt-cacher + jesred. Let’s have a look on each component:

  • apt-cacher: .deb and Packages/Source cache. You can also import data from another source (for exemple from a cd-rom).
  • squid: THE proxy, we use it as a transparent proxy in our case.
  • jesred: rewrite squid URL and redirect access to the Ubuntu archive to apt-cacher.

The installation described below was made on a Ubuntu 8.04. The machine is a Xen virtual machine (I’ll talk about Xen another time ;-) ). All the softwares are taken from Ubuntu repositories: squid beeing in main, other packages are in universe (ensure universe is enabled). Installation and configuration is really easy!

squid installation

# apt-get install squid

squid configuration

Edit /etc/squid.conf and add in ACL definititions:
acl mylan src 10.0.0.0/255.255.0.0

Allow traffic from you network:
http_access allow mylan

You can now test your squid. It should be operational.

apt-cacher installation

# apt-get install apt-cacher

I just changed the admin_email value in /etc/apt-cacher/apt-cacher.conf

As a quick test, set http_proxy env value and try to use apt. Everything should go throught the cache (check the logs).

jesred installation

# apt-get install jesred

jesred configuration

Edit /etc/jesred.acl to authorize your network (just add you lan at the end of the file).

Edit /etc/jesred.rules and add:
regex ^http://((.*)archive.ubuntu.com/ubuntu/(dists|pool)/.*)$    http://localhost:3142/\1
regex ^http://(security.ubuntu.com/ubuntu/(dists|pool)/.*)$    http://localhost:3142/\1

I have also added two aborts in order to use upgrade-manager:
abort .gpg
abort ReleaseAnnouncement

Last but not least, the glue between all the elements:

Edit /etc/squid.conf and add:
redirect_program /usr/lib/squid/jesred

Finished ! Now your squid redirect all requests to *archive.ubuntu.com and security.ubuntu.com to apt-cacher. Happy installation / upgrades!

, ,

5 Comments

Mandatory Ubuntu 8.04 LTS release post

I guess everybody has already read it, but Ubuntu has released a new release : 8.04 (8 for 2008, 4 for april). Note that this release is a LTS release (Ubuntu and Ubuntu Server only, Kubuntu and orther derivates are not ong term surpport release). As a result, you can upgrade from Ubuntu 7.10 (aka Gutsy) and 6.06.2 (aka Dapper).

I run 8.04 on my laptop and on my personal servers for several months now, and it run quite well. This bug on Ubuntu Kernel made my life at work a bit more difficult, but it should hopefully be fixed in 8.04.1 (due july 3rd).

Now, it’s time to be intrepid!

4 Comments