[Gllug] Apt and RPM caching experiences

[Richard W.M. Jones]

On Mon, Jul 04, 2011 at 05:20:16AM +0100, Stephen Nelson-Smith wrote:
> Hello,
> 
> Inspired by an interesting article I read in which someone wrote a
> very basic RPM cacher in Perl, I've revisited the way we do this
> ourselves.  For Debian/Ubuntu we use apt-cacher-ng, and for CentOS we
> maintain a local mirror.  Looking at it again for a few new clients,
> I'm leaning towards simply having a Squid/mod_cache setup that caters
> for both Debian/Ubuntu and CentOS/Redhat, rather than the (rather
> flakey) apt-cacher(-ng) that needs to be restarted once a month, and
> the rather primitive mirroring of the entire repo.
> 
> Has anyone got any experiences or ideas to contribute?

Yes, quite a lot actually.

We use squid to cache.  With the obvious proviso that this is a cache,
not a mirror, so it is only effective after the first time someone has
run yum, there are also some alterations you need to make to both the
squid and yum config in order to do this effectively.

This covers the changes to squid config:

https://fedoraproject.org/wiki/Extras/MockTricks#Using_Squid_to_Speed_Up_Mock_package_downloads

Make the maximum object size and the cache size as big as you can.
The suggested changes in the URL may not be big enough.

Where it says you need to change the mock config to not use mirror
lists, ignore that and change the yum config instead.  Edit
/etc/yum.repos.d/*.repo and comment out mirrorlist and uncomment (and
adjust) baseurl.  The reason for this is yum's stupid mirror rotation.
It will try a different mirror all the time, which means your cache
gets filled with multiple identical copies of files from different
mirrors.

Don't use mirrorservice.org as a baseurl since they deliberately break
caching of RPM files.  (Is anyone from mirrorservice.org on this list?
Please stop doing this!)  Alternatively you can add:

  refresh_pattern -i mirrorservice.org 1440 20% 10080 ignore-no-store

Also if you use the bytemark UK mirror and your squid proxy has any
IPv6 addresses and you don't have any IPv6 path to the internet, then
squid will try to use bytemark's AAAA record and fail.  So you would
need to reconfigure squid to disable all IPv6 in this case.

Rich.

-- 
Richard Jones
Red Hat
--
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug