Squid proxy – caching windows updates

Recently I’ve been playing with the squid caching proxy, and specifically it’s ability to cache windows updates.  At the shop, we only get 2 MB down (on a good day) so every bit of upstream bandwidth we can save it critical.  We’ve found that Windows updates in particular represent a large amount of the data we download each day, and the same files are downloaded over and over and over again–prime candidate for caching.

Microsoft has their WSUS technology, but this would require us to set up a Windows Server computer (very expensive licensing) and modify the registry of every computer that should take advantage of it–then we have to remember to change the registry back.  This just isn’t a good solution for us.  It’s great if you have a domain, but we can’t join each of our customer’s computers to a domain.

The squid website doesn’t say a whole lot about windows updates, and most of the reading I’ve done online seems to be skeptical of it’s performance; however, I’ve had plenty of good service out of it.  I think you just have to be willing to play with it.

Here’s part of a calamris report from our squid server:

Request-destinations by 2nd-level-domain
destination request % hit-% Byte % hit-%
*.microsoft.com 1642 29.48 20.52 7360777 1.62 10.73
*.windowsupdate.com 1005 18.05 62.49 298927K 67.27 81.13
*.google.com 435 7.81 16.09 4416384 0.97 4.01
*.trueweather.com 420 7.54 0.00 324354 0.07 0.00
*.yimg.com 226 4.06 71.24 1662990 0.37 47.46
*.avast.com 143 2.57 1.40 87107611 19.14 0.00
*.avg.com 128 2.30 34.38 2354658 0.52 10.74
*.washingtonpost.com 100 1.80 0.00 486564 0.11 0.00
*.weather.com 96 1.72 31.25 785460 0.17 31.83
*.asipartner.com 76 1.36 48.68 647773 0.14 47.91
*.cachefly.net 69 1.24 94.20 366680 0.08 96.66
*.encyclopedia-titanica.org 68 1.22 41.18 271594 0.06 3.58
<error> 68 1.22 1.47 57821 0.01 1.14
67.228.147.* 61 1.10 0.00 287265 0.06 0.00
*.yahoo.com 54 0.97 3.70 361335 0.08 0.50
*.sun.com 50 0.90 0.00 4354859 0.96 0.00
*.killertechtips.com 37 0.66 0.00 254334 0.06 0.00
*.dominos.com 35 0.63 2.86 729177 0.16 2.67
*.qarchive.org 32 0.57 0.00 286142 0.06 0.00
*.thesitestation.com 32 0.57 0.00 193273 0.04 0.00
other: 147 2nd-level-domains 792 14.22 18.06 36612812 8.05 2.03
Sum 5569 100.00 27.81 444358K 100.00 55.39

Note the hit percentage on windowsupdate.com – 81.13%!  That’s where the majority of the updates are coming from.  It would be even higher than that, but the other day we had a few computers that needed some of the older (large!) updates that hadn’t been cached.

The way we have it set up is as follows:

squid_diagram

Our local workstations connect to the squid proxy on it’s 192.168.2.1 interface, where the iptables firewall reroutes all requests on port 80 to the squid port (3128).  Then, squid either retrieves the requested data from the cache, or directs the request to the originally intended internet site through it’s 208.xxx.xxx.xxx interface.

For any who are interested, here is our squid.conf.  The only thing I’m really not sure about it is the persistent connection settings.  I was trying to fix a problem we had with connections being sustained with content delivery networks past when the clients had closed the connection, but I’m not sure if these directives are related to that.  I think that was actually caused by “range_offset_limit -1”, which we need in order to make sure the entire file is downloaded if a windows update is requested.

http_port 192.168.2.1:3128 transparent
range_offset_limit -1
acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src 192.168.1.0/24 # RFC1918 possible internal network
acl localnet src 192.168.2.0/24
acl SSL_ports port 443          # https
acl SSL_ports port 563          # snews
acl SSL_ports port 873          # rsync
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 631         # cups
acl Safe_ports port 873         # rsync
acl Safe_ports port 901         # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow localhost
http_access deny all
icp_access allow localnet
icp_access deny all
http_port 192.168.2.1:3128
hierarchy_stoplist cgi-bin ?
 cache_dir diskd /var/spool/squid 15000 16 256
maximum_object_size 819200 KB
access_log /var/log/squid/access.log squid
refresh_pattern windowsupdate.com/.*.(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims override-expire
refresh_pattern download.microsoft.com/.*.(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims override-expire
refresh_pattern www.microsoft.com/.*.(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims override-expire
refresh_pattern au.download.windowsupdate.com/.*.(cab|exe|dll|msi) 4320 100% 43200 reload-into-ims override-expire
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|?) 0     0%      0
refresh_pattern (Release|Package(.gz)*)$        0       20%     2880
refresh_pattern .               0       20%     4320
acl shoutcast rep_header X-HTTP09-First-Line ^ICYs[0-9]
upgrade_http0.9 deny shoutcast
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
extension_methods REPORT MERGE MKACTIVITY CHECKOUT
 client_persistent_connections off
 server_persistent_connections off
hosts_file /etc/hosts
forwarded_for off
coredump_dir /var/spool/squid
Request-destinations by 2nd-level-domain
destination request % hit-% Byte % hit-%
*.microsoft.com 1642 29.48 20.52 7360777 1.62 10.73
*.windowsupdate.com 1005 18.05 62.49 298927K 67.27 81.13
*.google.com 435 7.81 16.09 4416384 0.97 4.01
*.trueweather.com 420 7.54 0.00 324354 0.07 0.00
*.yimg.com 226 4.06 71.24 1662990 0.37 47.46
*.avast.com 143 2.57 1.40 87107611 19.14 0.00
*.avg.com 128 2.30 34.38 2354658 0.52 10.74
*.washingtonpost.com 100 1.80 0.00 486564 0.11 0.00
*.weather.com 96 1.72 31.25 785460 0.17 31.83
*.asipartner.com 76 1.36 48.68 647773 0.14 47.91
*.cachefly.net 69 1.24 94.20 366680 0.08 96.66
*.encyclopedia-titanica.org 68 1.22 41.18 271594 0.06 3.58
<error> 68 1.22 1.47 57821 0.01 1.14
67.228.147.* 61 1.10 0.00 287265 0.06 0.00
*.yahoo.com 54 0.97 3.70 361335 0.08 0.50
*.sun.com 50 0.90 0.00 4354859 0.96 0.00
*.killertechtips.com 37 0.66 0.00 254334 0.06 0.00
*.dominos.com 35 0.63 2.86 729177 0.16 2.67
*.qarchive.org 32 0.57 0.00 286142 0.06 0.00
*.thesitestation.com 32 0.57 0.00 193273 0.04 0.00
other: 147 2nd-level-domains 792 14.22 18.06 36612812 8.05 2.03
Sum 5569 100.00 27.81 444358K 100.00 55.39
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: