A Blog

Using a web caching proxy can help save bandwidth and provide a good log of web traffic. At the house, I have a slow-ish DSL link to the world and would certainly love to have video not freeze when also browsing the web. And as a friend points out, sometimes it’s easier to have a log of traffic when digging into problems. Although, my one biggest pet peeve is having to manually reconfigure my laptop when moving between campus and home. OSX does have the concept of “network locations,” but usually half the time I forget to change it. So, the only solution to this problem is a transparent web-caching proxy server. Thankfully, I run a server in the basement for fun and excitement anyway.
First things first, one needs to set up the “Web Cache Control Protocol” on the router. At home, I use a Cisco 871, so I issued these changes:

config terminal

ip wccp web-cache
interface vlan1000
  ip wccp web-cache redirect in

exit

This sets up WCCP and applies it to the virtual network my wireless network uses. Next, we need to set up the Ubuntu box with Squid, thankfully that’s pretty easy. “apt-get install squid” is good enough to get the package. Then we need to ensure these lines are in squid.conf, the rest of the default options are probably good enough for a first go:

wccp2_router IP_OF_ROUTER
wccp_version 4
wccp2_forwarding_method 1
wccp2_return_method 1
wccp2_service standard 0
http_port 3128 transparent
acl our_networks src WORKSTATION_SUBNET/24
http_access allow our_networks

So, at this point, the router is using WCCP and our server has Squid going. Now, we just need some magic to tie the two together. This is done using a magical combination of iptables and a GRE tunnel. First things first, we need to load the “ip_gre” kernel module on boot: “echo ip_gre >> /etc/modules” should do the trick. For now, “modprobe ip_gre” is good.
Then we need to construct the tunnel by issuing the following commands:

/sbin/ip link set wccp1 mtu 1476
/sbin/ip tunnel add wccp1 mode gre remote ROUTER_IP local MACHINE_IP dev ETHERNET_DEVICE
/sbin/ip addr add MACHINE_IP dev wccp1
/sbin/ip link set wccp1 up
/sbin/sysctl -w net.ipv4.conf.wccp1.rp_filter=0
/sbin/sysctl -w net.ipv4.conf.ETHERNET_DEVICE.rp_filter=0
/sbin/iptables-restore < /etc/default/iptables

This constructs the tunnel and brings it up. At the house, the server has a separate connection on a separate virtual network from everything else. WCCP should handle the situation where your web cache is in the subnet whose traffic is getting redirected, but I wanted to ensure there weren't going to be any difficulties.
Now, the final piece of the puzzle are the iptables rules that search for traffic coming down the tunnel and redirects them to Squid:

# /etc/default/iptables
# Allow in everything, from everywhere
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
# Reroute HTTP requests to the proxy server
-A PREROUTING -i wccp1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -i wccp1 -p tcp -m tcp --dport 8000 -j REDIRECT --to-ports 3128
-A PREROUTING -i wccp1 -p tcp -m tcp --dport 8080 -j REDIRECT --to-ports 3128
COMMIT

Once all of this is in place and running, you should be able to see web traffic get cached and logged into /var/log/squid3/access.log. From my logs after several days, I'm seeing the cache serve about 4% of my traffic. Given the amount of dynamic content on the Internet today, that's not very surprising. What's it worth all this effort to get going? Probably not, but it was sure fun.
I need to point out that most of the content of this post can be found on the Squid site and wiki, and that community should receive any positive credit for their fine work.