A Blog

Ever wondered exactly where all your network cabling goes? Have you been using Cisco and wished your computers spoke CDP too? Apparently you and everyone else would love for the computers to just say where they connected instead of chasing down network cables by hand. That seems to be the goal of the Link Layer Discovery Protocol (LLDP or 802.11ab). Unlike the Cisco Discovery Protocol (CDP), LLDP is the vender-neutral attempt to get it all right.

There is a LLDP daemon that is published at Luffy.cx that implements this under Linux. There are other daemons out there too, but this showed up first when I searched the Debian repositories for precompiled versions. Simply installing it via apt and starting up the service is enough to get your computers and your network devices discovering themselves. Although, if you’re like me and don’t have fancy new networking gear that supports LLDP, lldpd supports a wide range of other network discovery protocols too.

Once I installed lldpd on all my computers and enabled the CDP option (the -c option when starting up lldpd), I saw the magic happen:

c2950-01#show cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
ospf-01
                 Gig 0/1            107           R       Linux     eth1
home-1
                 Gig 0/2            114                   Linux     eth0
storage
                 Fas 0/15           106                   Linux     eth0
c831
                 Fas 0/8            146           R       C831      Eth 2
c871
                 Fas 0/1            142         R S I     871       Fas 0

To query the neighbors discovered by lldpd on the computer side, lldpctl outputs all the current neighbors:

Interface: eth1
 ChassisID: c2950-01 (local)
 SysName:   c2950-01
 SysDescr:
   cisco WS-C2950G-48-EI running on
   Cisco Internetwork Operating System Software
   IOS (tm) C2950 Software (C2950-I6K2L2Q4-M), Version 12.1(22)EA8a, RELEASE SOFTWARE (fc1)
   Copyright (c) 1986-2006 by cisco Systems, Inc.
   Compiled Fri 28-Jul-06 17:00 by weiliu
 MgmtIP:    172.0.0.0
 Caps:      Bridge(E) 

 PortID:    GigabitEthernet0/1 (ifName)
 PortDescr: GigabitEthernet0/1

To finish up the post, I’ll note that discovery protocols have in the past, and potentially still now, have been susceptible to attacks by flooding devices with too many neighbor relations. Because of this, it might be best to ensure these protocols are disabled on switches ports connected to untrusted machines.

Tinkergeek recently moved to its own dedicated server hosted by fdc servers. The transition wasn’t quite as smooth as one would hope, but they do provide cheap hosting. With moving to a dedicated server based around cheap PC hardware, I thought it’d be a great idea to bring back an old backup solution. Dirvish is a neat set of scripts that combine the goodness of hard-links and rsync. The goal is that Dirvish creates a full backup once and then stores just the changes of the target file system on the backup system.

Installing dirvish on a Debian system is fairly easy:

apt-get install dirvish

Then, one merely has to copy the example configuration files into place. The first one is the master configuration file that goes in /etc/dirvish and can be found in /usr/share/doc/dirvish/master.conf.

## Example dirvish master configuration file:
bank:
	/backup
exclude:
	lost+found/
	core
	*~
	.nfs*
Runall:
	root	22:00
expire-default: +15 days
expire-rule:
#       MIN HR    DOM MON       DOW  STRFTIME_FMT
	*   *     *   *         1    +3 months
#	*   *     1-7 *         1    +1 year
#	*   *     1-7 1,4,7,10  1
	*   10-20 *   *         *    +4 days
#	*   *     *   *         2-7  +15 days

Under bank: is going to be the place on your machine that contains all the backups. I’d suggest making this its own file system, as dirvish can eat inodes like there’s no tomorrow. Next, I’d suggest adding /proc and /sys under the global exclude: section just to ensure you don’t back these directories up.

Now, you have to make your first machine directory for backups (known as a vault in dirvish speak). This directory structure will be under the directory in the bank section from above.

mkdir -p /backup/example.com/dirvish

Now, just copy the default.conf example from /usr/share/doc/dirvish/examples into example.com/dirvish and edit.

client: thishost
tree: /
xdev: 1
index: gzip
log: gzip
image-default: %Y%m%d
exclude:
	/var/cache/apt/archives/*.deb
	/var/cache/man/**
	/tmp/**
	/var/tmp/**
        *.bak

The key points in this file file are the client, xdev, and exclude directives. Merely change your client: to be the machine IP or hostname that you’re backing up. Xdev tells rsync to traverse file systems on the machine; generally, you’ll want to be careful with this setting, specially if you mount NFS shares. Lastly, update the exclude list for this particular machine. If you’re backing up the local machine using dirvish, be sure to exclude the dirvish bank directory!

Now, you’re all set to create the first backup with:

dirvish --vault example.com --init

If all goes well, you’ll have a new directory under /backup/example.com with the current data and a copy of the target. If there was an error, be sure to remove the failed backup attempt from /backup/example.com and rerun the dirvish command after fixing the error.

Now, the only thing left is to run dirvish-runall via cron at some convenient time and you’re on your way to having a decent backup solution. Besure to read the remainder of the dirvish documentation to pick up the finer points of configuration.

So, I didn’t have much of a chance to update the blog with blow-by-blow action updates at SC this year.. Oops. However, the show was a blast. The Purdue team won the “lowest power consumption” award. It was certainly a surprise, given another team had a generally more efficient power design. In any case, we’re certainly happy we won an award.

I did not get much of a chance to walk around the floor between working both of Purdue’s booths. However, I did hear about and see one or two interesting things. The first was the new ethernet gear coming out of Voltaire. They can confederate several of their switch chassises to form a virtual switch that allows an LACP bond between the chassises to be active-active, which seems to be all the rage these days. After visiting the Voltaire booth, Cisco also has various solutions.. from the virtual port channel on the nexus gear and the virtual switch chassis on the catalyst gear. Also, it appears that perhaps Cisco will be implementing something like Woven’s multi-path layer 2 magic, so that could be fun too.

Lastly, the most interesting company on the floor that I saw was the “Two Guys and a Cluster” folks. Really, it’s just two guys that live on opposite coasts that help people acquire and run clusters. They seem to be a nice middle ground between a value-added cluster vendor and just buying hardware off a website and doing it all yourself. Their web address, with the start of an interesting blog, can be found here.

Using a web caching proxy can help save bandwidth and provide a good log of web traffic. At the house, I have a slow-ish DSL link to the world and would certainly love to have video not freeze when also browsing the web. And as a friend points out, sometimes it’s easier to have a log of traffic when digging into problems. Although, my one biggest pet peeve is having to manually reconfigure my laptop when moving between campus and home. OSX does have the concept of “network locations,” but usually half the time I forget to change it. So, the only solution to this problem is a transparent web-caching proxy server. Thankfully, I run a server in the basement for fun and excitement anyway.
First things first, one needs to set up the “Web Cache Control Protocol” on the router. At home, I use a Cisco 871, so I issued these changes:

config terminal

ip wccp web-cache
interface vlan1000
  ip wccp web-cache redirect in

exit

This sets up WCCP and applies it to the virtual network my wireless network uses. Next, we need to set up the Ubuntu box with Squid, thankfully that’s pretty easy. “apt-get install squid” is good enough to get the package. Then we need to ensure these lines are in squid.conf, the rest of the default options are probably good enough for a first go:

wccp2_router IP_OF_ROUTER
wccp_version 4
wccp2_forwarding_method 1
wccp2_return_method 1
wccp2_service standard 0
http_port 3128 transparent
acl our_networks src WORKSTATION_SUBNET/24
http_access allow our_networks

So, at this point, the router is using WCCP and our server has Squid going. Now, we just need some magic to tie the two together. This is done using a magical combination of iptables and a GRE tunnel. First things first, we need to load the “ip_gre” kernel module on boot: “echo ip_gre >> /etc/modules” should do the trick. For now, “modprobe ip_gre” is good.
Then we need to construct the tunnel by issuing the following commands:

/sbin/ip link set wccp1 mtu 1476
/sbin/ip tunnel add wccp1 mode gre remote ROUTER_IP local MACHINE_IP dev ETHERNET_DEVICE
/sbin/ip addr add MACHINE_IP dev wccp1
/sbin/ip link set wccp1 up
/sbin/sysctl -w net.ipv4.conf.wccp1.rp_filter=0
/sbin/sysctl -w net.ipv4.conf.ETHERNET_DEVICE.rp_filter=0
/sbin/iptables-restore < /etc/default/iptables

This constructs the tunnel and brings it up. At the house, the server has a separate connection on a separate virtual network from everything else. WCCP should handle the situation where your web cache is in the subnet whose traffic is getting redirected, but I wanted to ensure there weren't going to be any difficulties.
Now, the final piece of the puzzle are the iptables rules that search for traffic coming down the tunnel and redirects them to Squid:

# /etc/default/iptables
# Allow in everything, from everywhere
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
# Reroute HTTP requests to the proxy server
-A PREROUTING -i wccp1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -i wccp1 -p tcp -m tcp --dport 8000 -j REDIRECT --to-ports 3128
-A PREROUTING -i wccp1 -p tcp -m tcp --dport 8080 -j REDIRECT --to-ports 3128
COMMIT

Once all of this is in place and running, you should be able to see web traffic get cached and logged into /var/log/squid3/access.log. From my logs after several days, I'm seeing the cache serve about 4% of my traffic. Given the amount of dynamic content on the Internet today, that's not very surprising. What's it worth all this effort to get going? Probably not, but it was sure fun.
I need to point out that most of the content of this post can be found on the Squid site and wiki, and that community should receive any positive credit for their fine work.

I also just noticed that I restarted my blog one year ago this evening. Man has it become much more useful than any of my previous attempts!

What is cloud computing? To me, it is when hardware or software is run someplace “else” for you. At Purdue, it appears we have interest in doing this for researchers with computing and storage needs. But, I have been doing “computing in the cloud” for a long time before getting involved with Purdue’s effort..

Tinkergeek.com has existed in one form or another for quite some time. Usually, it is hosted in a virtual machine running in someone else’s data center. Over the years, one or both of the machines behind Tinkergeek have been hosted with Linode, Slicehost, or a now defunct Unixshell. I’ve greatly enjoyed having accounts with both Slicehost and Linode. For $20/month or the like, they have provided very solid performance and uptime. To save costs and not host all my data entirely in cloud, the secondary machine for Tinkergeek is hosted at home on my DSL modem.

The next service that can really be defined as cloud computing is my Google Apps hosted domain. It provides my Google Calendar, gMail, and Google Docs access. Both calendar and mail have been very stable since I moved into Google over a year ago and after going with the “Premium” service, I have not had a single hiccup. Of course, even though I pay Google for service and “technically” have a phone number to call if everything goes to hell, I also make a local copy of all my mail and calendar data..

For entertainment, who wants to set their schedule according to some network’s broadcast schedule? A lot of people I know use TiVo devices to get time-shift their television watching. Running a dvr off of over-the-air broadcast is a pain in the rear, so I just use Hulu. This is probably the only “free” cloud service I use, although it does make me watch commercials. No big deal, and it has been quite nice to be able to watch just the shows I care about whenever I want to view them.. Although, Big Media seems scared by Hulu and keeps playing games with the availability of show episodes, making this definitely not the most reliable service on the planet.

A while ago, I attempted to use Amazon’s S3 service for storage in the cloud. I never found a very convenient interface for pushing and retrieving files from S3 and eventually stopped using the service. Then, DropBox came out. It is backed by S3 for storage and provides an excellent interface that integrates very well into OS X. Plus, even though I am using the free-account, DropBox syncs all my data to the machines joined to my account. So, even if Amazon goes down, my data is still available. I like that automatic insurance.

So, there is how I do my computing in the cloud. Nothing very funky or very far out there. Webmail, shared web hosting, and file services (like Xdrive) have been around since before I even got my first computer; now its all just called “cloud computing.” Maybe its the pretty, dynamic Web2.0 interfaces on everything that makes computing deserve to be high in the clouds?

So, after some thinking, some prodding, and some money, I started to play with networking. My home network has for the longest time always had a Linux box at the front of it doing firewalling, nat, and all the other goodies one needs on a network. Ebay provided the cheap avenue to get some “real” networking gear, and after some frantic tabs and ?’s, I got a shiny new Cisco 871 configured up to do NAT and take a DHCP address.

Eventually, I wanted to get IPv6 networking back into my house. After dealing with SiXXS for the longest time, I moved to Hurricane Electric’s free tunnel service. I’ve never been happier with a free service. However, my home Internet connection is a Verizon DSL line with a dynamic IP address. HE just uses a point to point link for providing connectivity, so I choose to home my tunnel and IPv6 space to a machine at the Purdue Computer Society. Then, I set up a static tunnel to my house and routed some space to my 871. The goal here is to always have a constant connection for the world to me, even if that constant connection leads to a box and then dead-ends. Plus, it seems sort of silly to waste a whole /48 at my house when there could be all sorts of more useful places to send my IPv6 subnet space to.

This is where Quagga comes in. I do not really want to have to maintain static routes pointed everything, I just want the routers to know about eachother and figure out the hard parts. While I’m digging the Cisco stuff, I’m certainly not interested in buying another router, but Linux does that routing thing nicely. Quagga provides the routing protocols to populate the kernel’s routing table. It seems like a match made in heaven. (As a side note, recent Quagga builds are broken with respect to advertising IPv6 routes in BGP… Check before pulling your hair out too!)

So, for testing, I installed Quagga on two Linux boxes and gave them a simple configuration (ASN’s changed to protect the innocent):

Fremont:

router bgp 65220

 bgp router-id 128.46.156.55

 neighbor 2001:470:c180:aa01::2 remote-as 65221

 neighbor 2001:470:c180:aa01::2 next-hop-self

!

 address-family ipv6

 network 2001:470:1f11:6e5::/64

 network 2001:470:c180::/48

 aggregate-address 2001:470:c180::/48

 neighbor 2001:470:c180:aa01::2 activate

 exit-address-family

Saratoga:

router bgp 65221

 bgp router-id 128.46.156.11

 neighbor 2001:470:c180:aa01::1 remote-as 65220

 neighbor 2001:470:c180:aa01::1 next-hop-self

!

 address-family ipv6

 network 2001:470:1f11:53f::/64

 network 2001:470:c159::/48

 neighbor 2001:470:c180:aa01::1 activate

 exit-address-family

A friend noted that the “customer service” on SiXXS was getting to be quite a problem. It appears with popularity comes more trouble tickets and increased user support effort, something SiXXS appears unwilling to provide in a kind manner. Then, we started checking Hurricane Electric’s tunnel offerings..

HE does not have a “credits” system nor do they have strict requirements on things like user registration or tunnel performance. One merely requests a tunnel from one of their POPs and you get it automatically (along with another /64 so you can provide service to your LAN right off the bat). Then, once the tunnel is up and running, it is a simple click of a link to get a /48 allocation. No human intervention or long justifications required.

A big problem with tunnels is that latency can start to get out of hand. My native IPv4 connection gets under 80ms to most places on the Internet and usually it’s under 40ms to popular locations. To get anywhere interesting (Google and the like) over IPv6, it’s at least a 100ms journey.

Now after getting set up with a HE tunnel, I find my latency problems are a little less severe. In fact, I started running “traceroute” and noticed that most of the SiXXS stuff usually bounced around their POP for a bit and then jumped onto the HE network. After looking more into HE, it appears they have one of the largest IPv6 networks out there and they actually have customers using their network! SiXXS was a great starting point, but I wonder why people do not just go with the kinder, friendlier, and faster tunnel provider?

As a side note though, HE only provides static tunnels. No strange UDP encapsulated tunnels here to get past NAT or firewall issues.

After noticing that my WordPress install was yet again out of date, I went to go run Apt to see if there was anything pressing to be updated on tinkergeek.com. A surprise came when my repo no longer knew about my release of Ubuntu. I remember a while back choosing to go on the “bleeding edge” route of releases, but somewhere along the line I stopped upgrading.

According to the Ubuntu wiki page on releases, my server running Feisty Fawn was End of Life’d last year. Oops. So, some editing of /etc/apt/sources.lst and a couple “apt-get dist-upgrades” later, tinkergeek.com moved from Feisty to Gutsy and landed on the Hardy LTS release.

After a reboot to ensure everything would come back fine on its own, I’m glad the Debian folks made apt. Nothing like jumping major releases on a whim in the middle of the night!

In a move that may be quite unwise, I obtained a Sun Enterprise 6500 system and put it in my basement. I’m not exactly sure why I made such a decision, but the hardware has been interesting to play with.

The system was billed as the ultimate Enterprise system of the time, featuring massive expansion capabilities and maximum uptimes. The system was engineered to be hot swappable so one could repair or upgrade without trouble. Normally, the system came in a rack by itself with a little bit of room to add near line storage to it. Purdue used several of these systems in various departments, including the mail group where my machine came from. Although, my system does not sit in the large, purple-ish colored rack. Instead, it was pulled out and now sits on a shelf humming away.

The rack required a 30amp 208v plug, the system rated at taking upto 24amps of that power. On the advice of a coworker, the system is powered from a 15amp 120v circuit. Of course, most of the system is bare. My system is configured with the standard I/O board so I can boot from either a wide scsi disk or a CD-rom and four 400MHz CPU boards. The system contains 14GB of memory. The chassis can hold 16 boards, 15 of which can hold CPUs and memory. Making the system able to contain upto 30 487MHz UltraSparcII’s and 60GB of memory.

Thankfully, it doesn’t appear the computer is consuming too much power (as the lights don’t yet dim when I turn it on…)

I just got Debian Etch installed and serving up a 7GB ram disk over NFS to my iMac. Sadly, the system only comes with 100Mbps Ethernet, with only two expensive options for 1Gbps: a rare SBus adapter or the PCI I/O board. The system backplane is capable of 2.8GBps of traffic, so hopefully one day I can obtain the PCI board.

Since I’ve been playing with my employer’s SiCortex machine, I decided to run my PI estimation program against this beast. On a per-core basis, the code runs at about the same speed on both systems. Though, I’m guessing my system is using a few more watts than the equivalent 8 cores in a SiCortex…