noticeboard.ru.ac.za

2007/02/01 - Central Services Outage
At approximately 4.30PM on Thursday 1 February there'll be a brief disruption of all centrally provided network services. We anticipate that the outage will last no longer than fifteen minutes. During this time users will be unable to do anything that requires access to a central server including, but not limited to, accessing the Internet, e-mail, and central Novell servers.

This outage is to allow us to implement the virtual router redundancy protocol (VRRP) between the routers in our Struben and AMM data centres for our central services subnet (146.231.128.0/21). This in turn will allow us to move some some of our central servers to the second data centre.
The VRRP change over went as planned, and didn't take more than a few seconds to complete. Unfortunately it seems that the ARP caches on our two core switches, as well as those on some of our servers, didn't like the fact that the routers' MAC addresses changed in the process. The most noticeable effect of this is that for the last hour or so, DNS resolution to sites outside the University has been erratic. This will have had knock-on effects; the proxy server, for instance, couldn't fetch pages because it couldn't do DNS lookups.

The problem appears to have been resolved by a hard reset of the two routers involved. It may, however, take a little while longer for things to stabilize.
post.5523892