noticeboard.ru.ac.za

At around 2AM this morning, we lost all access to the Internet as a result of a problem affecting SANReN's Port Elizabeth point of presence.

You will likely experience this outage as an ERR_DNS_FAIL or ERR_CONNECT_FAIL error when trying to access websites. When accessing secure (https://) web sites, your browser may report that the connection has timed out or the site is unreachable.

The problem was reported to TENET at about 7:15AM. We've since had feedback that the cause of the problem is a fibre break on the circuit from Port Elizabeth to Durban. This has caused the Port Elizabeth PoP to become an island for reasons as yet unknown (there should be a second route from Port Elizabeth to Cape Town). As a result, all sites connected from Port Elizabeth (including Rhodes) cannot access the Internet.

We attempted to work around this problem in our usual manner, such that email continued to work via an alternative connection. Unfortunately, this seems to have crashed both our border firewalls. At this stage, it isn't clear whether the fault on TENET's network has been resolved, since our internal fault will be masking this. We're working on resolving the problem at the moment.

One of our border firewalls is now up again, confirming that the TENET problem remains.

I've just phone TENET's service desk for an update. I'm told Telkom is still trying to isolate the problem, and that as yet there is no estimated time to repair available.

I've just had an explanation of what's happened: the link from Port Elizabeth to Cape Town failed somewhere around Somerset West at 2AM this morning. Subsequently, at around 6AM, the link from East London to Durban failed. As a result, both of Port Elizabeth's links are currently not operational (Port Elizabeth connects to Cape Town, and to Durban via East London). This means we can probably reach higher educational sites in the Eastern Cape, but certainly not beyond. It still isn't clear why we were without access between 2AM and 6AM.

Unfortunately Telkom haven't yet come back to TENET with an estimate of the time to repair.

Access was restored at about 16:35.