Archive for the 'Network Issues' Category

Packet Loss in Global Switch London 2 & BlueSquare Maidenhead

Published on April 16, 2012 in Network Issues. Closed

Update 19/04/2012 20:01

We have been supplied an interim outage report for this issue.

Please contact the NOC if you would like a copy of this.

Update 14:29:

We have temporarily rerouted traffic, normal service should now be resumed.

Update 14:17:

Our upstream is working on re-routing the traffic.

Update 13:30:

One of our fibre providers have issued an outage notification. Updates will follow.

Update 13:14:

All EXN devices appear to be functioning with no problems or alerts. We raised the issue with our upstream provider at Global Switch 2 who have confirmed there is an issue on their side which they are currently investigating.

We will update you once we hear back.

Original Post:

Dear Customer,

We have noticed some packet loss and high latency at Global Switch 2. We are currently investigating the issue.

Updates will follow.

Kind Regards,
Network Operations Centre

 

 

Network issues at Maidenhead

Published on April 11, 2012 in Network Issues and Outages. Closed

Dear Client,

We are currently experiencing network issues at our Maidenhead site. Engineers are on site and investigating.  Updates will be posted.

Kind Regards,
Network Operations Centre

*Resolved* Intermittent Packet Loss @ Global Switch 2

Published on March 9, 2012 in Network Issues and Transit. Closed

Update 16:38

Upon investigation, our upstream have discovered OSPF route flapping between their network devices which caused routing instabilities and high CPU on few of the devices in GS2. As the result of the high CPU caused by routing table updates and re-calculations, packet loss on the interconnect links had increased causing the high latencies and almost complete packet loss at some time points. A full RFO has now been produced, please contact your TAM or raise a ticket at http://noc.exn.org.uk/ to receive the document.

Update 15:57

Our upstream is still investigating the cause of the issue.

Update 10:18

At approximately 09:27 GMT we received alerts of  increased latency and major packet loss on our devices at GS2. At approximately 10:09 full service was restored. During our investigations we found there were no issues on our devices so we raised a call with our upstream NOC to see if there were any issues on their end.

Around the same time they saw a drop of around 600 Mbps from their devices and have concluded the issue was on their side, they are currently investigating the issue and will produce an RFO which will be posted here.

Original Post

We appear to be having intermittent packet loss at Global Switch London 2. We will update you with more information shortly.

Kind Regards,
NOC.

*Completed* GSL2 Connectivity

Published on December 3, 2011 in Completed, Network Issues and Outages. Closed

Dear Customers,

We are experiencing an issue with one of our core switches in Global Switch 2. We are currently investigating the issue however it appears stella.gsl2.exn.org.uk is not passing traffic which is affecting a number of services.

Update 23:47

The device has now come back online. It appears this switch stack rebooted unexpectedly.

Update 00:00

We have raised an issue with our supplier to get to the bottom of the reboot. We don’t expect any more problems on this device and have put a network freeze on this device to prevent any more potential issues.