Archive for the 'Transit' Category

*Scheduled* Maidenhead to Global Switch Fibre Maintenance

Published on May 3, 2012 in Planned Work and Transit. Closed

Dear Customer,

You are receiving this email as notification of the following maintenance work that will be carried in BlueSquare Maidenhead – please note this is “Service Affecting” maintenance. This work will be carried out on Friday 4th May 2012 19:00 – 00:00 GMT.

What?

Our fibre provider who we use to connect our Maidenhead site to Global Switch London 2 will be performing essential maintenance to their network to increase capacity. They will be upgrading their core switches in this site, and also adding a new 10 Gbe connection between BlueSquare 1 and 3. During the insertion of the new switch and 10Gb links, customers will see traffic drop as layer 2 and layer 3 re-converges. There will also be some packet loss experienced when Vlans are migrated to the new 10Gb infrastructure.

Why?
Certain customers within Bluesquare Maidenhead have been experiencing latency issues due to bottlenecks on the fibre network. As traffic flows have increased the latency issues and packet loss during peak times have reached unacceptable levels and need to be addressed as a priority. We wish restore the confidence with our customers in the fibre network and would ask that you bear with us whilst we implement this upgraded infrastructure.

When?

Start Date and Time: 19:00 05/05/2012
End Date and Time: 00:00 06/05/2012

If you have any questions please contact the NOC.

Kind Regards,
Network Operations Centre.

*Scheduled* – At Risk – Milton Keynes, Maidenhead Fibre Work

Published on March 9, 2012 in Emergency Work, Planned Work and Transit. Closed

Dear Customer,

We have a planned maintenance window starting Friday 9th March 22:00 GMT.

What?
Our fibre provide will be carrying out short notice maintenance in order to investigate & fix a deterioration of service in one of our ODF trays located in the Telecity HEX datacentre. This will affect our fibre links between Maidenhead – HEX and Milton Keynes – HEX.

The maintenance will take place over two evenings, your services should be considered at risk during this period.

Services At Risk:

BlueSquare Milton Keynes – All Services
BlueSquare Maidenhead – All Services

Investigations are required to confirm the fault, a fix will then be put in place the following evening.

When?
The maintenance will take place at the following times:

Start: 09/03/2012  22:00 GMT
End: 10/03/2012 04:00 GMT
Start: 10/03/2012 22:00 GMT
End: 11/03/2012 04:00 GMT

We apologise for any inconvenience this may cause, please feel free to contact support through the usual channels if you have any questions.

*Resolved* Intermittent Packet Loss @ Global Switch 2

Published on March 9, 2012 in Network Issues and Transit. Closed

Update 16:38

Upon investigation, our upstream have discovered OSPF route flapping between their network devices which caused routing instabilities and high CPU on few of the devices in GS2. As the result of the high CPU caused by routing table updates and re-calculations, packet loss on the interconnect links had increased causing the high latencies and almost complete packet loss at some time points. A full RFO has now been produced, please contact your TAM or raise a ticket at http://noc.exn.org.uk/ to receive the document.

Update 15:57

Our upstream is still investigating the cause of the issue.

Update 10:18

At approximately 09:27 GMT we received alerts of  increased latency and major packet loss on our devices at GS2. At approximately 10:09 full service was restored. During our investigations we found there were no issues on our devices so we raised a call with our upstream NOC to see if there were any issues on their end.

Around the same time they saw a drop of around 600 Mbps from their devices and have concluded the issue was on their side, they are currently investigating the issue and will produce an RFO which will be posted here.

Original Post

We appear to be having intermittent packet loss at Global Switch London 2. We will update you with more information shortly.

Kind Regards,
NOC.

*Completed* 28/12/2011- Global Switch London 2 Router Maintenance

Published on December 28, 2011 in Completed, Planned Work and Transit. Closed

Dear Customer,

We have a planned maintenance window starting 28/12/2011 -23:00 GMT

What?
EX Networks will be performing upgrade works on our core equipment in Global Switch 2. This will affect a number of services including all colocation clients on the 7th and 9th floors. During this time, both edge routers will be upgraded and circuits will be rerouted. Customers may notice an outage of up to 10 minutes during this time. This maintenance will be carried out by on-site EX Networks engineers as well as engineers from our upstream providers.

Why?
This maintenance is essential to increasing our capacity at Global Switch 2 and is in line with our upgrade roadmap for 2012.

When?
Start: 28/12/2011 23:00 GMT
End: 28/12/2011 23:59 GMT

We apologise for any inconvenience this may cause, please feel free to contact support through the usual channels if you have any questions.

Kind Regards,
EXN NOC

Update 23:31 – Engineers have started the above work.

Update 00:02 – All works have now been completed. We will continue to monitor the routers throughout the night. If customers are experiencing any issues please contact support.

*Completed* – At Risk – Loss of redundancy on Maidenhead – London link

Published on November 30, 2011 in Completed, Planned Work and Transit. Closed

What?
We have been informed by our fibre provider that they will be performing maintenance work in the window below on one of our fibre from Telecity HEX 8/9 to Maidenhead. During this time our Maidenhead site will be ‘at risk’

Why?
Our fibre provider requires to complete cable re-splicing in order to divert fibre around a major National Rail project

When?
Start: 10/12/2011 22:00 UTC
End: 11/12/2011 09:00 UTC

Update 14:62 11/12/2011

The planned maintenance on the fibre between Telecity HEX 8/9 to Maidenhead has been completed and given an all clear. This is an update to say that the at risk period is now over.