cPanel PHP Version Upgrade – 15/05/2012

Published on May 15, 2012 in Planned Work and Shared Hosting. Closed

Update 18:27 15/05/2012

Dear Customers,

All servers have now been upgraded. This concludes this maintenance window.

Kind Regards,
NOC.

Original Message:

Dear Customers,

We will be upgrading PHP from version 5.3.8 to 5.3.10 today after 17:00 GMT. We do not expect any adverse affects as this is a minor upgrade, however we would say all cPanel servers are “at risk” during the hours outlined below.

Why?

PHP 5.3.8 is now end of life and an upgrade is required to keep up with the latest security patches.

When?

Start: Tuesday 15/05/2012 – 17:00 GMT
End: Tuesday 15/05/2012 – 20:00 GMT

If you have any questions please don’t hesitate to contact support.

Kind Regards,
Network Operations Centre.

*Scheduled* Maidenhead to Global Switch Fibre Maintenance

Published on May 3, 2012 in Planned Work and Transit. Closed

Dear Customer,

You are receiving this email as notification of the following maintenance work that will be carried in BlueSquare Maidenhead – please note this is “Service Affecting” maintenance. This work will be carried out on Friday 4th May 2012 19:00 – 00:00 GMT.

What?

Our fibre provider who we use to connect our Maidenhead site to Global Switch London 2 will be performing essential maintenance to their network to increase capacity. They will be upgrading their core switches in this site, and also adding a new 10 Gbe connection between BlueSquare 1 and 3. During the insertion of the new switch and 10Gb links, customers will see traffic drop as layer 2 and layer 3 re-converges. There will also be some packet loss experienced when Vlans are migrated to the new 10Gb infrastructure.

Why?
Certain customers within Bluesquare Maidenhead have been experiencing latency issues due to bottlenecks on the fibre network. As traffic flows have increased the latency issues and packet loss during peak times have reached unacceptable levels and need to be addressed as a priority. We wish restore the confidence with our customers in the fibre network and would ask that you bear with us whilst we implement this upgraded infrastructure.

When?

Start Date and Time: 19:00 05/05/2012
End Date and Time: 00:00 06/05/2012

If you have any questions please contact the NOC.

Kind Regards,
Network Operations Centre.

*Emergency Maintenance* – 24/04/2012 – Line Card Replacement

Published on April 24, 2012 in Emergency Work. Closed

Dear Customer,

The following notification is to inform you of emergency remedial works required on our upstream’s core router located at Telehouse East. Please note this is “service affecting” work and will involve replacing the line card in slot 9 as a result of the issues experienced last week.

Affected Services:

IP Transit HSRP enabled customers will experience a short outage/packet loss whilst traffic re-routes over alternative paths. Customers directly connected to this line card will also be affected whilst being moved to the replacement hardware.

When?

Tuesday 24/04/2012 23:00 – 25/04/2012 – 01:00 GMT

*Emergency Maintenance* – 24/04/2012 – BGP Configuration

Published on April 24, 2012 in Emergency Work. Closed

Dear Customer,

The following notification is to inform you of emergency remedial works required on our upstream core routers located at Telehouse East and Telehouse North.

We will be performing maintenance to connect the two neighbours on Tuesday 24th April, starting at 19:00 UTC (20:00 BST).  While this procedure should be non-disruptive, an “at risk” status is being announced as these are core routers having inherently complex configurations, and so difficult-to-predict side effects may be more likely to occur.  It is anticipated that all work will be completed within one hour of the commencement of the maintenance.

When?

Tuesday 24/04/2012 – 19:00 – 20:00 GMT (The actually duration shall be no more than 1 hour).

If you have any questions please let us know.

Kind Regards,
Network Operations Centre

*Scheduled* – Maidenhead Generator Maintenance.

Published on April 24, 2012 in Planned Work. Closed

Dear Customer,

We have been informed by BlueSquare/Pulsant that they will be performing remedial generator maintenance on 01/05/2012 and 02/05/2012

What?

As part of our ongoing remedial works Pulsant will be changing and improving the bund alarms on the installed generators, and taking the opportunity to replace fan and alternator belts with new at the same time.

Why?

As part of our ongoing planned and preventative maintenance regime, and our commitment to environmental best practice, we carry out this work to ensure that the service offered to our clients is the best possible.

This work forms part of this regime.

When?

The work will take place over 2 evenings, to ensure that only one generator is being worked on, and returned to service at any one time.

The first engine will be worked on on Tuesday 1st May at 21:00 and is scheduled to last for 4 hours.

The second engine will be worked on on Wednesday 2nd May at 21:00 and is scheduled to last for 4 hours.

If you have any questions please get in touch through the usual channels.

Kind Regards,
Network Operations Centre

Packet Loss in Global Switch London 2 & BlueSquare Maidenhead

Published on April 16, 2012 in Network Issues. Closed

Update 19/04/2012 20:01

We have been supplied an interim outage report for this issue.

Please contact the NOC if you would like a copy of this.

Update 14:29:

We have temporarily rerouted traffic, normal service should now be resumed.

Update 14:17:

Our upstream is working on re-routing the traffic.

Update 13:30:

One of our fibre providers have issued an outage notification. Updates will follow.

Update 13:14:

All EXN devices appear to be functioning with no problems or alerts. We raised the issue with our upstream provider at Global Switch 2 who have confirmed there is an issue on their side which they are currently investigating.

We will update you once we hear back.

Original Post:

Dear Customer,

We have noticed some packet loss and high latency at Global Switch 2. We are currently investigating the issue.

Updates will follow.

Kind Regards,
Network Operations Centre

 

 

Network issues at Maidenhead

Published on April 11, 2012 in Network Issues and Outages. Closed

Dear Client,

We are currently experiencing network issues at our Maidenhead site. Engineers are on site and investigating.  Updates will be posted.

Kind Regards,
Network Operations Centre

*Scheduled* – At Risk – Milton Keynes, Maidenhead Fibre Work

Published on March 9, 2012 in Emergency Work, Planned Work and Transit. Closed

Dear Customer,

We have a planned maintenance window starting Friday 9th March 22:00 GMT.

What?
Our fibre provide will be carrying out short notice maintenance in order to investigate & fix a deterioration of service in one of our ODF trays located in the Telecity HEX datacentre. This will affect our fibre links between Maidenhead – HEX and Milton Keynes – HEX.

The maintenance will take place over two evenings, your services should be considered at risk during this period.

Services At Risk:

BlueSquare Milton Keynes – All Services
BlueSquare Maidenhead – All Services

Investigations are required to confirm the fault, a fix will then be put in place the following evening.

When?
The maintenance will take place at the following times:

Start: 09/03/2012  22:00 GMT
End: 10/03/2012 04:00 GMT
Start: 10/03/2012 22:00 GMT
End: 11/03/2012 04:00 GMT

We apologise for any inconvenience this may cause, please feel free to contact support through the usual channels if you have any questions.

*Resolved* Intermittent Packet Loss @ Global Switch 2

Published on March 9, 2012 in Network Issues and Transit. Closed

Update 16:38

Upon investigation, our upstream have discovered OSPF route flapping between their network devices which caused routing instabilities and high CPU on few of the devices in GS2. As the result of the high CPU caused by routing table updates and re-calculations, packet loss on the interconnect links had increased causing the high latencies and almost complete packet loss at some time points. A full RFO has now been produced, please contact your TAM or raise a ticket at http://noc.exn.org.uk/ to receive the document.

Update 15:57

Our upstream is still investigating the cause of the issue.

Update 10:18

At approximately 09:27 GMT we received alerts of  increased latency and major packet loss on our devices at GS2. At approximately 10:09 full service was restored. During our investigations we found there were no issues on our devices so we raised a call with our upstream NOC to see if there were any issues on their end.

Around the same time they saw a drop of around 600 Mbps from their devices and have concluded the issue was on their side, they are currently investigating the issue and will produce an RFO which will be posted here.

Original Post

We appear to be having intermittent packet loss at Global Switch London 2. We will update you with more information shortly.

Kind Regards,
NOC.

*Scheduled* Client Portal Maintenance Tonight

Published on February 14, 2012 in Client Portal and Planned Work. Closed

Dear Customer,

We have a planned maintenance window starting 14/02/2012 – 17:00 GMT

What?
EX Networks will be performing upgrade works on our Client Portal and Ticketing system. During the maintenance window customers may experience sporadic outages of http://noc.exn.org.uk/. Our support email support@exn.org.uk will remain active and monitored.

Why?
This maintenance is essential to stay up to date with the latest patches of our software we use to power the Client Portal. With this upgrade we will also be rolling out some new features.

When?
Start: 14/02/2012 17:00 GMT
End: 15/02/2012 05:00 GMT

We apologise for any inconvenience this may cause, please feel free to contact support through the usual channels if you have any questions.

Kind Regards,
EXN NOC