Site was down

[Basil]

In case you notice earlier the site was down, apparently my data center that hosts my server had a major issue:

“We are getting reports of network disruptions from customers in Tampa and Los Angeles. Our Network Engineers are on the scene working on it diligently. We will have an update for you all shortly. We appreciate your patience.”

 

[Basil]

They have some of the data center back up, but not all. As our site seems to be back online, I'm assuming my server is in the part that is back on line.

 

[Gliderman8]

I had a problem with the site late in the afternoon.

 

[Basil]

Another note:

“Our Network Engineers are still investigating the events that caused today’s network event that impacted pockets of customers within TPA1, TPA2 and DAL1 data centers. Our Network Engineers do believe they understand what happened but we ask that you give us until tomorrow...”

 

[DrEntropy]

Hope you get an honest explanation. That was a fairly significant event.

 

[Basil]

I received a good and thorough explaination of what caused the problem. I won't go into all the details, but it was basically human error that occured while an engineer in Dallas was applying "policy updates" to group of routers and it didn't go as planned. The issue propagated throughout the entire network, which ended up affecting data centers in Dallas as well as two centers in Tampa. They also told us (their customers) what new procedures they are implementing so that the same error won't occur again.

I can't complain too much - over the years I've been with them their up time has been very good and they have always been extremely responsive to issues when I've had to submit a ticket.

 

[JPSmit]

it was basically human error that occured while an engineer in Dallas was applying "policy updates" to group of routers and it didn't go as planned. .......... They also told us (their customers) what new procedures they are implementing so that the same error won't occur again.