close
MENU
1 mins to read

Civil Defence web servers ran out of memory


The Department of Internal Affairs said a change to the Civil Defence site to accomodate the Christchurch earthquake response caused the site's outage last Friday, just as Kiwis were going online in search of tsunami arrival time data.

Alex Walls
Fri, 18 Mar 2011

The Department of Internal Affairs said that a change made to its Civil Defence website in response to the Christchurch Earthquake caused last week’s outage. The site crash happened as news of the Japanese quake hit, and people went online looking for information about tsunami arrival times.

The website was offline intermittently for about two hours from 9.54pm and was unavailable in total for about 50 minutes, the department said (the department's home page was down at the same time, NBR found).

The site's servers failed and restarted about 20 times in those two hours.

The Department said the problem had been identified as a change made to accommodate part of the Christchurch Earthquake response, which in turn changed the way network traffic overload was handled.

Instead of queuing the web page requests to and from the servers, the overload data was dropped.

Senior communications manager Tony Wallace said the web servers had continued to hold connections data, but had repeatedly run out of memory and restarted themselves, which caused the website interruptions.

26,000 visits
There had been 26,000 individual visitors who successfully reached the site during the two hour period, Mr Wallace said, and that during other emergencies such as the Samoan Tsunami, more than double this volume had been successfully accommodated.

Mr Wallace said contingency measures had been put in place on Friday and the problem was resolved within about two hours. 

“The Department is putting fixes in place to prevent any similar problems. These include network changes and removing the source of the incident by queuing instead of dropping overload traffic.” 

Mr Wallace said the first report of the Japanese earthquake was received just after 8pm on 11 March, and the first information went up on the Civil Defence website at 9.10pm.  The tsunami potential threat was issued at 8.52 pm “based on analysis of data and scientific advice”.

Alex Walls
Fri, 18 Mar 2011
© All content copyright NBR. Do not reproduce in any form without permission, even if you have a paid subscription.
Civil Defence web servers ran out of memory
13186
false