Ticket Number: HEA-NOC/20080219-3      Ticket Status: UPDATE
Ticket Type:   unscheduled             Resolver:      HEA-NOC
Ticket Opened: 20080219 10:58 UTC      Problem Start: 20080218 17:45 UTC
Ticket Update: 20080219 13:12 UTC      Problem End:   20080219 12:05 UTC

Site/line:     Web Hosting

Problem Description:
Disk failure, leading to complete failure of 'samhain', the server. Due to
way in which the disk(s) failed, the RAID volumes were unable to
mount, resulting in the complete failure of the server.

Problem Effects:
Multiple hosted websites unavailable, including some internal HEAnet web

Problem History:
20080219 10:55 GAR Created ticket in ticketing system.
Services are being restored and tested.

Problem History:
20080219 09:45 GAR Created ticket manually to further inform clients of
failure. Due to the failure, production of NOC tickets is affected also.

MNS are working to bring this machine and related services back to live,
no estimate for restoration is available, as yet.

20080219 10:04 GAR Updated title of ticket, which was incorrectly pasted
by me. Apologies for any confusion caused. The problem is with web
hosted on only.
Work continues to fail over resilient services and restore service to

20080219 12:05 GAR Services have been brought back and provisionally
tested on the original hardware, and we are now ready to fail back from
the interim solution. Brief outages to web hosting services may occur as
the failback takes place.

This ticket will be updated in the next 60 minutes to confirm that the
failback has successfully taken place and that sites will be available for
read and write access, as normal.

20080219 13:09 GAR All services restored and available for production use.
No problems observed over the past hour of monitoring.

We will continue to monitor for 24 hours to ensure that no further
problems remain with the system.

Time to Fix:
18h 20m

This ticket can be monitored at

HEAnet Limited. Registered in Ireland, No. 275301.