Ticket Number: HEA-NOC/20031010-7 Ticket Status: UPDATE
Ticket Type: scheduled Resolver: HEA-NOC
Ticket Opened: 20031010 15:02 UTC+1 Problem Start: 20031022 08:00 UTC+1
Ticket Closed: Problem End:
Site/line: DCU, DIAS, DIT, HEA-NOC, NCAD, NCIRL, NUIG, NUIM, Tipp Inst,
UCC, UL, Citywest PoP, Dublin PoP, Kilcarbery Park PoP
Due to the well publicised Cisco IOS exploit in July almost all the
equipment in the HEAnet network had to be upgraded. The majority of this
work was done very shortly after the exploit was anounced however some
equipment was protected with ACLs and we are now scheduling upgrades to
secure this equipment without the need for ACLs.
The upgrades will be done over the course of four maintenance windows as
22th October - Citywest & Limerick: LS1010 x 2 & 2924 x 2
29th October - Cork & Galway: LS1010 & 2924 x 2
12nd November - BH: Ambreville, Prospero, Cooper, Orinoco & Balthazar.
19th November - Kilcarbery: 2950 & 3508
The corresponding outages in each case will be:
22th October - c. 10 minute outage to UL
29th October - c. 10 minute outage to both UCC and NUIG
12nd November - c. 10 minute outage to all Dublin ATM customers,
co-location LAN, video services LAN and HEAnet Office LAN.
19th November - No outages.
The ticket will be updated with further specific outage information at a
time closer to each ticket as it is possible that higher priority work may
affect the stated timeline.
20031010 14:42 BN Timeline set and ticket opened.
20031021 09:08 BN IOS' have been obtained. First maintenance on Citywest
PoP and Limerick PoP will go ahead as planned. As mentioned there will be
a roughly 10 minute outage to UL during this window.
20031022 08:25 BN Upgrades on Citywest and Limerick PoPs went smoothly.
Brief outage to UL, but for no more than a couple of minutes. New IOS' in
place. Work in Cork and Galways due to go ahead on Oct 29th.
20031030 10:05 BN Work in Cork and Galway on the 29th was delayed. This
work will now take place on the 12th November, pushing other work back by
one week. New running order is:
12th November - Cork & Galway: LS1010 & 2924 x 2
19th November - BH: Ambreville, Prospero, Cooper, Orinoco & Balthazar.
26th November - Kilcarbery: 2950 & 3508
20031111 11:09 BN Work is going ahead in Cork and Galway as planned
tomorrow. This will result in a five minute outage for both UCC and NUI
Galway as early as possible during the 2 hour window.
20031112 08:29 BN All upgrades in Cork and Galway PoPs completed, all
links back up, some further investigation needed on log messages observed
during the work, however all traffic is flowing normally.
20031118 09:55 BN Working going head in Brooklawn House as planned. This
will result in a five minute outage for all Dublin ATM customers (DCU,
DIAS, DIT, NCAD, NCI, Tipp Inst) and a five minute outage for co-located
equipment (HEA Oracle server, E-Grants, EPA, 6xxs).
20031119 08:42 BN All reloads went as planned. Outages were kept to
minimum, all switches now upgraded to new IOS'.
Time to Fix:
Multiple 2 hour windows across 1 month.
This ticket can be monitored at http://www.hea.net/tickets/20031010-7