Print

Print


Re: [HEAnet #1528346] Faults List: National and Dublin core upgrades
====================================================================


Ticket Details

 1528346                                                              
 Re: [HEAnet #1528346] Faults List: National and Dublin core upgrades 
 open                                                                 
 Mon Feb 25 23:00:00 2019                                             
 Fri Mar 01 03:00:00 2019                                             
 YES                                                                  

Problem Description

We are upgrading all MX480s on both National and Dublin core. Monday 25th of
Feb 23:00 -> 03:00 ---------------------------------- core2-dcu core2-tcd
core1-gal1 Tuesday 26th of Feb 23:00 -> 03:00
---------------------------------- core1-tcd core2-ucd core1-cork2 Wednesday
27th of Feb 23:00 -> 03:00 ------------------------------------ core1-ucd
core1-dcu core1-cork1 core1-lim2 Thursday 28th of Feb 23:00 -> 03:00
----------------------------------- core1-gal2 core1-lim1


Problem Effects

All of the listed clients will see one of their two BGP sessions with AS1213 go
down during reload. As we will only reload one at a time, you should not see a
loss of connectivity. You will need to verify you are receiving 2 default
routes on your routers form AS 1213, to ensure connectivity: sh ip bgp 0.0.0.0
[Cisco] show route protocol bgp 0.0.0.0/0 [JunOS] $user@$router> sh ip bgp
0.0.0.0 BGP routing table entry for 0.0.0.0/0 Paths: 2 available 1213
193.1.9.197 from 193.1.9.197 (193.1.9.197) Origin IGP, metric 5, localpref 100,
IGP metric 1, weight 0, received 3d12h ago, valid, internal, best 1213
87.44.56.113 from 87.44.56.113 (87.44.48.36) Origin IGP, metric 10, localpref
100, IGP metric 1, weight 0, received 3d12h ago, valid, external The metric of
5 and 10 are important. If you have any queries please contact [log in to unmask]


Last Update Message

Tue Feb 26 01:07:49 2019

Link to formatted ticket detailed the issues:
https://ticketing.heanet.ie/public/show.html?id=1528346


Affected Clients

 DCU, NUIG, SCHOOLS, TCD, UCC, UCD, UL, RCSI 

Affected Services

 IPv4 Connectivity 
 IPv6 Connectivity 

Transaction History

 Date Created              Message                                                                  
 Mon Feb 18 14:17:06 2019  Security patches                                                         
 Mon Feb 18 14:21:39 2019  Link to public ticket:                                                   
                              https://ticketing.heanet.ie/public/show.html?id=1528346                  
 Mon Feb 18 14:42:18 2019  Updated date and time.                                                   
 Tue Feb 26 01:05:56 2019  core2-dcu core1-gal1 were upgraded ok. We had an issue with core2-tcd,   
                              the downstream router edge2-tcd had a technical issue when core2-tcd     
                              was rebooted. A routing process needed to be killed. Consequently it     
                              wasn't accepting traffic from the resilient path via edge1-tcd. The      
                              following sites had a 15min approx outage while we troubleshooted the    
                              issue. edge1-esri edge1-ncad1 edge1-heanet-gd5 edge1-ucd-svuh            
                              edge1-ichec edge1-dit-mountjoy edge1-revenue edge1-tcd-stjames           
                              edge1-depot edge1-dit-bolton edge1-tcd-stackb edge1-oir                  
                              edge1-dit-chatham edge1-ucd-eccles edge1-tcd-ent edge1-ndrc              
                              edge1-educampus edge1-ria edge1-depot edge1-ichec edge1-ndrc We plan     
                              to further investigate the issue tomorrow. Please contact                
                              [log in to unmask] or the Service Desk on 01-6609040 for more information.    
 Tue Feb 26 01:06:38 2019  core2-dcu core1-gal1 were upgraded ok. We have an issue with             
                              core2-tcd, the downstream router edge2-tcd had a technical issue when    
                              core2-tcd was rebooted, and routing process needed to be killed.         
                              Consequently it wasn't accepting traffic from the resilient path via     
                              edge1-tcd. The following sites had a 15min approx outage while we        
                              troubleshooted the issue. edge1-esri edge1-ncad1 edge1-heanet-gd5        
                              edge1-ucd-svuh edge1-ichec edge1-dit-mountjoy edge1-revenue              
                              edge1-tcd-stjames edge1-depot edge1-dit-bolton edge1-tcd-stackb          
                              edge1-oir edge1-dit-chatham edge1-ucd-eccles edge1-tcd-ent edge1-ndrc    
                              edge1-educampus edge1-ria edge1-depot edge1-ichec edge1-ndrc We plan     
                              to further investigate the issue tomorrow. Please contact                
                              [log in to unmask] or the Service Desk on 01-6609040 for more information.    
 Tue Feb 26 01:07:48 2019  Link to formatted ticket detailed the issues:                            
                              https://ticketing.heanet.ie/public/show.html?id=1528346