Category: Updates

Another migration complete! No major issues. This leaves us EI and GT Savannah in two weeks. The College of Computing is awaiting scheduling pending some network equipment upgrades, at which point they will be added to NI.

Our largest migration to date is now complete. North Interconnect comprised of 126 virtual firewalls and included some large audiences such as the School of Electrical and Computer Engineering and the school of Civil and Environmental Engineering. The migration was largely trouble free except for an ICMP App-ID caveat that we resolved during the maintenance window and an issue that presented itself as a spanning tree issue with Nanotechnology that we were unable to resolve, so we left them on the ASAs for now. We will address that network during a future maintenance.

The end is near! Only three firewall sets remain in our primary migration phase!

We have moved the remaining networks on DC router to the Palo Altos. This completes the migration of the OIT data centers to Palo Alto appliances. As of now we have migrated 7 HA pairs of appliances (14 appliances total) and we have 4 pairs remaining plus one stand alone unit.

On Sunday at 6am we migrated all of the networks on rich-rtr and the most of the networks on dc-rtr to Palo Alto appliances. With improved coordination and communication with other OIT units this time around, the migration went significantly smoother and most services were restored by 9am. The remaining networks on dc-rtr are for services managed by ITG and they will migrate on Thursday September 28th.

We have some changes to the upcoming maintenance schedule:

Rich departmental firewalls and some DC departmental firewalls will be moved Sunday September 24th, during a 6am – 12pm window.

The remaining DC departmental firewalls, which service ITG networks, will be moved Thursday September 28th during a 5am – 7am window.

The College of Computing migration is postponed indefinitely pending some network architectural changes that need to happen first.

This one covered one of our data centers, and while the firewall migration itself went fine, the act of migration unfortunately broke a few services that relied on remote storage. Some lessons learned from this will hopefully reduce the impact of our next data center migration.

Another item of maintenance on Tuesday was to replace one of our new border firewalls with a warranty replacement. On Saturday August 26th, the data plane spontaneously restarted on one of our units. There was a successful failover, so there was minimal impact, however, Palo Alto determined that we had a hardware fault and sent us a new unit NBD. The migration of configuration, licenses, and management settings was pretty straightforward. It perhaps took an hour of preparation the previous work day, and a 10 minute swap-out time Tuesday morning.

In contrast to that we have had a drive failure in one of our Panorama boxes used for management of the appliances, and replacing that has been a bit more vexing for TAC. They finally sent us a correct replacement drive this week and it was installed today.

 

While we do plan to have some automated reporting in place in the near future, we have made a couple of quick graphs do show what sort of traffic is being dropped since our second of two border firewalls was deployed on August 15th. The Threat Prevention feature replaces the Intrusion Prevention feature of our old firewalls and is much more thorough. The Wildfire-Virus category in this graph typically covers viruses that are just hours old.

One new feature that we have implemented is URL Filtering, and we are filtering on two high risk categories: malware and phishing. Most of these blocks are due to adware programs or compromised machines visiting malicious websites, rather than users manually clicking on links.

We have now migrated the second of our two border firewalls to Palo Alto. This covers the remainder of campus and LAWN networks. So ends the era of Cisco ASAs at our border after many years of reliable service. Let us remember with this final denied flow.

2017-08-15T05:03:00.160049-04:00 campus-fw.gatech.edu %ASA-4-106023: Deny tcp src outside:221.182.236.2/43559 dst inside:128.61.235.19/1433 by access-group “inbound” [0x860a12f, 0x7e106aac]

We have migrated the first of our two border firewalls, covering Resnet and a portion of LAWN. The new border firewalls add AntiVirus, Anti-Spyware, Vulnerability Protection (aka Intrusion Prevention) and URL filtering for known malware and phishing sites.

 

All hands on deck for this one!

This morning we migrated the Resnet departmental firewall to a new Palo Alto unit. This includes all residence hall wired connections, as well as some affiliated units such as ATDC and Georgia Tech Dentistry.