Verio experienced an outage on ns2.secure.net in the Dulles, Virginia datacenter on March 5th, 2012 from approximately 12:08 PM to 13:02 PM EST. In analysis the outage was initiated by a significant DOS attack that was directed at ns2.secure.net. Verio Operations worked to bring ns2 back online but was unable to do so due to the ongoing DOS attack and due to hardware/software failures on this machine as a result of the outage. Verio Operations then took ns2 completely offline so that ns1 in Dulles and the corresponding sets of nameservers in San Jose and Tokyo could handle the traffic from ns2. During the outage some customers may have experienced time-outs on DNS lookups if they were trying to reach ns2 in Dulles. The other nameservers in Dulles, San Jose, and Tokyo were not affected during this outage.
Verio Operations has been working with Nominum to get ns2 in Dulles back online and has built a spare server if needed. Over the past few days Verio Operations has detected several brief windows of DNS lookup issues. These issues tend to be experienced by customers that would primarily be using the Verio nameservers in Dulles. Additionally, customers with low TTL settings in their DNS would be most impacted. It is recommended that Partners review DNS setting for impacted domains and consider resetting their TTL back to 24 hours to maximize the usage of DNS caching. Verio Operations is working to bring ns2 in Dulles back online over this weekend. However, there are other redundant nameservers in San Jose and Tokyo that are handling the additional load. Verio Operations is working to restore the nameservers to a stable state, as they were prior to this event.
AlpineWeb has been working pro-actively with Verio to review and manage TTL settings for our customers who are using Verio DNS.
If you are an AlpineWeb Hosting Customer and have any further questions please contact us directly:
