New Horizon Hosting LLC - Notice history

The cables have been replaced, and the network is traversing traffic as expected. We will continue to monitor logs over the next 24 hours for interface errors. As stated previously, we sincerely apologize for any inconvenience this incident has caused.

Our network team has concluded the investigation, and we have determined the root cause to be failing direct attached copper (DAC) cables between our spines and edge routers. Due to the severity of failing cables, the impact was felt more drastically throughout the network. As a result, we plan to replace the failing cables in the early morning of Wednesday, April 30th, at off-peak hours to minimize disruptions. We will be performing replacements of these DACs with backups from our storage at an off-peak time. This is scheduled to occur at 10:00 UTC on Wednesday, April 30th. We will provide an update when this is completed. We apologize for any inconvenience this incident has caused.

Our network team is investigating deeper, but we can confirm that one of our core spine switches flapped all LACP interfaces. Physical interfaces remained online.

The flapping of the LACP interfaces caused some connections between our top-of-rack switches and carriers to flap that were connected to said spine switch. More info will be posted soon.

Everything has came back up again we will continue monitoring.

We are currently investigating this incident. We are working to resolve.

Services have appeared back online, we are still investigating the cause of this outage and we are monitoring services closely.

We have just had another mass drop over the network we are investigating.

All services have been successfully restored, and our teams will continue to closely monitor system performance to ensure stability and reliability. We remain committed to maintaining seamless service operations and will promptly address any further issues if they arise. Thank you for your patience and understanding.

We have identified an issue with a faulty network routing package. Our network engineering team is actively working to restore affected prefixes, and we anticipate that services will be back online within the next 30 minutes. We appreciate your patience and will provide further updates as necessary.

US-CHI-P02 is now operational! This update was created by an automated monitoring service.