NOC – Status Update

DFW Upgrade Complete

Thanks to everyone for your patience! Due to some large accounts, the transfer took longer than expected because the data had to be verified to make sure it all copied over and nothing was corrupted. After everything was transferred, we had to swap the IP addresses so customers did not have to change anything on their end. This also took extra time and the DNS cluster was out of sync and we had to rebuild the DNS cluster. It took time for this to all sync back up. Nginx also had to be repaired as it was bound to the original IP address and did not dynamically update. After the change, the license did not update automatically either, so it had to be reissued and then all services were restarted. After this was completed, websites came back up. We will continue to monitor performance and make any optimization tweaks.

DFW Upgrade

We will be upgrading the DFW server on Thursday 12/04 starting at 9PM CST and ┬áhope to have this completed by 5AM CST. We expect very little downtime, but there’s a chance once the transfer is complete and we move over the DNS and the IP address, there might be a small window when sites are not accessible. Any new post or e-commerce transactions between the start time and the finish time may not show up on the new server, so please plan accordingly.

DFW is our oldest server and while the sustained traffic is below its limits, the traffic spikes can push it over the edge. While we optimize to get the most out of the hardware, there comes a time when every server must be upgraded. The new server will have at least twice the processing power and five times the hard drive performance. Since the majority of our customers are on WordPress, these two improvements will drastically reduce load times.

We thank you for your patience in this matter and apologize for any inconvenience this may cause. Happy Holidays!

Nginx Bug – Nginx crashes due to improper subdomain creation

We discovered a bug in Nginx that causes the Nginx service to crash whenever a subdomain is created improperly. In all of the cases where Nginx crashed and failed to restart, we were able to view the logs and identify the account that caused the crash. Upon inspection of the account, we see that there is a subdomain folder, however, there is no subdomain or addon domain inside of cPanel. Nginx sees this folder as a subdomain and creates a virtual host but crashes since there is no identifier inside of cPanel. We have contacted Nginx and they have released a new version that addresses this bug. We will be applying this update shortly to all servers.

To avoid issues in the future, it’s important to always follow the cPanel wizards whenever creating or modifying accounts. In this case, the proper way to create a subdomain or addon domain is by logging into cPanel and following the wizard in either Subdomain or Addon Domains under the Domains section. This will properly create the folder structures and permissions.