In the evening of Tuesday 18th February we experienced a serious disk error on our primary Cambodia web server. This has brought the service down for an extended amount of time.
We made the decision to attempt to fix and restore the disk in full rather than the quick fix option of reverting to the last full backup from 16/02. Reverting to the backup would have brought the server back online much quicker, but it would have meant we would have lost three days of data - emails and changes to your website. We felt the best option was to pursue the proper fix - even if it took some time.
Due to the large amounts of data involved the restore process took significantly longer than expected. Restoring the data completed in the early afternoon of Wednesday 19th February.
You may have seen intermittent access to your website and cPanel since around 15:00 on Wednesday while we worked to bring the server back online. Unfortunately as of 18:00 we were still encountering stability issues.
We finally resolved the issue and the server came back online and was stable as of 18:40. We have also now installed some brand new additional disks into the server to combat the root cause of the issue.
We are very pleased to report there has been no sign of data loss, so it was worth the extended resolution time for this result. Any emails sent to you during the outage would have been queued and will be delivered over the next day or so as the sending server retries. So everything should be back to normal for you by tomorrow morning.
It is the worst outage in our 5 year history and we hope it will not repeat itself anytime soon. While we make every effort to prevent situations like this, these things do happen sometimes.
We're deeply sorry for the inconvenience caused by this and hope no one has been too badly affected. If you require any support or have any further questions please don't hesitate to contact us at firstname.lastname@example.org.
Wednesday, February 19, 2020