Resolved -
Our cluster services are running stably and we are not seeing any indication that another disruption will occur.
We continue to monitor our services t ensure uptime and stability.
May 6, 20:05 SAST
Update -
Cluster services are up and all functionality is restored to https://app.cloudafrica.net
Customers can again interact with VMs using the webapp.
We are monitoring the cluster to ensure stability.
May 6, 19:05 SAST
Update -
Our initial diagnosis was not complete and there are some residual issues to deal with.
While we bring the environment back online https://app.cloudafrica.net will remain unreachable.
Client VMs are still accessible over the internet (ssh, rdp , telnet etc)
May 6, 17:50 SAST
Monitoring -
All cluster services have successfully restarted and the cluster is restored to peak health.
We'll continue to monitor the cluster and specifically the one host affected earlier to ensure that disruptions are kept to a minimum.
All functionality on https://app.cloudafrica.net have been restored.
May 6, 16:55 SAST
Update -
Cluster services are still restarting and the CloudAfrica webapp will remain unavailable for another 30min-1 hour.
We are unfortunately not able to speed up this process, but all customer VM that were reachable before the cluster service restart will remain reachable and responsive.
May 6, 15:02 SAST
Update -
To resolve this issue and get the single host back into the cluster we need to restart all our cluster services.
https://app.cloudafrica.net will be unavailable for about 1 hour while the cluster restarts.
VMS not on the affected host, or VMs that were reachable via ssh/telnet/rdp before the cluster restart will remain available over the internet.
May 6, 14:27 SAST
Identified -
We've identified the processes on the host that are causing problems with reachability and basic functionality of guest VMs.
We have initiated the host reboot and we've working on these processes to get them all back up as soon as possible.
Some VMs may be reachable, but others will be offline for a short time.
May 6, 14:05 SAST
Investigating -
A single host on in our cluster has failing processes which required us to reboot the host.
Customers who have VMs on this host will lose access to the VM for a brief period and operations such as starting/stopped the VMs may not be functioning as expected via the CloudAfrica Webapp.
May 6, 14:00 SAST
