THIS IS A SCHEDULED EVENT Oct <var data-var='date'>13</var>, <var data-var='time'>07:00</var> - <var data-var='time'>16:00</var> NZDT Oct <var data-var='date'> 6</var>, <var data-var='time'>15:28</var> NZDT Update - We will be undergoing scheduled maintenance during this time. Oct <var data-var='date'> 6</var>, <var data-var='time'>15:27</var> NZDT Scheduled - We will be undertaking maintenance of the compute service in in the nz-por-1 region. During this period there will be a few brief periods of compute API downtime that should take less than five minutes. Servers and containers will continue operating as normal.

Compute maintenance (nz-por-1)

THIS IS A SCHEDULED EVENT Oct <var data-var='date'>12</var>, <var data-var='time'>07:00</var> - <var data-var='time'>16:00</var> NZDT Oct <var data-var='date'> 6</var>, <var data-var='time'>15:29</var> NZDT Update - We will be undergoing scheduled maintenance during this time. Oct <var data-var='date'> 6</var>, <var data-var='time'>15:26</var> NZDT Scheduled - We will be undertaking maintenance of the compute service in in the nz-hlz-1 region. During this period there will be a few brief periods of compute API downtime that should take less than five minutes. Servers and containers will continue operating as normal.

Compute maintenance (nz-hlz-1)

Sep <var data-var='date'>30</var>, <var data-var='time'>17:31</var> NZDT Resolved - This incident has been resolved. Sep <var data-var='date'>30</var>, <var data-var='time'>16:53</var> NZDT Monitoring - A fix has been implemented and we are monitoring the results. Sep <var data-var='date'>30</var>, <var data-var='time'>16:23</var> NZDT Investigating - We are currently investigating this issue.

issues scheduling new instances in nz-por-1

Sep <var data-var='date'>30</var>, <var data-var='time'>15:29</var> NZDT Resolved - This incident has been resolved. Sep <var data-var='date'>30</var>, <var data-var='time'>14:10</var> NZDT Monitoring - A fix has been implemented and we are monitoring the results. Sep <var data-var='date'>30</var>, <var data-var='time'>13:57</var> NZDT Identified - The issue has been identified and a fix is being implemented. Sep <var data-var='date'>30</var>, <var data-var='time'>13:45</var> NZDT Investigating - We're seeing an increase in packet loss in the Porirua region.

Packet loss in nz-por-1 region

Sep <var data-var='date'>23</var>, <var data-var='time'>18:34</var> NZST Completed - The scheduled maintenance has been completed. Sep <var data-var='date'>23</var>, <var data-var='time'>18:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>23</var>, <var data-var='time'>16:05</var> NZST Scheduled - We will be carrying out maintenance on our support ticketing system during this window. The support panel in our dashboard may not be usable at times during this window. If you need to raise an urgent support ticket, and the panel isn't available, and have premium support, please use that process to raise a ticket.

Maintenance of support ticketing system

Sep <var data-var='date'>21</var>, <var data-var='time'>12:30</var> NZST Completed - The scheduled maintenance has been completed. Sep <var data-var='date'>21</var>, <var data-var='time'>10:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>21</var>, <var data-var='time'>09:57</var> NZST Scheduled - We are performing an upgrade of the underlying systems of CCKS (Kubernetes/Magnum) API service in Porirua (nz-por-1). During this maintenance, the API will be unavailable for approximately 30 minutes in Porirua. This will impact operations to create, delete, resize, and upgrade Kubernetes clusters for the duration. Existing clusters, their Kubernetes API endpoints, and workloads will be unaffected.

Maintenance of CCKS (Kubernetes) service API

Sep <var data-var='date'>14</var>, <var data-var='time'>12:28</var> NZST Completed - The scheduled maintenance has been completed. Sep <var data-var='date'>14</var>, <var data-var='time'>10:26</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>14</var>, <var data-var='time'>10:25</var> NZST Scheduled - We are performing an upgrade of the underlying systems of CCKS (Kubernetes/Magnum) API service in Hamilton (nz-hlz-1). During this maintenance, the API will be unavailable for approximately 30 minutes in Hamilton. This will impact operations to create, delete, resize, and upgrade Kubernetes clusters for the duration. Existing clusters, their Kubernetes API endpoints, and workloads will be unaffected.

Aug <var data-var='date'>29</var>, <var data-var='time'>16:00</var> NZST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>29</var>, <var data-var='time'>10:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>21</var>, <var data-var='time'>15:00</var> NZST Update - This maintenance has been postponed by 1 week. It will now take place on Saturday 29th August. Aug <var data-var='date'>17</var>, <var data-var='time'>11:31</var> NZST Scheduled - We will be carrying out an upgrade on the system that powers our billing and user management services. Between 10:00 and 16:00 NZST the following functionality not be available for extended periods: * Sign-ups to Catalyst Cloud via https://catalystcloud.nz/signup will be unavailable. The rest of the website is unaffected. * The Project Usage page on the dashboard, and parts of the billing API, will be unavailable. Project usage quotations for the current month, and invoices for previous months, will not be able to be queried during the outage period. * The Project Details page on the dashboard will be unavailable. This means that you will not be able to view or change your account details, or credit card/payment method settings. Full service is expected to be restored by 16:00 NZST. If you wish to raise a support ticket, the Support Centre will remain available throughout the maintenance period. If you are a Premium Support customer and need to raise an urgent support ticket, please use that process as normal.

Billing and user management service upgrade

Aug <var data-var='date'>20</var>, <var data-var='time'>15:11</var> NZST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>20</var>, <var data-var='time'>14:43</var> NZST Verifying - Verification is currently underway for the maintenance items. Aug <var data-var='date'>20</var>, <var data-var='time'>10:30</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>19</var>, <var data-var='time'>12:33</var> NZST Scheduled - We will be performing maintenance on management routers. No impact to customer resources is expected.

Routine router maintenance in nz-hlz-1

Aug <var data-var='date'>19</var>, <var data-var='time'>14:45</var> NZST Resolved - This incident has been resolved. Aug <var data-var='date'>19</var>, <var data-var='time'>14:42</var> NZST Monitoring - A fix has been implemented and we are monitoring the results. Aug <var data-var='date'>19</var>, <var data-var='time'>14:37</var> NZST Identified - `api.nz-hlz-1.catalystcloud.nz` and `api.nz-por-1.catalystcloud.nz` are currently inaccessible. This will only affect any customers using these endpoints specifically; the standard endpoints most clients currently interact with, located at `api.nz-hlz-1.catalystcloud.io` and `api.nz-por-1.catalystcloud.io` are still working correctly. We are working on a solution to this issue.

OpenStack APIs not accessible from some domains

Aug <var data-var='date'>19</var>, <var data-var='time'>12:28</var> NZST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>19</var>, <var data-var='time'>10:30</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>19</var>, <var data-var='time'>09:39</var> NZST Scheduled - We will be undergoing scheduled maintenance during this time.

Aug <var data-var='date'>19</var>, <var data-var='time'>12:20</var> NZST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>19</var>, <var data-var='time'>12:08</var> NZST Verifying - Verification is currently underway for the maintenance items. Aug <var data-var='date'>19</var>, <var data-var='time'>09:40</var> NZST Scheduled - We will be undergoing scheduled maintenance during this time.

Aug <var data-var='date'>18</var>, <var data-var='time'>13:30</var> NZST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>18</var>, <var data-var='time'>12:40</var> NZST Verifying - Verification is currently underway for the maintenance items. Aug <var data-var='date'>18</var>, <var data-var='time'>11:30</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>18</var>, <var data-var='time'>11:11</var> NZST Scheduled - We will be doing some maintenance on the networking of C3 GPU nodes. This should have no impact on running VMs.

C3-GPU network maintenance

Aug <var data-var='date'>17</var>, <var data-var='time'>18:25</var> NZST Resolved - This incident has been resolved. Aug <var data-var='date'>17</var>, <var data-var='time'>15:49</var> NZST Monitoring - A fix has been implemented and we are monitoring the results. Aug <var data-var='date'>17</var>, <var data-var='time'>15:16</var> NZST Investigating - We have detected an issue with the networking on one of our GPU hypervisors and performing maintenance on the hypervisor.

Network interface issue with GPU hypervisor

Aug <var data-var='date'>13</var>, <var data-var='time'>08:23</var> NZST Resolved - This incident has been resolved. Aug <var data-var='date'>13</var>, <var data-var='time'>08:07</var> NZST Monitoring - A fix has been implemented and we are monitoring the results. Aug <var data-var='date'>13</var>, <var data-var='time'>08:01</var> NZST Identified - The issue has been identified and a fix is being implemented. Aug <var data-var='date'>13</var>, <var data-var='time'>07:53</var> NZST Investigating - We are currently investigating this issue.

Block storage performance degradation in nz-hlz-1

Aug <var data-var='date'>10</var>, <var data-var='time'>15:00</var> NZST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>10</var>, <var data-var='time'>11:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>10</var>, <var data-var='time'>10:44</var> NZST Scheduled - No customer impact expected.

Routine router maintenance in nz-por-1

Jul <var data-var='date'>31</var>, <var data-var='time'>08:16</var> NZST Resolved - This incident has been resolved. Jul <var data-var='date'>30</var>, <var data-var='time'>13:55</var> NZST Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'>30</var>, <var data-var='time'>13:50</var> NZST Identified - The issue has been identified and a fix is being implemented. Jul <var data-var='date'>30</var>, <var data-var='time'>12:21</var> NZST Investigating - We are seeing issues with connectivity to C3-GPU instances and are investigating.

C3-GPU networking issues

Jul <var data-var='date'>29</var>, <var data-var='time'>18:32</var> NZST Completed - The scheduled maintenance has been completed. Jul <var data-var='date'>29</var>, <var data-var='time'>18:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jul <var data-var='date'>29</var>, <var data-var='time'>17:42</var> NZST Scheduled - We will be carrying out maintenance on our support ticketing system during this window. The support panel in our dashboard may not be usable at times during this window. If you need to raise an urgent support ticket, and the panel isn't available, and have premium support, please use that process to raise a ticket.

Jul <var data-var='date'>28</var>, <var data-var='time'>17:18</var> NZST Resolved - This incident has been resolved. Jul <var data-var='date'>28</var>, <var data-var='time'>10:57</var> NZST Monitoring - The immediate issue has been resolved but we will continue to monitor the situation. Jul <var data-var='date'>28</var>, <var data-var='time'>10:18</var> NZST Update - We are investigating a potential cause. Jul <var data-var='date'>28</var>, <var data-var='time'>09:57</var> NZST Investigating - We are currently investigating this issue.

Degraded networking performance in nz-por-1

Jul <var data-var='date'>15</var>, <var data-var='time'>12:00</var> NZST Completed - The scheduled maintenance has been completed. Jul <var data-var='date'>15</var>, <var data-var='time'>11:30</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jul <var data-var='date'>15</var>, <var data-var='time'>11:27</var> NZST Scheduled - We will be undergoing scheduled maintenance during this time.

Maintenance of the identity servce

Jul <var data-var='date'> 9</var>, <var data-var='time'>14:10</var> NZST Completed - The scheduled maintenance has been completed. Jul <var data-var='date'> 9</var>, <var data-var='time'>13:40</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jul <var data-var='date'> 9</var>, <var data-var='time'>13:36</var> NZST Scheduled - We will be undergoing scheduled maintenance during this time.

Dashboard maintenance

Jun <var data-var='date'>24</var>, <var data-var='time'>19:00</var> NZST Completed - The scheduled maintenance has been completed. Jun <var data-var='date'>24</var>, <var data-var='time'>18:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jun <var data-var='date'>24</var>, <var data-var='time'>17:24</var> NZST Scheduled - We will be carrying out maintenance on our support ticketing system during this window. The support panel in our dashboard may not be usable at times during this window. If you need to raise an urgent support ticket, and the panel isn't available, and have premium support, please use that process to raise a ticket.

Jun <var data-var='date'>23</var>, <var data-var='time'>12:23</var> NZST Resolved - This incident has been resolved. Jun <var data-var='date'>23</var>, <var data-var='time'>12:13</var> NZST Monitoring - The object storage API in nz-hlz-1 is now working again. It was previously reported that the API in nz-por-1 was also unavailable, but it was unaffected and continued working throughout the outage period. We are now monitoring for any further issues. Jun <var data-var='date'>23</var>, <var data-var='time'>12:04</var> NZST Update - We are continuing to work on a fix for this issue. Jun <var data-var='date'>23</var>, <var data-var='time'>12:02</var> NZST Identified - The object storage API is currently unavailable in all regions. We have found the rootcause and are currently working on implementing a fix.

Object storage API outage

Jun <var data-var='date'> 3</var>, <var data-var='time'>17:41</var> NZST Resolved - The emergency maintenance is complete. Jun <var data-var='date'> 3</var>, <var data-var='time'>17:19</var> NZST Monitoring - The emergency maintenance in nz-hlz-1 is now complete. We are now monitoring for potential issues. Jun <var data-var='date'> 3</var>, <var data-var='time'>17:11</var> NZST Update - The emergency maintenance in nz-por-1 is complete. nz-hlz-1 is still ongoing. Jun <var data-var='date'> 3</var>, <var data-var='time'>17:01</var> NZST Identified - We are currently performing emergency maintenance on the network services in both the nz-por-1 and nz-hlz-1 regions. Workloads may see temporary interruptions in connectivity until the maintenance is complete.

Emergency network maintenance

May <var data-var='date'>28</var>, <var data-var='time'>13:05</var> NZST Completed - The scheduled maintenance has been completed. May <var data-var='date'>28</var>, <var data-var='time'>11:00</var> NZST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'>28</var>, <var data-var='time'>10:48</var> NZST Scheduled - We are performing an upgrade of the underlying systems of CCKS (Kubernetes/Magnum) API service in Hamilton (nz-hlz-1). During this maintenance, the API will be unavailable for approximately 30 minutes in Hamilton. This will impact operations to create, delete, resize, and upgrade Kubernetes clusters for the duration. Existing clusters, their Kubernetes API endpoints, and workloads will be unaffected.

Is Catalyst Cloud down?

Catalyst Cloud status is UP

Lastest published events

issues scheduling new instances in nz-por-1

Packet loss in nz-por-1 region

Maintenance of support ticketing system

Catalyst Cloud Services, Components and Regions

Recent Catalyst Cloud events

Compute maintenance (nz-por-1)

Compute maintenance (nz-hlz-1)

issues scheduling new instances in nz-por-1

Packet loss in nz-por-1 region

Maintenance of support ticketing system

Maintenance of CCKS (Kubernetes) service API