Aug <var data-var='date'>20</var>, <var data-var='time'>14:15</var> UTC Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>20</var>, <var data-var='time'>13:15</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>20</var>, <var data-var='time'>00:01</var> UTC Scheduled - We will be performing scheduled security maintenance across all Konnect regions. This maintenance is part of our ongoing efforts to maintain the security and reliability of the platform. Expected Impact: * Maintenance will be performed across all production regions. * Customers may experience brief periods of degraded Control Plane functionality while the maintenance is in progress. * Gateway Data Plane traffic will not be affected and will continue to process requests normally. We appreciate your understanding as we continue to enhance the security, stability, and resilience of the Konnect platform.

Scheduled Security Maintenance – All Regions

Aug <var data-var='date'>20</var>, <var data-var='time'>01:16</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'>19</var>, <var data-var='time'>20:39</var> UTC Identified - Analytics queries may be unavailable or returning incomplete data in EU.

Analytics queries may be unavailable or returning incomplete data in EU.

Aug <var data-var='date'>17</var>, <var data-var='time'>22:10</var> UTC Resolved - From 21:33z to 21:51z the Konnect Control Plane had an elevated error rate when updating some entities. This was due to a misconfiguration in our encryption policy evaluation that temporarily rejected all fields that required encryption.

Elevated error rates from control plane APIs in US Region

Aug <var data-var='date'>12</var>, <var data-var='time'>06:00</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'>11</var>, <var data-var='time'>14:31</var> UTC Investigating - Some customers may notice some missing data from charts and graphs while this lag occurs. The team is working to fix the processing and back-fill any missing data.

Some analytics ingestion is lagging in US prod

Aug <var data-var='date'> 1</var>, <var data-var='time'>00:07</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'>31</var>, <var data-var='time'>23:42</var> UTC Monitoring - Infrastructure maintenance on the US region is complete. We will continue to monitor this region to ensure stability of the platform. Jul <var data-var='date'>31</var>, <var data-var='time'>23:25</var> UTC Investigating - We will be performing maintenance. Customers may experience brief intermittent service disruptions or increased latency. We are working to minimize any impact and will provide updates here as the maintenance progresses. We apologize for any inconvenience and appreciate your patience. Expected Impact: - Maintenance will occur in the US region only. - During this time, configuration or policy updates in Konnect may be delayed until the maintenance completes in that region. - Gateway data plane traffic will not be affected and will continue to process requests normally. We do not anticipate customer impact, but brief transient connectivity disruptions may occur while changes are being applied.

Performing Infrastructure Maintenance - US Region

Jul <var data-var='date'>24</var>, <var data-var='time'>13:30</var> UTC Resolved - We identified a brief period of elevated API errors in the EU region, from 13:21 - 13:23 UTC, resulting in HTTP 5xx responses across multiple Konnect services and potentially impacting actions taken on the Konnect UI. The underlying cause is still under investigation.

Instability across Konnect services in EU

Jul <var data-var='date'>13</var>, <var data-var='time'>09:10</var> UTC Resolved - This issue has been resolved. Jul <var data-var='date'>13</var>, <var data-var='time'>08:44</var> UTC Update - We are continuing to investigate the issue. Jul <var data-var='date'>13</var>, <var data-var='time'>07:31</var> UTC Update - We are continuing to investigate the issue. Jul <var data-var='date'>13</var>, <var data-var='time'>07:10</var> UTC Investigating - Login to Konnect intermittently failing. Customers can retry login in order to gain access to the platform. API calls to Konnect are succeeding.

Konnect UI Login Degraded

Jul <var data-var='date'>10</var>, <var data-var='time'>02:41</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'>10</var>, <var data-var='time'>01:55</var> UTC Investigating - Customers are unable to visualize the platform usage analytics data. This does not impact data ingestion.

Konnect Observability UI Experience Degraded

Jul <var data-var='date'> 2</var>, <var data-var='time'>14:01</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'> 2</var>, <var data-var='time'>13:56</var> UTC Monitoring - We are monitoring an incident that resulted in an increased error rate in the Portal APIs. A fix has been applied, and error rates have decreased.

Increased error rate in the Portal APIs

Jul <var data-var='date'> 2</var>, <var data-var='time'>09:00</var> UTC Resolved - The Konnect control plane in the India was incorrectly rejecting all user tokens between 08:30am UTC to 9:08am UTC. A misconfiguration of the token validation URL was deployed, noticed by our monitoring, and rolled back. No dataplane traffic or proxy behavior would have been impacted. Impacts were: - API calls, including from tools like DecK and Terraform would receive 401 responses - Users on the Konnect UI may have been logged out if the UI made any control plane calls that resulted in a 401 response

Authentication issue in IN Region for Konnect Control Plane

Jun <var data-var='date'>29</var>, <var data-var='time'>15:56</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'> 2</var>, <var data-var='time'>19:48</var> UTC Update - Konnect Middle East Region: Indefinite Outage The Konnect Middle East Region is undergoing a prolonged outage due to the operational loss of AWS datacenters in the region. The duration of this outage is undetermined. As a result, we have no timeline for service restoration in the ME Region Customers are encouraged to migrate to one of our other Konnect control plane hosting regions in the EU, Australia, India or the United States. If assistance is required, please reach out to your Kong support representative. Mar <var data-var='date'>23</var>, <var data-var='time'>23:19</var> UTC Update - We are observing increased error rates across services in the Konnect Middle East region. Data plane proxy traffic is not impacted. We are actively working with AWS to resolve the issue and will provide updates as more information becomes available. Mar <var data-var='date'>20</var>, <var data-var='time'>21:12</var> UTC Update - Customers in the Middle East (Bahrain) region are operating in a reduced Availability Zone (AZ) configuration due to an ongoing AWS infrastructure issue. The platform remains available, but some customers in this region may experience intermittent performance degradation or elevated error rates. Data plane proxy traffic is not impacted. We are actively working with AWS to resolve the issue and will provide updates as more information becomes available. Mar <var data-var='date'> 5</var>, <var data-var='time'>13:16</var> UTC Update - It was discovered that DNS for KIC reconciliation endpoints were still pointing to the out-of-service region. We have failed over these DNS records now to properly point to our new region. Customers using KIC would have seen 500s' during the reconciliation to Konnect. This was resolved at 10:30 UTC on March 5th. Mar <var data-var='date'> 3</var>, <var data-var='time'>08:53</var> UTC Update - Current Status Summary for Konnect Middle East Region: All services have been moved to our failover region in Middle East Almost all services are operational, with 2 caveats: - Analytics data is delayed several hours. Konnect is accepting and persisting Analytics data, but it is not yet visible or queryable as we continue investigating restoring service. - The Debugger is operational including trace data; however session logs will not be available in this region. - Observability is operational; however UUID to display name resolution will not be available in this region. Everything else is back online. The team will update as we make progress on these remaining items. Mar <var data-var='date'> 3</var>, <var data-var='time'>05:35</var> UTC Update - Current Status Summary for Konnect Middle East Region: All services have been moved to our failover region in Middle East Almost all services are operational, with 2 caveats: - Analytics data is delayed several hours. Konnect is accepting and persisting Analytics data, but it is not yet visible or queryable as we continue investigating restoring service. - Debugger is unable to accept incoming trace data. While debugging sessions can be started, none of the trace data will be persisted in Konnect until the datastore for traces is restored. Everything else is back online. The team will update as we make progress on these remaining items. Mar <var data-var='date'> 3</var>, <var data-var='time'>05:13</var> UTC Update - The data store for traces sent to Debugger is still experiencing issues in the Middle East region. Debugging sessions can be started but no spans will be able to be ingested. The team will update as they bring that service online. Mar <var data-var='date'> 3</var>, <var data-var='time'>03:45</var> UTC Update - Service remains restored, and Kong engineers will continue to monitor the apps in the new region. Mar <var data-var='date'> 3</var>, <var data-var='time'>03:03</var> UTC Update - User permission data is now propagating normally again. Mar <var data-var='date'> 2</var>, <var data-var='time'>22:45</var> UTC Update - Status Update (PDP – ME-SOUTH-1) User permission data replication in ME-SOUTH-1 (Bahrain) region are experiencing extended latencies. Mitigation has been implemented to cap this to 15 minutes. Updated data should now propagate within a maximum of 15 minutes. We continue to monitor the situation and will provide further updates as needed. Mar <var data-var='date'> 2</var>, <var data-var='time'>21:50</var> UTC Update - The following services have been restored: - All Konnect UI-accessible products except for Event Gateway and Metering - Dataplane connections to the Konnect Control Plane for configuration download and analytics upload Analytics data is delayed several hours. Konnect is accepting and persisting Analytics data, but it is not yet visible or queryable as we continue investigating restoring service. Mar <var data-var='date'> 2</var>, <var data-var='time'>21:39</var> UTC Monitoring - The Kong engineering team is now verifying restored services in the Middle East region Mar <var data-var='date'> 2</var>, <var data-var='time'>21:35</var> UTC Update - We have restored partial service to Konnect. The UI is working again for Gateway Manager and Dev Portal, as well as all backing APIs. Dataplane connections to the Control Plane and Telemetry endpoints have been restored for 3 hours and should continue to work. Mar <var data-var='date'> 2</var>, <var data-var='time'>20:05</var> UTC Update - We continue to make progress on restoring to a failover region for the Middle East region. Key AWS services to support our failover efforts are beginning to work as of 11am PST. The Konnect engineering team is working to restore and validate all services and data and will update in one hour. Please note the control plane and telemetry endpoints are already restored, so dataplanes can connect to Konnect. Mar <var data-var='date'> 2</var>, <var data-var='time'>17:36</var> UTC Update - Control Plane restored for Config Download and Telemetry We have restored the control plane and telemetry endpoints. Dataplanes can now connect to the control plane api at `me.cp0.konghq.com` and `me.tp0.konghq.com` This is a read-only instance set up for this incident. Configuration changes are still blocked. But now customers should be able to connect new dataplanes and download their configuration, as well as send telemetry. The Admin API for changing control plane configuration remains down. Mar <var data-var='date'> 2</var>, <var data-var='time'>17:26</var> UTC Update - We are actively migrating the affected services to operate on other Availability Zones in the Bahrain region (ME-SOUTH-1 Region), but critical AWS services are not responding in that region. Our engineers are working to restore full service functionality and we are communicating with AWS to attempt to determine our failover options. At this time we cannot provide an ETA. We will provide further updates as they become available. Thank you for your patience. Mar <var data-var='date'> 2</var>, <var data-var='time'>15:26</var> UTC Update - We are actively migrating the affected services to operate on the two healthy Availability Zones in the Bahrain region (ME-SOUTH-1 Region). Our engineers are working to restore full service functionality and expect the migration to be completed within the next few hours. We will provide further updates as they become available. Thank you for your patience. Mar <var data-var='date'> 2</var>, <var data-var='time'>11:27</var> UTC Update - The incident is ongoing. We are working closely with our cloud provider to monitor restoration efforts. Due to significant physical infrastructure damage in both the UAE and Bahrain regions, recovery is expected to take more time. We appreciate your patience as we work toward a full resolution. Mar <var data-var='date'> 2</var>, <var data-var='time'>06:31</var> UTC Update - We are currently observing renewed elevated error rates across multiple Konnect services in ME region related to the AWS outage in that region. Mar <var data-var='date'> 2</var>, <var data-var='time'>01:54</var> UTC Identified - We are seeing many services in ME region recovering and some services are in degraded state. Dataplanes are not affected as part of this incident. All other Konnect regions are 100% operational and healthy. Our engineers are closely monitoring the situation. We will provide additional updates as soon as more information becomes available. Mar <var data-var='date'> 1</var>, <var data-var='time'>20:07</var> UTC Investigating - We are currently observing renewed elevated 5XX error rates across multiple Konnect services in ME region. Our engineering team is actively investigating the issue and assessing customer impact. We will provide additional updates as soon as more information becomes available. Mar <var data-var='date'> 1</var>, <var data-var='time'>18:41</var> UTC Monitoring - AWS continues to experience issues some services. However most of Konnect services are now recovered and operating as expected. Our engineers are closely monitoring the services and it's operational efficiency. Mar <var data-var='date'> 1</var>, <var data-var='time'>14:45</var> UTC Identified - The increased error rates are associated with the AWS me-central-1 outage. Our engineers are monitoring the situation closely. Mar <var data-var='date'> 1</var>, <var data-var='time'>13:08</var> UTC Investigating - We are seeing increased 5xx in the ME Region. Our engineers are currently investigating the issue.

Major outage in Konnect Middle East region due to loss of mutliple availability zones

Jun <var data-var='date'>17</var>, <var data-var='time'>13:21</var> UTC Resolved - This incident has been resolved. The Konnect UI has recovered, and users should no longer experience application errors when accessing product areas in Konnect. The issue was only affecting a very small set of Konnect users. We will continue to monitor the service to ensure ongoing stability. Jun <var data-var='date'>16</var>, <var data-var='time'>21:56</var> UTC Monitoring - A fix has been implemented and our team is actively monitoring the state. Jun <var data-var='date'>16</var>, <var data-var='time'>18:43</var> UTC Update - We are continuing to investigate this issue. Jun <var data-var='date'>16</var>, <var data-var='time'>18:43</var> UTC Investigating - We are investigating reports of a small subset of users encountering application errors when accessing Konnect UI. Our team is actively investigating the issue and will provide updates as more information becomes available.

Konnect UI experiencing application error state

Jun <var data-var='date'>15</var>, <var data-var='time'>06:38</var> UTC Resolved - Customers were unable to access the Konnect UI. Upon loading the interface, an error message reading "Could not fetch user session data - Please reload the page to try again" was displayed, preventing access to the platform. This issue affected all regions globally. Control plane configurations and data plane payloads using direct API access were not impacted during this time.

Konnect UI Degradation

Jun <var data-var='date'>12</var>, <var data-var='time'>16:00</var> UTC Completed - The scheduled maintenance has been completed. Jun <var data-var='date'>12</var>, <var data-var='time'>15:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jun <var data-var='date'> 4</var>, <var data-var='time'>15:44</var> UTC Scheduled - We will be performing scheduled maintenance to upgrade the Konnect database infrastructure in Australia region. This update will improve the overall reliability and performance of the platform. Expected Impact: - Maintenance will occur in AU experiencing a brief (up to 2 minutes) control plane downtime. - During this time, configuration or policy updates in Konnect may not apply until the maintenance completes in that region. - Gateway data plane traffic will not be affected and will continue to process requests normally. We appreciate your understanding as we continue to enhance the stability and resilience of the Konnect platform.

Konnect Database Infrastructure Maintenance [AU]

Jun <var data-var='date'>11</var>, <var data-var='time'>15:00</var> UTC Completed - The scheduled maintenance has been completed. Jun <var data-var='date'>11</var>, <var data-var='time'>14:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jun <var data-var='date'> 9</var>, <var data-var='time'>20:08</var> UTC Scheduled - We will be performing planned network infrastructure enhancements in the US region during this maintenance window. We do not anticipate customer impact, but brief transient connectivity disruptions may occur while changes are being applied.

Scheduled Infrastructure Maintenance - US Region

Jun <var data-var='date'> 9</var>, <var data-var='time'>19:34</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'> 9</var>, <var data-var='time'>19:25</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jun <var data-var='date'> 9</var>, <var data-var='time'>19:15</var> UTC Identified - The issue has been identified and a fix is being implemented. Jun <var data-var='date'> 9</var>, <var data-var='time'>19:08</var> UTC Investigating - Investigating incorrectly filtered results from the Developers list API

Investigating incorrectly filtered results from Developers list API

May <var data-var='date'>26</var>, <var data-var='time'>17:12</var> UTC Resolved - The incident has been resolved. The cause is still being investigated and will be posted when available. May <var data-var='date'>26</var>, <var data-var='time'>15:41</var> UTC Update - We are continuing to monitor for any further issues. May <var data-var='date'>26</var>, <var data-var='time'>15:41</var> UTC Monitoring - From 14:00UTC to 15:38UTC some customers were unable to update custom plugins that required validation from the Konnect control plane. This would include schema and handler code validation. The konnect team has reloaded the validation service which allowed validations to continue again. The team is investigating the cause.

Custom Plugin Validation was failing in US region

May <var data-var='date'>20</var>, <var data-var='time'>14:25</var> UTC Resolved - This incident has been resolved. May <var data-var='date'>20</var>, <var data-var='time'>14:24</var> UTC Update - We are continuing to monitor for any further issues. May <var data-var='date'>20</var>, <var data-var='time'>14:23</var> UTC Update - A fix has been implemented and we are monitoring the results. May <var data-var='date'>20</var>, <var data-var='time'>14:23</var> UTC Monitoring - The cause has been identified. The networking load balancer was unable to reach the gateway. Dev portals are now reachable. May <var data-var='date'>20</var>, <var data-var='time'>14:04</var> UTC Investigating - Developers portals in the SG regions are not reachable. Existing app-registration and dataplane traffic continue to work. The team is investigating the cause.

Dev Portals unreachable in SG Region

May <var data-var='date'>19</var>, <var data-var='time'>21:55</var> UTC Resolved - The control plane in US continues to remain operational. The incident has been resolved. May <var data-var='date'>19</var>, <var data-var='time'>21:28</var> UTC Update - We are continuing to monitor for any further issues. May <var data-var='date'>19</var>, <var data-var='time'>21:28</var> UTC Monitoring - The Konnect control plane in the US region has recovered. The team is continuing to monitor. May <var data-var='date'>19</var>, <var data-var='time'>21:22</var> UTC Investigating - Konnect Control Plane is unavailable in the US East region. Customer dataplanes with valid configurations will remain up, but new dataplanes will not be able to pull configurations. The team is investigating.

Konnect Control Plane down in US Region

May <var data-var='date'>11</var>, <var data-var='time'>03:19</var> UTC Resolved - The issue affecting Analytics in the AU (ap-southeast-2) region has been resolved. The problem was caused by an incorrect node configuration, which has now been corrected and deployed. All services have recovered, and analytics data is returning as expected. No other regions were impacted. May <var data-var='date'>11</var>, <var data-var='time'>03:11</var> UTC Update - We are seeing analytics services in AU region recovering. May <var data-var='date'>11</var>, <var data-var='time'>03:09</var> UTC Monitoring - A fix has been deployed to the affected AU region. Analytics services are recovering, and we are actively monitoring the system to ensure stability. May <var data-var='date'>11</var>, <var data-var='time'>03:06</var> UTC Investigating - Analytics in the AU region is currently experiencing issues. No other regions are impacted at this time. Our engineering team is actively investigating the problem.

Analytics in AU region is experiencing issues

May <var data-var='date'> 8</var>, <var data-var='time'>20:00</var> UTC Resolved - Kong proactively deployed an updated license to control planes ahead of an upcoming expiration to avoid any customer disruption. As a side effect of this update, data plane nodes were incorrectly displayed as out-of-sync in the Konnect UI. The data planes themselves continued operating normally with a valid configuration throughout. The team performed a progressive rollout that triggered a full configuration sync, which cleared the inaccurate status. All nodes returned to an in-sync state shortly after. No traffic was impacted at any point. Data planes continued routing requests normally and would receive configuration updates. The only customer-visible symptom was the inaccurate out-of-sync indicator in the UI. The full configuration sync step has been added to the standard runbook for license updates, so future license rotations will not produce this false positive. Engineering is also improving the out-of-sync detection logic to reduce false positives going forward.

Some dataplanes incorrectly showing 'out of sync' in Konnect UI

Apr <var data-var='date'>10</var>, <var data-var='time'>21:11</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>10</var>, <var data-var='time'>19:00</var> UTC Monitoring - We have rolled out the fix and will continue monitoring the situation. The system is now functioning as expected, but we are keeping a close eye on performance to ensure stability. We will provide another update if any issues arise. Apr <var data-var='date'>10</var>, <var data-var='time'>18:41</var> UTC Update - We have a fix and are rolling the change out. We will begin monitoring the situation closely and will provide another update once the rollout is complete. Apr <var data-var='date'>10</var>, <var data-var='time'>18:21</var> UTC Identified - We have identified the issue and are working on a fix Apr <var data-var='date'>10</var>, <var data-var='time'>18:07</var> UTC Investigating - Some analytics data in the US may be experiencing data loss. We are currently investigating the cause of this issue and will provide updates as they become available.

Loss of analytics payload from data plane node (US Only)

Apr <var data-var='date'> 9</var>, <var data-var='time'>18:06</var> UTC Resolved - Customers running 3.13 and below who use the OIDC plugin with `tls_client_auth_ssl_verify` unset would have seen this value change to `false` if they updated the config after the rollback of new defaults following the 3.14 release. The rollback incorrectly flipped the oidc plugin tls_client_auth_ssl_verify to false as a default, which was not one of the items recently switched to default true and has instead been defaulted true for some time. We have rolled out a fix to prod to change this default back to true. Updates to the OIDC plugin should once again keep this value set to true if not specifically defined.

OIDC tls_client_auth_ssl_verify field defaulted to `false` for new updates not explicitly setting the property

Apr <var data-var='date'> 8</var>, <var data-var='time'>21:33</var> UTC Resolved - We have completed the rollback to the original default values. Customers applying their configurations without explicitly defining ssl_verify and hide_credentials will default to `false` again. Apr <var data-var='date'> 8</var>, <var data-var='time'>16:20</var> UTC Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'> 8</var>, <var data-var='time'>16:20</var> UTC Investigating - With the release of 3.14 and changes to default security settings for Kong’s secure by default initiatives, Konnect customers running dataplanes less than 3.14 and updating certain plugins without providing overrides to the new defaults began experiencing the following issues: Konnect would begin reporting that a default had been overridden that did not apply to the connected dataplane. This is a warning that Konnect gives when the configuration on Konnect control plane appears to have user-defined changes that do not apply to the dataplane version the customer is using. This message is provided to avoid a user configuring properties on a plugin that their dataplane would not utilize, to make it clear to users why a new field isn’t taking effect.  Since our defaults changed, this caused the reporting in some cases to see this as an ‘override’ if the configuration didn’t match the new default, causing the message.  This had no impact on dataplane configurations or behavior, but it was a confusing message, and we have removed it. The second and more impactful issue is the updating of default values in 3.14. After the 3.14 release, some fields like ssl_verify and hide_credentials in various entities started defaulting to true instead of false . This is causing customers who run a deck sync without these fields defined, will see their config values change from false to true which is an issue. Konnect is working on rolling back to the old default values. Once the default values are restored on the API, the next time the config is updated without the default values, the previous values will be applied. Plugins using ssl_verify: ace acme ai-aws-guardrail ai-azure-content-safety ai-llm-as-judge ai-proxy-advanced ai-rag-injector ai-rate-limiting-advanced ai-request-transformer ai-response-transformer ai-semantic-cache ai-semantic-prompt-guard ai-semantic-response-guard aws-lambda azure-functions basic-auth confluent confluent-consume datakit forward-proxy graphql-proxy-cache-advanced graphql-rate-limiting-advanced header-cert-auth http-log jwt-signer kafka-consume kafka-log kafka-upstream ldap-auth ldap-auth-advanced mtls-auth opa openid-connect proxy-cache-advanced rate-limiting rate-limiting-advanced request-callout response-ratelimiting saml service-protection tcp-log upstream-oauth Plugins using hide_credentials: Key-auth Key-auth-enc Basic-auth Hmac-authldap-auth Oauth2 Oauth2-introspection vault-auth (EE) ldap-auth-advanced (EE)

Konnect Control Plane Default Changes due to 3.14 Release and Secure by Default

Apr <var data-var='date'> 2</var>, <var data-var='time'>19:00</var> UTC Resolved - On 4/2/2026. Konnect introduced a change as part of our 'secure by default' initiatives for the upcoming 3.14 release, which caused expected_hash syncing issues for some customers' data planes. For background, when a data plane connects to the Konnect control plane, the configuration is produced for that specific version of the Kong gateway. A configuration hash is calculated based off of the produced config, and stored as the expected config hash. When configuration changes occur, the control plane calculates the new configuration, broadcasts it to the connected data planes, and updates to the new expected hash. The issue here was the preparation of the control plane for supporting SHA256 to store credential hashes. When this change was introduced, it resulted in a minor structural difference in the produced configuration, but no actual change to the configurations occurred. This led to the same configuration yielding a different hash before and after the deploy. The way this could get a data plane to report 'out of sync' was the following: - A data plane is connected and in sync with DP hash and Expected Config Hash `a111111a` - The data plane is disconnected/reconnected, and since it is still in sync with DP hash and Expected Config Hash `a111111a`, a new config is not calculated - The change to embed SHA256 support in the control plane is deployed - A new data plane connects with no config loaded, and reports an empty config to the Control Plane - The control plane generates the slightly altered config, and ends up with a new config hash, which updates the expected config hash to `b22222b` - Since no configuration change occurred, the data planes already connected and on `1aaaaaa1` would not have been pushed a change event, and would still be running the same configuration as `b222222b` but the hash would not match, causing the DPs to appear out of sync Note: Once *any* change to the control plane configuration occurred, a new config event would have been delivered, and all previous data planes would have been updated to the hash based on the new structure. Additionally, if an impacted data plane were to disconnect and reconnect, they also would have been brought into sync with the new hash. The issue was the system failing to consider the new hash as an 'event' since it did not change configured proxy behavior. To address this in the short term, the Konnect engineering team has ensured structural changes like this which result in a config hash will trigger a configuration change event, even when no config has changed. To address this in the long term, Konnect is moving to a config version tracking methodology that will not be impacted by structural, non-behavioral changes as the hash is.

Is Kong down?

Kong status is UP

Lastest published events

Scheduled Security Maintenance – All Regions

Analytics queries may be unavailable or returning incomplete data in EU.

Elevated error rates from control plane APIs in US Region

Kong Services, Components and Regions

Recent Kong events

Scheduled Security Maintenance – All Regions

Analytics queries may be unavailable or returning incomplete data in EU.

Elevated error rates from control plane APIs in US Region

Some analytics ingestion is lagging in US prod

Performing Infrastructure Maintenance - US Region

Instability across Konnect services in EU