Aug <var data-var='date'>21</var>, <var data-var='time'>08:59</var> PDT Resolved - This incident has been resolved. Aug <var data-var='date'>21</var>, <var data-var='time'>08:25</var> PDT Monitoring - We have deployed the fix and are seeing recovery of normal Canvas functionality. Aug <var data-var='date'>21</var>, <var data-var='time'>08:21</var> PDT Identified - Canvas is not loading for customers in the US and EU regions and we have identified what we believe to be the cause and are rolling out a fix.

Canvas not loading

Aug <var data-var='date'>10</var>, <var data-var='time'>13:35</var> PDT Resolved - The fix is in place and we've confirmed resolution. Queries and SLOs are processing normally. Aug <var data-var='date'>10</var>, <var data-var='time'>06:01</var> PDT Monitoring - From 11:36UTC to 12:43UTC, queries in US1 that read older data returned errors or timed out. Queries over recent data were un-affected, and no telemetry is being lost. The issue has been identified, and a fix is in place. We are currently monitoring the situation to confirm resolution.

US1 cold query outage

Jul <var data-var='date'>31</var>, <var data-var='time'>07:30</var> PDT Resolved - Between 10:30 and 11:00 AM EDT, we experienced elevated querying failures and slowness. The impact has subsided, and we are currently investigating and monitoring the situation.

Querying issues

Jul <var data-var='date'>28</var>, <var data-var='time'>11:30</var> PDT Resolved - The degradation in querying and triggers has been resolved and we're back to full functionality. Jul <var data-var='date'>28</var>, <var data-var='time'>11:21</var> PDT Update - We are continuing to monitor for any further issues. Jul <var data-var='date'>27</var>, <var data-var='time'>15:51</var> PDT Update - The partial degradation has been resolved for all customers except for those specifically contacted. We are working on mitigation measures so that we can fully restore trigger functionality for everyone. Jul <var data-var='date'>27</var>, <var data-var='time'>12:46</var> PDT Update - We are seeing a regression in performance with partial degradation of both triggers and querying. We are working to remediate. Jul <var data-var='date'>27</var>, <var data-var='time'>09:01</var> PDT Monitoring - We have identified the source of the issue, applied a temporary fix, and are working on a permanent fix. We continue to monitor the situation. Jul <var data-var='date'>27</var>, <var data-var='time'>06:52</var> PDT Update - We are continuing to see issues with triggers and are also noticing issues with querying. We are investigating for the cause of both. Jul <var data-var='date'>27</var>, <var data-var='time'>06:13</var> PDT Investigating - We are seeing triggers intermittently failing in the EU region. We are actively investigating the cause.

Triggers intermittently failing in EU region

Jul <var data-var='date'> 2</var>, <var data-var='time'>17:31</var> PDT Resolved - This incident has been resolved. Jul <var data-var='date'> 2</var>, <var data-var='time'>11:13</var> PDT Monitoring - Clearing the CDN cache appears to have solved the issue. Jul <var data-var='date'> 2</var>, <var data-var='time'>11:10</var> PDT Investigating - We are currently investigating this issue.

honeycomb.io marketing website not working

Jun <var data-var='date'>29</var>, <var data-var='time'>17:47</var> PDT Resolved - Backfill of the Activity Log events has completed. All events during the affected period should be present. Jun <var data-var='date'>29</var>, <var data-var='time'>15:07</var> PDT Monitoring - Starting at 14:24 PT, there was an issue with the Activity Log causing events to stop being processed. You may notice a gap starting at that time. We are slowly backfilling these events and do not expect any data loss to occur. We expect the full data log to be restored around 17:30 PT.

Activity Log delayed in US

Jun <var data-var='date'>25</var>, <var data-var='time'>13:41</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'>25</var>, <var data-var='time'>11:31</var> PDT Monitoring - We are monitoring the situation. Events may have been delayed between 17:53 and 18:04 UTC. Jun <var data-var='date'>25</var>, <var data-var='time'>11:10</var> PDT Investigating - We are investigating an issue with delayed ingest in the EU region. SLOs and Trigger evaluations may also be delayed.

Ingest outage in EU

Jun <var data-var='date'>24</var>, <var data-var='time'>12:04</var> PDT Resolved - All services have been fully recovered. Impact window: 17:55–18:17 UTC Regions affected: Primary impact in US. The EU region was affected to a much lesser degree. During this period, the following effects may have occurred: Ingest: Some data was not ingested, resulting in complete or partial data loss for events sent during the window. Triggers: Triggers may have failed to fire within the impact window. SLOs: SLI values across the affected window are skewed by the missing data and may show artificial dips or accelerated budget burn. Service Maps: Maps covering the outage window are incomplete. Services and dependencies may be under-counted, as their traces were not ingested. We apologize for the disruption. Please reach out if you have any questions about how this may have affected your data. Jun <var data-var='date'>24</var>, <var data-var='time'>11:31</var> PDT Update - Triggers, SLOs and Service Maps are recovered. We are continuing to monitoring the Ingest Service Jun <var data-var='date'>24</var>, <var data-var='time'>11:22</var> PDT Monitoring - We have rolled back the deploy and ingest service has recovered. There has been an ingest outage from 17:55 - 18:14 UTC. We have also observed an delay for Service Maps. Jun <var data-var='date'>24</var>, <var data-var='time'>11:14</var> PDT Investigating - We are investigating an issue with delayed ingest in the US and EU region. Our engineers are rolling back a recent deploy. SLOs and Trigger evaluations may also be delayed.

Ingest Outage in US and EU

Jun <var data-var='date'>19</var>, <var data-var='time'>10:21</var> PDT Resolved - MCP Write Tools have been restored. MCP users may need to reconnect to see all available tools. Jun <var data-var='date'>19</var>, <var data-var='time'>10:18</var> PDT Monitoring - We are aware of an issue affecting Honeycomb MCP. Write tool functionality may be degraded. Read tools are unaffected. Jun <var data-var='date'>19</var>, <var data-var='time'>09:39</var> PDT Investigating - We are aware of an issue affecting Honeycomb MCP. Write tool functionality may be degraded. Read tools are unaffected.

MCP tool access degraded

Jun <var data-var='date'>18</var>, <var data-var='time'>12:10</var> PDT Resolved - Beginning June 17th, enhance-related requests were failing to complete. We've since identified and deployed a fix, and enhance is now fully operational. Thank you for your patience.

Enhance Issues

Jun <var data-var='date'>17</var>, <var data-var='time'>14:14</var> PDT Resolved - We have identified the root cause of the intermittent slowness and query failures affecting our Production EU Region. The issue was triggered by a large volume of dataset deletions that caused elevated database load, resulting in query instability. No data was lost during this incident. Service has been restored. A fix has been applied to help prevent recurrence. Jun <var data-var='date'>17</var>, <var data-var='time'>08:29</var> PDT Monitoring - Querying in our Production EU Region is fully operational. We will continue to monitor for any abnormal behavior. Jun <var data-var='date'>17</var>, <var data-var='time'>07:01</var> PDT Investigating - We are continuing to investigate intermittent slowness and query failures affecting our Production EU Region. We will provide an update as soon as we have more information.

Querying Issues

Jun <var data-var='date'>17</var>, <var data-var='time'>05:32</var> PDT Resolved - The issue is now resolved and querying is back to fully functional. Jun <var data-var='date'>17</var>, <var data-var='time'>05:01</var> PDT Investigating - We are currently investigating the cause of slowness and query failures in our Production EU Region.

Querying Issues in EU

Jun <var data-var='date'>11</var>, <var data-var='time'>08:30</var> PDT Resolved - Requests to Honeycomb's Query Data and Management APIs in the production-eu1 environment encountered increased error rates between approximately 15:35 and 22:24 UTC. Event ingestion was not affected. The issue has been resolved.

Elevated API errors in production-eu1

Jun <var data-var='date'>10</var>, <var data-var='time'>15:02</var> PDT Resolved - All services are now healthy. Jun <var data-var='date'>10</var>, <var data-var='time'>14:29</var> PDT Identified - The issue has been identified and a fix is being implemented. Jun <var data-var='date'>10</var>, <var data-var='time'>14:28</var> PDT Update - Queries should no longer be delayed. SLOs and Trigger evaluations remain impacted. Jun <var data-var='date'>10</var>, <var data-var='time'>13:58</var> PDT Investigating - We are investigating an issue with delayed ingest in the EU region. Received events are still being stored, but there may be a delay in event retrieval in queries. SLOs and Trigger evaluations may also be delayed.

Delayed ingest in EU

Jun <var data-var='date'> 8</var>, <var data-var='time'>10:41</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'> 8</var>, <var data-var='time'>09:37</var> PDT Investigating - We are currently investigating this issue.

Canvas UI issues

Jun <var data-var='date'> 3</var>, <var data-var='time'>13:51</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'> 3</var>, <var data-var='time'>13:50</var> PDT Identified - Canvas agent cannot load honeycomb internal tools.

Canvas tools unavailable

May <var data-var='date'>22</var>, <var data-var='time'>09:42</var> PDT Resolved - This is an administrative update, marking the incident as Resolved. There has been no further impact since 23:20UTC on May 21 2026. May <var data-var='date'>21</var>, <var data-var='time'>17:49</var> PDT Update - Impact from this issue has concluded. For a period of time (different between US and EU instances, noted below), queries spanning data older than the most recent 2 hours with certain GROUP / WHERE clauses returned inconsistent results. Data was never lost, and reruns of the same queries will show the previously missing data. US impact times: 21:50 UTC - 23:20 UTC EU impact times: 20:50 UTC - 23:20 UTC May <var data-var='date'>21</var>, <var data-var='time'>16:22</var> PDT Investigating - We have identified and are working to resolve an issue that is causing query results to return inconsistent results.

Investigating querying issues

May <var data-var='date'>11</var>, <var data-var='time'>09:37</var> PDT Resolved - This incident has been resolved. May <var data-var='date'> 8</var>, <var data-var='time'>13:46</var> PDT Update - Activity Log recovered at 07:00 UTC-07:00 today (about 6.5h ago).  All streams are caught up. May <var data-var='date'> 8</var>, <var data-var='time'>03:40</var> PDT Update - All activity log streams are now caught up, except for the query runs table, which should have all data since the start of the incident backfilled by 1600 PDT. May <var data-var='date'> 8</var>, <var data-var='time'>03:22</var> PDT Update - We are continuing to monitor for any further issues. May <var data-var='date'> 8</var>, <var data-var='time'>03:21</var> PDT Update - We believe that the activity log should now be caught up. May <var data-var='date'> 8</var>, <var data-var='time'>03:13</var> PDT Monitoring - Backfilling of the past 2h30m of data is now in progress and should complete shortly. May <var data-var='date'> 8</var>, <var data-var='time'>01:00</var> PDT Identified - Activity log delay is rising, and is more than an hour behind due to a MySQL replica failover. Estimated recovery by 04:00 PDT.

activity log for US instance is delayed

May <var data-var='date'> 7</var>, <var data-var='time'>15:44</var> PDT Completed - Scheduled maintenance is complete. All services are healthy and stable. Ingest, querying SLO processing, anomaly detection, Service Maps, and Activity Log are all confirmed operating normally. Backfill and SLO repair are complete. Thank you for your patience. May <var data-var='date'> 7</var>, <var data-var='time'>15:27</var> PDT Update - We are now in our final verification stage. Query results should begin returning fresh data now. May <var data-var='date'> 7</var>, <var data-var='time'>15:13</var> PDT Verifying - All consumer services have been successfully cut over. SLO backfill is currently running. Anomaly detection, Service Maps, and Activity Log processing are all on the new cluster and catching up. All data will be fully present within approximately 2 hours of the maintenance window closing. May <var data-var='date'> 7</var>, <var data-var='time'>13:53</var> PDT Update - Scheduled maintenance is still in progress. We will provide updates as necessary. May <var data-var='date'> 7</var>, <var data-var='time'>13:10</var> PDT In progress - We have begun scheduled maintenance on our Kafka infrastructure in Production US. During this window, event ingestion and query results may experience delays. Apr <var data-var='date'>28</var>, <var data-var='time'>11:49</var> PDT Scheduled - We will be performing scheduled maintenance on our Kafka infrastructure in the US region. This maintenance will only impact the US cluster. During this window, you may experience the following: Ingest will remain uninterrupted - events will continue to be accepted, although their processing will be delayed. You may notice a gap in your data during the migration window; this data will be backfilled after the migration completes. Expect all data to be present within approximately 3 hours of the maintenance window closing. Query results will reflect a delay in fresh data during the cutover window, in line with the ingestion delays. Triggers and alerts: Triggers watching for missing events (e.g. heartbeats) will fire during this window. Triggers watching for values above a threshold are not expected to fire. Other alerting sources will process delayed events late, up to approximately 3 hours. SLO burn alerts will also fire late during this window. Activity Log and Service Maps will experience delays in processing events. We appreciate your patience and will provide updates as the maintenance progresses

Scheduled Maintenance - US Region

May <var data-var='date'> 4</var>, <var data-var='time'>13:00</var> PDT Completed - Scheduled maintenance is complete. All services are healthy and stable. Ingest, Querying, SLO processing, Anomaly Detection, Service Maps, Triggers and Activity Log are all confirmed operating normally. Backfill and SLA repair are complete. Thank you for your patience. May <var data-var='date'> 4</var>, <var data-var='time'>12:08</var> PDT Verifying - We are in our final verification phase. Query results, SLO calculations, anomaly detection, and Service Maps are all confirmed processing as expected. May <var data-var='date'> 4</var>, <var data-var='time'>11:14</var> PDT Update - Query results should begin returning fresh data shortly. SLO processing and Service Maps migration are still in progress. May <var data-var='date'> 4</var>, <var data-var='time'>10:28</var> PDT Update - We have begun scheduled maintenance on our Kafka infrastructure in Production EU. May <var data-var='date'> 4</var>, <var data-var='time'>10:00</var> PDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>27</var>, <var data-var='time'>13:58</var> PDT Scheduled - We will be performing scheduled maintenance on our Kafka infrastructure in the EU region. This maintenance will only impact the EU cluster. During this window, you may experience the following: * Ingest will remain uninterrupted - events will continue to be accepted, although their processing will be delayed. You may notice a gap in your data during the migration window; this data will be backfilled after the migration completes. Expect all data to be present within approximately 3 hours of the maintenance window closing. * Query results will reflect a delay in fresh data during the cutover window, in line with the ingestion delays. * Triggers and alerts: Triggers watching for missing events (e.g. heartbeats) will fire during this window. Triggers watching for values above a threshold are not expected to fire. Other alerting sources will process delayed events late, up to approximately 3 hours. * SLO burn alerts will also fire late during this window. * Activity Log and Service Maps will experience delays in processing events. We appreciate your patience and will provide updates as the maintenance progresses

Scheduled Maintenance – EU Region

May <var data-var='date'> 1</var>, <var data-var='time'>06:00</var> PDT Resolved - The database failover that caused query issues earlier in the EU (https://status.honeycomb.io/incidents/n855d8kzp32y) has also had knock-on effects on our Activity Log beta feature. Due to recovery issues on the data replication flow used for this mechanism, activity log events will be missing from 12:45 UTC until 19:45 UTC, representing a 5 hour gap in events. We have identified configuration parameters that will be adjusted to reduce the likelihood of such losses in the future.

Gap in Activity Log data in the EU region.

May <var data-var='date'> 1</var>, <var data-var='time'>05:30</var> PDT Resolved - At 12:50, our main database underwent an automated failover. This failover led to issues with some of our query engine connection pools, and caused queries to fail for roughly 10 minutes before self resolving.

Query interuption due to database failover in the EU region

Apr <var data-var='date'>30</var>, <var data-var='time'>09:37</var> PDT Resolved - We are confident the issue is fully resolved. Apr <var data-var='date'>30</var>, <var data-var='time'>08:07</var> PDT Monitoring - The fix has been implemented and links on Slack triggers are functional again. Apr <var data-var='date'>30</var>, <var data-var='time'>08:03</var> PDT Identified - We discovered that links in Slack triggers are not working and have identified the cause and are in process of implementing a fix

Links in Slack triggers not working

Apr <var data-var='date'>20</var>, <var data-var='time'>10:02</var> PDT Resolved - Activity Log is back to operational, and most events should be recovered. Apr <var data-var='date'>20</var>, <var data-var='time'>09:56</var> PDT Identified - Activity Log events are interrupted, starting from around 15:51UTC. Customers may experience some data lost since the beginning of the incident.

Activity Log events interrupted for EU

Apr <var data-var='date'>15</var>, <var data-var='time'>12:25</var> PDT Resolved - Activity Log is ingesting events again. We had no loss in data. Apr <var data-var='date'>15</var>, <var data-var='time'>12:04</var> PDT Identified - Activity Log is experiencing a delay in processing events. We are working on remediating the issue.

Is Honeycomb.io down?

Honeycomb.io status is UP

Lastest published events

Canvas not loading

US1 cold query outage

Querying issues

Honeycomb.io Services, Components and Regions

Recent Honeycomb.io events

Canvas not loading

US1 cold query outage

Querying issues

Triggers intermittently failing in EU region

honeycomb.io marketing website not working

Activity Log delayed in US