Jul <var data-var='date'> 2</var>, <var data-var='time'>17:31</var> PDT Resolved - This incident has been resolved. Jul <var data-var='date'> 2</var>, <var data-var='time'>11:13</var> PDT Monitoring - Clearing the CDN cache appears to have solved the issue. Jul <var data-var='date'> 2</var>, <var data-var='time'>11:10</var> PDT Investigating - We are currently investigating this issue.

honeycomb.io marketing website not working

Jun <var data-var='date'>29</var>, <var data-var='time'>17:47</var> PDT Resolved - Backfill of the Activity Log events has completed. All events during the affected period should be present. Jun <var data-var='date'>29</var>, <var data-var='time'>15:07</var> PDT Monitoring - Starting at 14:24 PT, there was an issue with the Activity Log causing events to stop being processed. You may notice a gap starting at that time. We are slowly backfilling these events and do not expect any data loss to occur. We expect the full data log to be restored around 17:30 PT.

Activity Log delayed in US

Jun <var data-var='date'>25</var>, <var data-var='time'>13:41</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'>25</var>, <var data-var='time'>11:31</var> PDT Monitoring - We are monitoring the situation. Events may have been delayed between 17:53 and 18:04 UTC. Jun <var data-var='date'>25</var>, <var data-var='time'>11:10</var> PDT Investigating - We are investigating an issue with delayed ingest in the EU region. SLOs and Trigger evaluations may also be delayed.

Ingest outage in EU

Jun <var data-var='date'>24</var>, <var data-var='time'>12:04</var> PDT Resolved - All services have been fully recovered. Impact window: 17:55–18:17 UTC Regions affected: Primary impact in US. The EU region was affected to a much lesser degree. During this period, the following effects may have occurred: Ingest: Some data was not ingested, resulting in complete or partial data loss for events sent during the window. Triggers: Triggers may have failed to fire within the impact window. SLOs: SLI values across the affected window are skewed by the missing data and may show artificial dips or accelerated budget burn. Service Maps: Maps covering the outage window are incomplete. Services and dependencies may be under-counted, as their traces were not ingested. We apologize for the disruption. Please reach out if you have any questions about how this may have affected your data. Jun <var data-var='date'>24</var>, <var data-var='time'>11:31</var> PDT Update - Triggers, SLOs and Service Maps are recovered. We are continuing to monitoring the Ingest Service Jun <var data-var='date'>24</var>, <var data-var='time'>11:22</var> PDT Monitoring - We have rolled back the deploy and ingest service has recovered. There has been an ingest outage from 17:55 - 18:14 UTC. We have also observed an delay for Service Maps. Jun <var data-var='date'>24</var>, <var data-var='time'>11:14</var> PDT Investigating - We are investigating an issue with delayed ingest in the US and EU region. Our engineers are rolling back a recent deploy. SLOs and Trigger evaluations may also be delayed.

Ingest Outage in US and EU

Jun <var data-var='date'>19</var>, <var data-var='time'>10:21</var> PDT Resolved - MCP Write Tools have been restored. MCP users may need to reconnect to see all available tools. Jun <var data-var='date'>19</var>, <var data-var='time'>10:18</var> PDT Monitoring - We are aware of an issue affecting Honeycomb MCP. Write tool functionality may be degraded. Read tools are unaffected. Jun <var data-var='date'>19</var>, <var data-var='time'>09:39</var> PDT Investigating - We are aware of an issue affecting Honeycomb MCP. Write tool functionality may be degraded. Read tools are unaffected.

MCP tool access degraded

Jun <var data-var='date'>18</var>, <var data-var='time'>12:10</var> PDT Resolved - Beginning June 17th, enhance-related requests were failing to complete. We've since identified and deployed a fix, and enhance is now fully operational. Thank you for your patience.

Enhance Issues

Jun <var data-var='date'>17</var>, <var data-var='time'>14:14</var> PDT Resolved - We have identified the root cause of the intermittent slowness and query failures affecting our Production EU Region. The issue was triggered by a large volume of dataset deletions that caused elevated database load, resulting in query instability. No data was lost during this incident. Service has been restored. A fix has been applied to help prevent recurrence. Jun <var data-var='date'>17</var>, <var data-var='time'>08:29</var> PDT Monitoring - Querying in our Production EU Region is fully operational. We will continue to monitor for any abnormal behavior. Jun <var data-var='date'>17</var>, <var data-var='time'>07:01</var> PDT Investigating - We are continuing to investigate intermittent slowness and query failures affecting our Production EU Region. We will provide an update as soon as we have more information.

Querying Issues

Jun <var data-var='date'>17</var>, <var data-var='time'>05:32</var> PDT Resolved - The issue is now resolved and querying is back to fully functional. Jun <var data-var='date'>17</var>, <var data-var='time'>05:01</var> PDT Investigating - We are currently investigating the cause of slowness and query failures in our Production EU Region.

Querying Issues in EU

Jun <var data-var='date'>11</var>, <var data-var='time'>08:30</var> PDT Resolved - Requests to Honeycomb's Query Data and Management APIs in the production-eu1 environment encountered increased error rates between approximately 15:35 and 22:24 UTC. Event ingestion was not affected. The issue has been resolved.

Elevated API errors in production-eu1

Jun <var data-var='date'>10</var>, <var data-var='time'>15:02</var> PDT Resolved - All services are now healthy. Jun <var data-var='date'>10</var>, <var data-var='time'>14:29</var> PDT Identified - The issue has been identified and a fix is being implemented. Jun <var data-var='date'>10</var>, <var data-var='time'>14:28</var> PDT Update - Queries should no longer be delayed. SLOs and Trigger evaluations remain impacted. Jun <var data-var='date'>10</var>, <var data-var='time'>13:58</var> PDT Investigating - We are investigating an issue with delayed ingest in the EU region. Received events are still being stored, but there may be a delay in event retrieval in queries. SLOs and Trigger evaluations may also be delayed.

Delayed ingest in EU

Jun <var data-var='date'> 8</var>, <var data-var='time'>10:41</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'> 8</var>, <var data-var='time'>09:37</var> PDT Investigating - We are currently investigating this issue.

Canvas UI issues

Jun <var data-var='date'> 3</var>, <var data-var='time'>13:51</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'> 3</var>, <var data-var='time'>13:50</var> PDT Identified - Canvas agent cannot load honeycomb internal tools.

Canvas tools unavailable

May <var data-var='date'>22</var>, <var data-var='time'>09:42</var> PDT Resolved - This is an administrative update, marking the incident as Resolved. There has been no further impact since 23:20UTC on May 21 2026. May <var data-var='date'>21</var>, <var data-var='time'>17:49</var> PDT Update - Impact from this issue has concluded. For a period of time (different between US and EU instances, noted below), queries spanning data older than the most recent 2 hours with certain GROUP / WHERE clauses returned inconsistent results. Data was never lost, and reruns of the same queries will show the previously missing data. US impact times: 21:50 UTC - 23:20 UTC EU impact times: 20:50 UTC - 23:20 UTC May <var data-var='date'>21</var>, <var data-var='time'>16:22</var> PDT Investigating - We have identified and are working to resolve an issue that is causing query results to return inconsistent results.

Investigating querying issues

May <var data-var='date'>11</var>, <var data-var='time'>09:37</var> PDT Resolved - This incident has been resolved. May <var data-var='date'> 8</var>, <var data-var='time'>13:46</var> PDT Update - Activity Log recovered at 07:00 UTC-07:00 today (about 6.5h ago).  All streams are caught up. May <var data-var='date'> 8</var>, <var data-var='time'>03:40</var> PDT Update - All activity log streams are now caught up, except for the query runs table, which should have all data since the start of the incident backfilled by 1600 PDT. May <var data-var='date'> 8</var>, <var data-var='time'>03:22</var> PDT Update - We are continuing to monitor for any further issues. May <var data-var='date'> 8</var>, <var data-var='time'>03:21</var> PDT Update - We believe that the activity log should now be caught up. May <var data-var='date'> 8</var>, <var data-var='time'>03:13</var> PDT Monitoring - Backfilling of the past 2h30m of data is now in progress and should complete shortly. May <var data-var='date'> 8</var>, <var data-var='time'>01:00</var> PDT Identified - Activity log delay is rising, and is more than an hour behind due to a MySQL replica failover. Estimated recovery by 04:00 PDT.

activity log for US instance is delayed

May <var data-var='date'> 7</var>, <var data-var='time'>15:44</var> PDT Completed - Scheduled maintenance is complete. All services are healthy and stable. Ingest, querying SLO processing, anomaly detection, Service Maps, and Activity Log are all confirmed operating normally. Backfill and SLO repair are complete. Thank you for your patience. May <var data-var='date'> 7</var>, <var data-var='time'>15:27</var> PDT Update - We are now in our final verification stage. Query results should begin returning fresh data now. May <var data-var='date'> 7</var>, <var data-var='time'>15:13</var> PDT Verifying - All consumer services have been successfully cut over. SLO backfill is currently running. Anomaly detection, Service Maps, and Activity Log processing are all on the new cluster and catching up. All data will be fully present within approximately 2 hours of the maintenance window closing. May <var data-var='date'> 7</var>, <var data-var='time'>13:53</var> PDT Update - Scheduled maintenance is still in progress. We will provide updates as necessary. May <var data-var='date'> 7</var>, <var data-var='time'>13:10</var> PDT In progress - We have begun scheduled maintenance on our Kafka infrastructure in Production US. During this window, event ingestion and query results may experience delays. Apr <var data-var='date'>28</var>, <var data-var='time'>11:49</var> PDT Scheduled - We will be performing scheduled maintenance on our Kafka infrastructure in the US region. This maintenance will only impact the US cluster. During this window, you may experience the following: Ingest will remain uninterrupted - events will continue to be accepted, although their processing will be delayed. You may notice a gap in your data during the migration window; this data will be backfilled after the migration completes. Expect all data to be present within approximately 3 hours of the maintenance window closing. Query results will reflect a delay in fresh data during the cutover window, in line with the ingestion delays. Triggers and alerts: Triggers watching for missing events (e.g. heartbeats) will fire during this window. Triggers watching for values above a threshold are not expected to fire. Other alerting sources will process delayed events late, up to approximately 3 hours. SLO burn alerts will also fire late during this window. Activity Log and Service Maps will experience delays in processing events. We appreciate your patience and will provide updates as the maintenance progresses

Scheduled Maintenance - US Region

May <var data-var='date'> 4</var>, <var data-var='time'>13:00</var> PDT Completed - Scheduled maintenance is complete. All services are healthy and stable. Ingest, Querying, SLO processing, Anomaly Detection, Service Maps, Triggers and Activity Log are all confirmed operating normally. Backfill and SLA repair are complete. Thank you for your patience. May <var data-var='date'> 4</var>, <var data-var='time'>12:08</var> PDT Verifying - We are in our final verification phase. Query results, SLO calculations, anomaly detection, and Service Maps are all confirmed processing as expected. May <var data-var='date'> 4</var>, <var data-var='time'>11:14</var> PDT Update - Query results should begin returning fresh data shortly. SLO processing and Service Maps migration are still in progress. May <var data-var='date'> 4</var>, <var data-var='time'>10:28</var> PDT Update - We have begun scheduled maintenance on our Kafka infrastructure in Production EU. May <var data-var='date'> 4</var>, <var data-var='time'>10:00</var> PDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>27</var>, <var data-var='time'>13:58</var> PDT Scheduled - We will be performing scheduled maintenance on our Kafka infrastructure in the EU region. This maintenance will only impact the EU cluster. During this window, you may experience the following: * Ingest will remain uninterrupted - events will continue to be accepted, although their processing will be delayed. You may notice a gap in your data during the migration window; this data will be backfilled after the migration completes. Expect all data to be present within approximately 3 hours of the maintenance window closing. * Query results will reflect a delay in fresh data during the cutover window, in line with the ingestion delays. * Triggers and alerts: Triggers watching for missing events (e.g. heartbeats) will fire during this window. Triggers watching for values above a threshold are not expected to fire. Other alerting sources will process delayed events late, up to approximately 3 hours. * SLO burn alerts will also fire late during this window. * Activity Log and Service Maps will experience delays in processing events. We appreciate your patience and will provide updates as the maintenance progresses

Scheduled Maintenance – EU Region

May <var data-var='date'> 1</var>, <var data-var='time'>06:00</var> PDT Resolved - The database failover that caused query issues earlier in the EU (https://status.honeycomb.io/incidents/n855d8kzp32y) has also had knock-on effects on our Activity Log beta feature. Due to recovery issues on the data replication flow used for this mechanism, activity log events will be missing from 12:45 UTC until 19:45 UTC, representing a 5 hour gap in events. We have identified configuration parameters that will be adjusted to reduce the likelihood of such losses in the future.

Gap in Activity Log data in the EU region.

May <var data-var='date'> 1</var>, <var data-var='time'>05:30</var> PDT Resolved - At 12:50, our main database underwent an automated failover. This failover led to issues with some of our query engine connection pools, and caused queries to fail for roughly 10 minutes before self resolving.

Query interuption due to database failover in the EU region

Apr <var data-var='date'>30</var>, <var data-var='time'>09:37</var> PDT Resolved - We are confident the issue is fully resolved. Apr <var data-var='date'>30</var>, <var data-var='time'>08:07</var> PDT Monitoring - The fix has been implemented and links on Slack triggers are functional again. Apr <var data-var='date'>30</var>, <var data-var='time'>08:03</var> PDT Identified - We discovered that links in Slack triggers are not working and have identified the cause and are in process of implementing a fix

Links in Slack triggers not working

Apr <var data-var='date'>20</var>, <var data-var='time'>10:02</var> PDT Resolved - Activity Log is back to operational, and most events should be recovered. Apr <var data-var='date'>20</var>, <var data-var='time'>09:56</var> PDT Identified - Activity Log events are interrupted, starting from around 15:51UTC. Customers may experience some data lost since the beginning of the incident.

Activity Log events interrupted for EU

Apr <var data-var='date'>15</var>, <var data-var='time'>12:25</var> PDT Resolved - Activity Log is ingesting events again. We had no loss in data. Apr <var data-var='date'>15</var>, <var data-var='time'>12:04</var> PDT Identified - Activity Log is experiencing a delay in processing events. We are working on remediating the issue.

Activity Log ingest is delayed

Apr <var data-var='date'>14</var>, <var data-var='time'>15:37</var> PDT Resolved - All activity log data is now live and up to date. Apr <var data-var='date'>14</var>, <var data-var='time'>15:08</var> PDT Monitoring - A fix has been implemented and we are monitoring recovery. Apr <var data-var='date'>14</var>, <var data-var='time'>14:36</var> PDT Identified - Activity Log events may be delayed, starting from around 19:55UTC. We're actively working on remediation.

Activity Log delayed ingest in EU

Apr <var data-var='date'>10</var>, <var data-var='time'>12:10</var> PDT Resolved - All activity log data is now live and up to date. Apr <var data-var='date'> 9</var>, <var data-var='time'>17:05</var> PDT Update - SLO processing has fully recovered. Apr <var data-var='date'> 9</var>, <var data-var='time'>16:43</var> PDT Update - SLOs experienced a period of additional processing delays. A fix has been deployed and recovery is ongoing. Apr <var data-var='date'> 9</var>, <var data-var='time'>14:03</var> PDT Update - We have identified an issue with triggers and SLOs and have implemented a fix. Customers should no longer see degraded performance for triggers or SLOs. A fix for the activity log delay has been implemented and is in the process of recovering. Customers may continue to see activity log unavailability while this completes. Apr <var data-var='date'> 9</var>, <var data-var='time'>13:24</var> PDT Update - We have identified an issue with triggers and SLOs and are actively investigating. Customers may experience delays when creating or updating triggers and SLOs. A fix for the activity log delay has been implemented and is in the process of recovering. Customers may continue to see activity log unavailability while this completes. Apr <var data-var='date'> 9</var>, <var data-var='time'>04:52</var> PDT Identified - We have identified the cause and are working to resolve the issue.

Activity log delay in production US

Apr <var data-var='date'>10</var>, <var data-var='time'>10:24</var> PDT Resolved - We can confirm that the fix was applied uniformly to all impacted and potentially impacted environments; no teams should see ingest errors related to this issue anymore. Apr <var data-var='date'>10</var>, <var data-var='time'>08:21</var> PDT Update - We are continuing to work on the systemic solution and monitor. We'll leave the incident open until we're able to confirm the issue won't recur. Apr <var data-var='date'>10</var>, <var data-var='time'>07:32</var> PDT Update - The manual fix has been applied to all environments. Apr <var data-var='date'>10</var>, <var data-var='time'>07:19</var> PDT Monitoring - We've applied the manual fix to nearly all affected datasets (less than 1% remaining). The systemic solution is still in progress. Apr <var data-var='date'>10</var>, <var data-var='time'>06:55</var> PDT Update - We've identified a manual fix and are deploying that by hand to each affected environment, while also investigating a systemic solution. Apr <var data-var='date'>10</var>, <var data-var='time'>06:46</var> PDT Update - We have further narrowed down that only teams on Classic environments are currently seeing issues. We are figuring out mitigation mechanisms at the moment. Apr <var data-var='date'>10</var>, <var data-var='time'>06:41</var> PDT Identified - We have confirmed reports of some teams getting issues ingesting their data. We have identified a probable source behind this behavior and are currently trying to correct it.

Ingest issues for some customers

Apr <var data-var='date'> 9</var>, <var data-var='time'>12:00</var> PDT Resolved - We have detected and remediated an issue around our SLO processing pipeline. From 17:00 to 17:15 UTC, events were being delayed, and alerts may likewise have been a few minutes late. Processing has caught up and is currently within normal operational ranges.

Is Honeycomb.io down?

Honeycomb.io status is UP

Lastest published events

honeycomb.io marketing website not working

Activity Log delayed in US

Ingest outage in EU

Honeycomb.io Services, Components and Regions

Recent Honeycomb.io events

honeycomb.io marketing website not working

Activity Log delayed in US

Ingest outage in EU

Ingest Outage in US and EU

MCP tool access degraded

Enhance Issues

Get your FREE account now,
5 monitors included!

Is Honeycomb.io down?

Honeycomb.io status is UP

Lastest published events

honeycomb.io marketing website not working

Activity Log delayed in US

Ingest outage in EU

Honeycomb.io Services, Components and Regions

Recent Honeycomb.io events

honeycomb.io marketing website not working

Activity Log delayed in US

Ingest outage in EU

Ingest Outage in US and EU

MCP tool access degraded

Enhance Issues

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!