Jul <var data-var='date'>13</var>, <var data-var='time'>20:29</var> PDT Resolved - The incident has been resolved and metrics are back online. There will be a dip in metric count graphs while the incident was experienced. Error event data was recorded normally and was unaffected. We apologize for the inconvenience. If you have any questions please reach us at support@airbrake.io. Jul <var data-var='date'>13</var>, <var data-var='time'>20:10</var> PDT Identified - We have identified the issue affecting service availability and are working through recovery steps. We are continuing to work on restoring normal operation and will share another update as soon as service is stable. Jul <var data-var='date'>13</var>, <var data-var='time'>08:15</var> PDT Investigating - Our dashboard graphs are currently not loading as expected. We are investigating the root cause and will share an update as soon as we have more information. All metrics may be affected including error counts in digest emails.

Metric graphs not loading

Jun <var data-var='date'> 1</var>, <var data-var='time'>08:53</var> PDT Resolved - We've resolved an issue with our email notifications. The issue was caused by email asset delivery which we've resolved. We apologize for any inconvenience. Please email support at support@airbrake.io if you have any questions or concerns. Jun <var data-var='date'> 1</var>, <var data-var='time'>07:22</var> PDT Investigating - Airbrake is currently experiencing an issue with email notification delivery. Errors are being processed as normal. We will update this incident as we work to resolve this issue.

Email notifications

May <var data-var='date'> 6</var>, <var data-var='time'>04:00</var> PDT Resolved - A subset of customers experienced an issue where the error dashboard list was not loading properly. This may have resulted in error filtering being faulty or the errors list being blank. The issue was escalated to our techops team which resolved the issue. We apologize for the inconvenience that this has caused. If you have any questions or concerns, please reach us at support@airbrake.io.

Error dashboard loading issues

Jul <var data-var='date'>24</var>, <var data-var='time'>10:24</var> PDT Resolved - The issue causing intermittent dashboard loading issues has been resolved. If you have any questions, please reach us at support@airbrake.io. We apologize for the inconvenience. Jul <var data-var='date'>23</var>, <var data-var='time'>18:22</var> PDT Monitoring - The error dashboard and settings pages are available again. We're continuing to monitor and investigate the issue. Jul <var data-var='date'>23</var>, <var data-var='time'>17:38</var> PDT Investigating - We are currently investigating an issue causing individual project dashboard and settings pages to be unavailable. We apologize for the inconvenience. Please email us at support@airbrake.io if you have any questions.

Dashboard unavailble

Jul <var data-var='date'>13</var>, <var data-var='time'>21:30</var> PDT Resolved - This incident has been resolved. Jul <var data-var='date'>13</var>, <var data-var='time'>19:45</var> PDT Update - The web application is back up and account API is no longer experiencing errors. We're continuing to monitor the resolution again. Please reach us at support@airbrake.io if you have any questions. Jul <var data-var='date'>13</var>, <var data-var='time'>17:58</var> PDT Update - The web app is experiencing errors which is preventing users from accessing the error dashboard. This is an issue that resurfaced after being resolved earlier today. We will update this post after further investigation. Jul <var data-var='date'>13</var>, <var data-var='time'>10:03</var> PDT Monitoring - The web application is back up and account API is no longer experiencing errors. We're continuing to monitor the resolution. Please reach us at support@airbrake.io if you have any questions. Jul <var data-var='date'>13</var>, <var data-var='time'>07:54</var> PDT Investigating - We are currently investigating an issue causing the login page and dashboard to fail loading due to an error. Please reach out to support@airbrake.io if you have any questions. We apologize for the inconvenience.

Airbrake dashboard unavailable

Jun <var data-var='date'>16</var>, <var data-var='time'>18:41</var> PDT Resolved - A resolution has been implemented for the metrics API issues that resulted in error graphs to not be visible on the dashboard. We're continuing to monitor the service. Please let us know if you have any questions at support@airbrake.io. Jun <var data-var='date'>16</var>, <var data-var='time'>13:48</var> PDT Investigating - We're investigating an issue where the metric graphs for errors are not displaying properly. Please reach out to us at support@airbrake.io if you have any questions. We apologize for the inconvenience. Error monitoring other than metrics is unaffected.

Dashboard graph availability

Jun <var data-var='date'> 2</var>, <var data-var='time'>18:16</var> PDT Resolved - Update: A subset of customers experienced service disruptions due to an unresponsive backend API instance. This was triggered by a log storage issue that correlated with a surge in error ingestion traffic. We are actively investigating automated solutions to prevent similar occurrences in the future. We apologize for any inconvenience this caused, please reach out with any questions to support@airbrake.io. May <var data-var='date'>28</var>, <var data-var='time'>17:24</var> PDT Update - We have resolved the backend issue where a subset of Airbrake projects' experienced blank error dashboards. The backlog in error events have been processed. Please reach out to support@airbrake.io if you have further questions. May <var data-var='date'>27</var>, <var data-var='time'>10:46</var> PDT Investigating - We have identified an issue affecting a certain subset of customers and are currently working on remediation. We'll keep this issue updated. Please reach out to support@airbrake.io if you have further questions.

Reduced performance in API and error dashboard

Nov <var data-var='date'> 2</var>, <var data-var='time'>16:55</var> PDT Resolved - This issue has been resolved. Our emails have been flowing for the past day and our sign-up page is live again. We apologize for any inconvenience. If you have any questions, please reach us at support@airbrake.io. Nov <var data-var='date'> 1</var>, <var data-var='time'>12:04</var> PDT Monitoring - Updating the status to "monitoring" as alert emails are fixed. We will update the status to "resolved" when the new sign-up page is live. Nov <var data-var='date'> 1</var>, <var data-var='time'>11:47</var> PDT Update - Our email provider has reenabled our alerts domain. All email notifications should be operational again. We still have our sign-up page disabled to implement an additional verification measure and will close this incident when the new sign-up page is live. If you have any questions, please reach us at support@airbrake.io. Oct <var data-var='date'>31</var>, <var data-var='time'>15:31</var> PDT Identified - Our alerts notifications have been disabled. Digests and other notification types (like Slack) are unaffected. We are working with our email service provider to reenable our alerts domain and will follow up as there is more information. Reach us at support@airbrake.io if you have any questions.

Email notifications for alerts have been disabled

Oct <var data-var='date'>25</var>, <var data-var='time'>12:23</var> PDT Resolved - This issue has been resolved. Emails are being delivered again. Reach out to support@airbrake.io if you have any questions. Oct <var data-var='date'>25</var>, <var data-var='time'>10:42</var> PDT Identified - We have identified an issue with delivering email alerts. We are investigating the cause of the disruption and will report more as we gather more info. Errors are still being recorded and are being delivered to your error dashboards. Integration alerts are also unaffected.

Email notifications not delivering

May <var data-var='date'>12</var>, <var data-var='time'>09:11</var> PDT Resolved - We apologize for any inconvenience this caused. The incident has been resolved and normal operations have resumed. May <var data-var='date'>11</var>, <var data-var='time'>16:34</var> PDT Identified - We have identified an issue affecting a certain subset of customers and are currently working on remediation. You may notice a delay in Error processing. We'll keep this issue updated. Please reach out to support@airbrake.io if you have further questions.

Processing delay for Errors and APM for some customers

Mar <var data-var='date'>15</var>, <var data-var='time'>15:25</var> PDT Resolved - The backlog in error notifications has been processed. Users should get error emails and see errors show up on their dashboards without delays again. We apologize for any inconvenience. If you have any questions, please reach us at support@airbrake.io. Mar <var data-var='date'>15</var>, <var data-var='time'>09:38</var> PDT Identified - We have identified the issue and are currently working to process the backlog of notifications. We will continue to monitor the situation and provide updates. Mar <var data-var='date'>15</var>, <var data-var='time'>07:53</var> PDT Investigating - We are currently investigating a delay in error processing. Errors on your error dashboard and error email notifications might take longer to appear than usual. We will share updates as we make progress in addressing this issue. If you have any questions or concerns, please reach us at support@airbrake.io. We apologize for any inconvenience.

Delay in error processing

Dec <var data-var='date'>28</var>, <var data-var='time'>17:13</var> PST Resolved - This incident has been resolved. Services should be back to normal. Dec <var data-var='date'>28</var>, <var data-var='time'>12:41</var> PST Investigating - We are currently investigating increased errors.

Increased Errors

Aug <var data-var='date'>24</var>, <var data-var='time'>12:25</var> PDT Resolved - This incident has been resolved. Aug <var data-var='date'>24</var>, <var data-var='time'>12:23</var> PDT Update - Queues are back to normal and notifications are once against being delivered. Aug <var data-var='date'>24</var>, <var data-var='time'>12:22</var> PDT Update - We are continuing to work on a fix for this issue. Aug <var data-var='date'>24</var>, <var data-var='time'>09:51</var> PDT Identified - We've identified an issue with our queue. Notification delivery is currently delayed. Thanks for your patience and sorry for any inconvenience this causes.  If you have any issues or questions please email support@airbrake.io

Increased queue size, Delayed notifications

Apr <var data-var='date'>25</var>, <var data-var='time'>04:13</var> PDT Resolved - This incident has been resolved, the Airbrake web app is now working as expected. If you have any issues or questions please email support@airbrake.io Apr <var data-var='date'>25</var>, <var data-var='time'>03:53</var> PDT Investigating - Some users may not be able to access the Airbrake web app. We have all hands on deck troubleshooting this dashboard outage and look forward to getting it back online ASAP. Thanks for your patience and sorry for any inconvenience this causes. If you have any issues or questions please email support@airbrake.io

Airbrake web app temporarily unavailable

Nov <var data-var='date'>11</var>, <var data-var='time'>06:53</var> PST Resolved - The cause of the incident has been identified and resolved. Systems have returned to their operational states. Nov <var data-var='date'>11</var>, <var data-var='time'>06:12</var> PST Investigating - Some users may not be able to load error, performance, or deploy data in their dashboard. The API is accepting errors as usual and you should be able to continue troubleshooting errors as soon as the dashboard has been restored. Thanks for your patience and sorry for any inconvenience this causes. If you have any issues or questions please email support@airbrake.io.

Degraded dashboard experience

Oct <var data-var='date'> 2</var>, <var data-var='time'>14:07</var> PDT Resolved - The backlog of notice and apm data has been processed and shard performance has stabilized. Please reach out to support (support@airbrake.io) with further questions. Oct <var data-var='date'> 2</var>, <var data-var='time'>13:53</var> PDT Monitoring - A fix has been implemented and we are monitoring while the backlog of notice data / APM events are being processed. Oct <var data-var='date'> 2</var>, <var data-var='time'>13:45</var> PDT Identified - The issue has been identified and a fix is being implemented. Oct <var data-var='date'> 2</var>, <var data-var='time'>13:39</var> PDT Update - We are continuing to investigate this issue. Oct <var data-var='date'> 2</var>, <var data-var='time'>13:36</var> PDT Investigating - We are currently investigating degraded performance on a database shard. Some customers may experience delays processing error and APM data.

Degraded performance

Oct <var data-var='date'>22</var>, <var data-var='time'>08:30</var> PDT Resolved - This incident has been resolved. Oct <var data-var='date'>22</var>, <var data-var='time'>07:57</var> PDT Investigating - Some users may not be able to access the error dashboard.  The API is accepting errors as usual and you should be able to continue troubleshooting your errors as soon as the dashboard has been restored.  We have all hands on deck troubleshooting this dashboard outage and look forward to getting it back online ASAP.  Thanks for your patience and sorry for any inconvenience this causes.  If you have any issues or questions please email support@airbrake.io

Error dashboard temporarily unavailable

Jun <var data-var='date'>25</var>, <var data-var='time'>21:37</var> PDT Resolved - We've applied a change that relieved the load on our production database that was causing a delay in processing performance data. Delays in application performance should be resolving. If you continue to experience any issues, please let us know at support@airbrake.io. Jun <var data-var='date'>25</var>, <var data-var='time'>15:51</var> PDT Update - We are continuing to work on a fix for this issue. Jun <var data-var='date'>25</var>, <var data-var='time'>14:33</var> PDT Update - We are continuing to investigate the issue causing a delay in processing performance data. Customer's project dashboards and email notifications should be returning to normal. If you experience any issues or have any questions, reach us at support@airbrake.io Jun <var data-var='date'>25</var>, <var data-var='time'>09:56</var> PDT Identified - We have identified an issue where for some requests, our API is failing due to data caching errors. Affected users might experience their error dashboards failing to load and a delay in email notifications. If you have any questions or concerns, reach out to support@airbrake.io. Jun <var data-var='date'>25</var>, <var data-var='time'>08:51</var> PDT Update - We are continuing to investigate an issue. Jun <var data-var='date'>25</var>, <var data-var='time'>08:02</var> PDT Investigating - We are currently investigating API errors which are affecting some projects.

Notice and APM processing delay

Jun <var data-var='date'>18</var>, <var data-var='time'>14:27</var> PDT Resolved - The overloaded ingress server which caused issues with deploy and error tracking was scaled up and the issues have been resolved. If you have any questions, please reach out to us at support@airbrake.io. Jun <var data-var='date'>18</var>, <var data-var='time'>12:31</var> PDT Monitoring - All systems are back to normal, we will continue to monitor the situation. If you experience any issues sending errors or tracking deploys for your project, please let us know support@airbrake.io Jun <var data-var='date'>18</var>, <var data-var='time'>12:00</var> PDT Update - We've received reports of issues with tracking deploys, we are investigating this and believe it to be a related symptom. If you are experiencing this issue please let us know support@airbrake.io, we apologize for any inconvenience this causes. Jun <var data-var='date'>18</var>, <var data-var='time'>11:53</var> PDT Update - We are continuing to investigate this issue. Jun <var data-var='date'>18</var>, <var data-var='time'>11:48</var> PDT Investigating - We are currently investigating API errors which are affecting some projects.

Inbound API errors

May <var data-var='date'>12</var>, <var data-var='time'>12:01</var> PDT Resolved - The issue causing the API errors and delay in error notice processing has been resolved. The backlog has been processed and systems are operating normally. We apologize for the inconvenience. Please reach us at support@airbrake.io if you have any questions. May <var data-var='date'>12</var>, <var data-var='time'>07:46</var> PDT Monitoring - A fix has been implemented and we are monitoring the results. Our APIs are back to normal and there are no more processing delays. Please contact the support at support@airbrake.io if you have any questions. May <var data-var='date'>11</var>, <var data-var='time'>21:59</var> PDT Identified - We have identified the issue causing API errors and processing delays which is affecting some projects. We are currently working to mitigate the outage in processing and are restarting core processing components. We apologize for the inconvenience. If you have any questions or concerns, please reach us at support@airbrake.io. May <var data-var='date'>11</var>, <var data-var='time'>05:58</var> PDT Investigating - We are currently investigating API errors and processing delays which are affecting some projects. Some errors will be delayed but not lost. Sorry for the inconvenience.

Processing delays and API errors affecting some projects

Oct <var data-var='date'>23</var>, <var data-var='time'>23:00</var> PDT Completed - The scheduled maintenance has been completed. Oct <var data-var='date'>23</var>, <var data-var='time'>22:01</var> PDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Oct <var data-var='date'>23</var>, <var data-var='time'>14:32</var> PDT Scheduled - At approximately 22:00 PDT Airbrake will undergo scheduled maintenance to upgrade its DB layer. How will you be affected? - The Airbrake UI will be unavailable for a period of up to 5 minutes - Error processing may be delayed - Notifications of errors won't be dropped but will be delayed We appreciate your patience as we continue to improve the reliability of our service. Please reach out to us at support@airbrake.io if you have any questions.

DB Upgrades

Oct <var data-var='date'> 9</var>, <var data-var='time'>15:03</var> PDT Resolved - This incident has been resolved, you should no longer see any delay in error processing or dashboard loading. No error data was lost during this incident, only delayed. We apologize for any inconvenience this has caused! If you have issues or questions, please email support@airbrake.io Oct <var data-var='date'> 9</var>, <var data-var='time'>14:48</var> PDT Update - We are continuing to investigate this issue. Oct <var data-var='date'> 9</var>, <var data-var='time'>14:13</var> PDT Investigating - We’re experiencing delays in error processing and some customers may experience timeouts when loading dashboards. No error data is being lost and we are actively working to resolve this issue. If you have any questions or issues, please contact us at support@airbrake.io.

Delay in error processing and dashboard timeouts

Jul <var data-var='date'>29</var>, <var data-var='time'>15:01</var> PDT Resolved - All systems are back to normal and the backlog of data has been fully processed. Customers impacted by the delay should see errors again in their dashboard. If you have any questions or issues, please contact us at support@airbrake.io. Jul <var data-var='date'>29</var>, <var data-var='time'>14:31</var> PDT Monitoring - A fix has been implemented and we are monitoring the results. For customers experiencing a delay the back log of errors is currently being processed. Jul <var data-var='date'>29</var>, <var data-var='time'>13:15</var> PDT Identified - We’re experiencing periodic delays in error processing. Some customers may also experience an empty error dashboard. No error data is being lost and we are actively working to resolve this issue. If you have any questions or issues, please contact us at support@airbrake.io.

Jun <var data-var='date'> 3</var>, <var data-var='time'>06:52</var> PDT Resolved - All systems are back to normal and the backlog of data has been fully processed. Some errors that occurred during the last (approximately) 12 hours will have been late to appear on your dashboard and to raise notifications. During this incident, we had some large delays in processing errors resulting in out-of-date dashboard data and late error notifications. This is not acceptable service and for this, we're very sorry. We're working hard to learn and improve from what happened. Jun <var data-var='date'> 3</var>, <var data-var='time'>04:32</var> PDT Monitoring - The fix was successful and is complete, we expect to be fully restored to normal operation very soon. We are processing a large backlog of error data and as a result, errors that occurred in your projects during the last ~12 hours are likely to start appearing and raising notifications. We are extremely sorry for this delay and unsatisfactory service. We'll close with a more detailed postmortem, soon. Jun <var data-var='date'> 3</var>, <var data-var='time'>03:12</var> PDT Identified - A fix is being implemented. We hope to be fully operational in approximately 1 hour, but it's difficult to predict at this time. We are very sorry for this major disruption. We are working hard to resolve it quickly. If you have any questions or concerns, you can reach us at support@airbrake.io. Jun <var data-var='date'> 3</var>, <var data-var='time'>01:10</var> PDT Update - Some customers may also experience an empty error dashboard. We're working hard to fix this as soon as possible. Jun <var data-var='date'> 3</var>, <var data-var='time'>01:08</var> PDT Update - We are continuing to investigate this issue. Jun <var data-var='date'> 2</var>, <var data-var='time'>21:25</var> PDT Investigating - Some accounts may be experiencing delays in error processing. We’re actively investigating and will post an update once we learn more. If you have any questions, please contact us at support@airbrake.io.

Oct <var data-var='date'>31</var>, <var data-var='time'>07:46</var> PDT Resolved - This incident has been resolved. Oct <var data-var='date'>31</var>, <var data-var='time'>07:17</var> PDT Identified - We’re experiencing some delays in error processing and we’re actively working on a fix. If you have any questions, please contact us at support@airbrake.io.

Is Airbrake down?

Airbrake status is UP

Lastest published events

Metric graphs not loading

Email notifications

Error dashboard loading issues

Airbrake Services, Components and Regions

Recent Airbrake events

Metric graphs not loading

Email notifications

Error dashboard loading issues

Dashboard unavailble

Airbrake dashboard unavailable

Dashboard graph availability