Jul <var data-var='date'>21</var>, <var data-var='time'>21:00</var> EDT Resolved - Between 8:50pm and 9:10pm America/New_York on July 21st, there were availability blips during an upgrade of our nginx proxy layer. SDK clients will have retried during this event and we recovered from a backlog by 9:30pm.

Ingestion failures during upgrade

Jun <var data-var='date'>15</var>, <var data-var='time'>14:21</var> EDT Resolved - This incident has been resolved. Jun <var data-var='date'>15</var>, <var data-var='time'>12:57</var> EDT Monitoring - Latency is returning to normal. We are waiting on our remaining queues to drain. Jun <var data-var='date'>15</var>, <var data-var='time'>11:05</var> EDT Identified - We are investigating elevated queue backlogs for our analytics ingestion. Analytics and alerts may be delayed.

Delayed analytics ingestion and alerts

Jun <var data-var='date'> 2</var>, <var data-var='time'>02:10</var> EDT Resolved - This incident has been resolved. Jun <var data-var='date'> 2</var>, <var data-var='time'>01:41</var> EDT Identified - We've identified the problem and are rolling out a fix. Jun <var data-var='date'> 2</var>, <var data-var='time'>01:23</var> EDT Investigating - We're investigating slow and failing dashboard queries.

Dashboard instability

Nov <var data-var='date'>18</var>, <var data-var='time'>09:52</var> EST Resolved - This incident has been resolved. Nov <var data-var='date'>18</var>, <var data-var='time'>08:37</var> EST Monitoring - Error rates are improving, we're continuing to monitor stability of the system. Nov <var data-var='date'>18</var>, <var data-var='time'>07:19</var> EST Identified - An outage from our upstream CDN provider is causing instability for our dashboard. Data collection is not impacted.

Nov <var data-var='date'> 5</var>, <var data-var='time'>17:49</var> EST Resolved - This incident is fully resolved. Nov <var data-var='date'> 5</var>, <var data-var='time'>14:43</var> EST Investigating - We are investigating instability in our ingestion endpoints.

Ingestion Instability

Oct <var data-var='date'>14</var>, <var data-var='time'>11:59</var> EDT Resolved - The performance issue with our API server has been resolved. Oct <var data-var='date'>14</var>, <var data-var='time'>11:35</var> EDT Identified - We have identified the cause of the issue and are deploying a mitigation. Oct <var data-var='date'>14</var>, <var data-var='time'>10:53</var> EDT Investigating - We are investigating performance issues with our API server causing dashboard instability. Data collection is not impacted.

Dashboard instability and slow performance

Aug <var data-var='date'> 4</var>, <var data-var='time'>22:06</var> EDT Resolved - This incident has been resolved. Aug <var data-var='date'> 4</var>, <var data-var='time'>15:03</var> EDT Monitoring - A fix has been implemented and we are monitoring the results. Aug <var data-var='date'> 4</var>, <var data-var='time'>13:56</var> EDT Update - We are continuing to investigate the issue and are working with our vendor to restore service. Aug <var data-var='date'> 4</var>, <var data-var='time'>12:55</var> EDT Investigating - We are currently investigating this issue.

Partial outage

Jun <var data-var='date'>12</var>, <var data-var='time'>21:18</var> EDT Resolved - This incident has been resolved. Jun <var data-var='date'>12</var>, <var data-var='time'>16:42</var> EDT Update - We are continuing to monitor for any further issues. Jun <var data-var='date'>12</var>, <var data-var='time'>16:42</var> EDT Monitoring - The upstream issue has been resolved. We are monitoring and beginning to work through the backlog of session data. Jun <var data-var='date'>12</var>, <var data-var='time'>15:07</var> EDT Identified - GCP is suffering a service outage. Google has opened an incident tracking this issue at https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW#2c2sBHWU84yPDJ8y1ar4 Jun <var data-var='date'>12</var>, <var data-var='time'>14:15</var> EDT Investigating - We are investigating a wide-spread issue with our cloud provider causing ingestion issues and limited access to the dashboard.

Platform wide issues

Mar <var data-var='date'> 6</var>, <var data-var='time'>21:43</var> EST Completed - The scheduled maintenance has been completed. Mar <var data-var='date'> 6</var>, <var data-var='time'>20:00</var> EST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Mar <var data-var='date'> 6</var>, <var data-var='time'>19:55</var> EST Update - We will be undergoing scheduled maintenance during this time. Mar <var data-var='date'> 6</var>, <var data-var='time'>19:01</var> EST Scheduled - We will be undergoing database maintenance. Data visibility in the dashboard may be slightly delayed, but product functionality should otherwise be unaffected

Database maintenance

Feb <var data-var='date'>24</var>, <var data-var='time'>16:32</var> EST Resolved - This incident has been resolved. Feb <var data-var='date'>24</var>, <var data-var='time'>15:04</var> EST Monitoring - A fix has been implemented and we are monitoring the results. Feb <var data-var='date'>24</var>, <var data-var='time'>14:14</var> EST Investigating - We are currently investigating this issue.

Degraded dashboard performance

Feb <var data-var='date'>20</var>, <var data-var='time'>12:00</var> EST Completed - The scheduled maintenance has been completed. Feb <var data-var='date'>20</var>, <var data-var='time'>10:00</var> EST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Feb <var data-var='date'>19</var>, <var data-var='time'>15:22</var> EST Scheduled - We will be performing scheduled maintenance on our Streaming Data Export system. During this period, data exports may be delayed.

Streaming Data Export Maintenance

Feb <var data-var='date'>19</var>, <var data-var='time'>20:35</var> EST Resolved - This incident has been resolved. Feb <var data-var='date'>19</var>, <var data-var='time'>18:19</var> EST Monitoring - We've applied a fix and are monitoring the results Feb <var data-var='date'>19</var>, <var data-var='time'>16:24</var> EST Update - We are continuing to work with a vendor on identifying the root cause of our ingestion delays. Feb <var data-var='date'>19</var>, <var data-var='time'>13:26</var> EST Investigating - We are investigating issues causing ingestion delays. We're working with a vendor to identify the root cause of the problem

Ingestion delays

Feb <var data-var='date'> 4</var>, <var data-var='time'>16:14</var> EST Resolved - This incident has been resolved. Feb <var data-var='date'> 4</var>, <var data-var='time'>15:38</var> EST Monitoring - We have identified a potential cause and observing significant improvements to ingestion delays. We are continuing to monitor. Feb <var data-var='date'> 4</var>, <var data-var='time'>14:11</var> EST Update - Dashboard performance should be improved. We are continuing to work with a vendor on identifying the root cause of our ingestion delays. Feb <var data-var='date'> 4</var>, <var data-var='time'>13:29</var> EST Investigating - We are investigating issues causing ingestion delays and performance issues loading the dashboard and metrics.

Jan <var data-var='date'>30</var>, <var data-var='time'>16:45</var> EST Resolved - This incident has been resolved. Jan <var data-var='date'>30</var>, <var data-var='time'>16:42</var> EST Update - We are continuing to work on a fix for this issue. Jan <var data-var='date'>29</var>, <var data-var='time'>20:10</var> EST Identified - We've resolved the issue for most Streaming Data Export destinations. A few customers have been temporarily disabled until another fix goes out in the morning, at which time they will be caught up and we will resolve the incident. Jan <var data-var='date'>29</var>, <var data-var='time'>17:55</var> EST Update - We are continuing to investigate this issue. Jan <var data-var='date'>29</var>, <var data-var='time'>17:04</var> EST Investigating - The Streaming Data Export service is delayed. We've identified the issue and are working to remediate.

Delayed Streaming Data Export

Jan <var data-var='date'> 8</var>, <var data-var='time'>11:37</var> EST Resolved - The system has stabilized and our processing backlog is recovering. Jan <var data-var='date'> 8</var>, <var data-var='time'>10:40</var> EST Investigating - We are seeing significant infrastructure instability from our hosting provider.

Infrastructure instability

Dec <var data-var='date'> 8</var>, <var data-var='time'>13:52</var> EST Completed - The scheduled maintenance has been completed. Dec <var data-var='date'> 8</var>, <var data-var='time'>12:01</var> EST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Dec <var data-var='date'> 7</var>, <var data-var='time'>12:01</var> EST Scheduled - We'll be performing a routine database upgrade. There may be dashboard instability, delayed alerts, and delayed streaming data export jobs. Session data collection will be unaffected.

Database upgrade

Aug <var data-var='date'> 6</var>, <var data-var='time'>22:56</var> EDT Completed - This upgrade is complete. Aug <var data-var='date'> 6</var>, <var data-var='time'>22:30</var> EDT Verifying - Upgrade has completed - we're validating the upgrade Aug <var data-var='date'> 6</var>, <var data-var='time'>22:00</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'> 1</var>, <var data-var='time'>15:01</var> EDT Scheduled - We will be performing routine database upgrades during this time. During this window, alerts will be disabled and there will be Dashboard errors. Session Ingestion will not be impacted.

Database Upgrade

Jul <var data-var='date'>30</var>, <var data-var='time'>23:09</var> EDT Completed - We've caught up on ingestion and re-enabled alerting so are closing the maintenance window Jul <var data-var='date'>30</var>, <var data-var='time'>22:33</var> EDT Verifying - Our upgrade is complete - we're verifying the upgrade and will begin catching up on any queues after that is complete. Jul <var data-var='date'>30</var>, <var data-var='time'>22:00</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jul <var data-var='date'>26</var>, <var data-var='time'>14:52</var> EDT Scheduled - We will be performing routine database upgrades during this time. During this window, ingestion of Issues will be delayed and alerts will be disabled. There will also be errors viewing Issues.

Jun <var data-var='date'>21</var>, <var data-var='time'>14:22</var> EDT Resolved - The backlog has fully recovered. Jun <var data-var='date'>21</var>, <var data-var='time'>13:21</var> EDT Update - We are continuing to work down a backlog of ingestion from the initial outage, but have otherwise recovered from the outage. Jun <var data-var='date'>21</var>, <var data-var='time'>11:57</var> EDT Monitoring - Ingestion has stabilized, we are continuing to monitor the recovery. Jun <var data-var='date'>21</var>, <var data-var='time'>11:00</var> EDT Investigating - We are investigating instability in our ingestion endpoints.

Apr <var data-var='date'>17</var>, <var data-var='time'>18:00</var> EDT Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>17</var>, <var data-var='time'>15:00</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>16</var>, <var data-var='time'>16:20</var> EDT Scheduled - We will be performing scheduled maintenance on our Streaming Data Export system. During this period, data exports may be delayed.

Apr <var data-var='date'> 4</var>, <var data-var='time'>19:00</var> EDT Completed - The scheduled maintenance has been completed. Apr <var data-var='date'> 4</var>, <var data-var='time'>17:00</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'> 4</var>, <var data-var='time'>13:12</var> EDT Scheduled - We will be upgrading our primary search database to improve performance and reliability. During this time period, data ingestion may be delayed, and alerts may not fire.

Search database upgrade

Mar <var data-var='date'>30</var>, <var data-var='time'>19:08</var> EDT Resolved - This incident has been resolved Mar <var data-var='date'>30</var>, <var data-var='time'>15:38</var> EDT Monitoring - A fix has been implemented and ingestion is recovering Mar <var data-var='date'>30</var>, <var data-var='time'>13:03</var> EDT Identified - We've identified an issue with our analytics database resulting in delayed ingestion and degraded alerting performance. The problem has been identified and we're working on a solution.

Degraded ingestion and alerting performance

Oct <var data-var='date'>30</var>, <var data-var='time'>16:25</var> EDT Resolved - This incident has been resolved. Oct <var data-var='date'>30</var>, <var data-var='time'>16:12</var> EDT Investigating - We're seeing elevated error rates loading the UI.

Application sporadically unavailable

Sep <var data-var='date'>30</var>, <var data-var='time'>17:00</var> EDT Completed - The scheduled maintenance has been completed. Sep <var data-var='date'>30</var>, <var data-var='time'>12:00</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>20</var>, <var data-var='time'>13:21</var> EDT Scheduled - We will be upgrading our primary search database to improve performance and reliability. During this time period, data ingestion may be delayed, and alerts may not fire.

Sep <var data-var='date'>22</var>, <var data-var='time'>15:59</var> EDT Resolved - This incident has been resolved. Sep <var data-var='date'>22</var>, <var data-var='time'>15:59</var> EDT Update - We've monitored several exports and believe this is resolved. Sep <var data-var='date'>22</var>, <var data-var='time'>13:24</var> EDT Monitoring - The most recent export window ran successfully but we're continuing to monitor it for issues. Sep <var data-var='date'>22</var>, <var data-var='time'>12:35</var> EDT Identified - Our Streaming Data Export system is not currently exporting data, we are working to resolve this. When this is reenabled it will catch up on data that would have been exported earlier.

Is LogRocket down?

LogRocket status is UP

Lastest published events

Ingestion failures during upgrade

Delayed analytics ingestion and alerts

Dashboard instability

LogRocket Services, Components and Regions

Recent LogRocket events

Ingestion failures during upgrade

Delayed analytics ingestion and alerts

Dashboard instability

Dashboard instability

Ingestion Instability

Dashboard instability and slow performance