Apr <var data-var='date'>14</var>, <var data-var='time'>09:16</var> UTC Resolved - This incident had been resolved Apr <var data-var='date'>11</var>, <var data-var='time'>15:43</var> UTC Update - Maintenance in US Cluster is completed. Alerts are functional again, Data Collectors are still offline Apr <var data-var='date'>11</var>, <var data-var='time'>10:36</var> UTC Update - Starting 11:00 AM UTC. Expected downtime for US Customers: ~3 hours Apr <var data-var='date'>11</var>, <var data-var='time'>10:33</var> UTC Update - Maintenance in Frankfurt Cluster is completed. Alerts are functional again, Data Collectors are still offline. Apr <var data-var='date'>11</var>, <var data-var='time'>06:21</var> UTC Update - Starting 7:00 AM UTC. Expected downtime for Frankfurt Customers: ~3 hours Apr <var data-var='date'> 6</var>, <var data-var='time'>08:18</var> UTC Update - We are experiencing issues in collecting data, and detecting and dispatching anomaly type alerts Apr <var data-var='date'> 4</var>, <var data-var='time'>21:32</var> UTC Update - All data collectors are currently not working. We are working on restoring them. Apr <var data-var='date'> 4</var>, <var data-var='time'>15:02</var> UTC Update - [CRUCIAL UPDATE] The following Data Collectors are currently affected and are not retreiving any samples: - Snowflake - S3 - Kinesis We are continuing to investigate the issue. Apr <var data-var='date'> 4</var>, <var data-var='time'>08:00</var> UTC Investigating - Snowflake Streams have encountered difficulties in retrieving data samples over the last few hours. This issue is not localized to a single region but is impacting all of our clusters across different geographical areas. Our team is actively conducting a thorough investigation to identify the root cause of this disruption and to implement a resolution as quickly as possible

Data collectors / Detection Issues

Jan <var data-var='date'>22</var>, <var data-var='time'>10:27</var> UTC Resolved - This incident had been resolved Jan <var data-var='date'>22</var>, <var data-var='time'>10:10</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jan <var data-var='date'>22</var>, <var data-var='time'>09:59</var> UTC Investigating - Anomaly detection and anomaly alerting in EU (Frankfurt) region are current not working. We've identified the issue and are working to resolve it

EU (Frankfurt) Region - Anomaly Detection and Anomaly Alerts not working

Dec <var data-var='date'>19</var>, <var data-var='time'>14:22</var> UTC Resolved - This incident has been resolved. Dec <var data-var='date'>19</var>, <var data-var='time'>14:05</var> UTC Update - We are continuing to investigate this issue. Dec <var data-var='date'>19</var>, <var data-var='time'>14:03</var> UTC Investigating - We are currently investigating this issue.

UI intermittently unavailable in US zone

Sep <var data-var='date'>22</var>, <var data-var='time'>07:39</var> UTC Resolved - This incident has been resolved. Sep <var data-var='date'>21</var>, <var data-var='time'>13:32</var> UTC Update - There is no impact on any other environments Sep <var data-var='date'>21</var>, <var data-var='time'>13:19</var> UTC Identified - We are performing urgent infrastructure maintenance in the Sydney environment. Expect downtime of 4 hours

Infrastructure maintenance

Jul <var data-var='date'>10</var>, <var data-var='time'>14:01</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'>10</var>, <var data-var='time'>13:26</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'>10</var>, <var data-var='time'>12:56</var> UTC Identified - Partial data collection issue for data collectors.

Partial Data collection issue

Jun <var data-var='date'> 2</var>, <var data-var='time'>12:38</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'> 1</var>, <var data-var='time'>04:23</var> UTC Monitoring - We has an issue that caused missing a 4 hours of data. It may have caused many false No-Data alerts. A fix has been implemented and we are monitoring the results.

US Region - Errors in Processing Incoming Metrics - Data Loss with High Probability

May <var data-var='date'> 4</var>, <var data-var='time'>07:20</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>19</var>, <var data-var='time'>16:16</var> UTC Update - Incident Resolved Apr <var data-var='date'>19</var>, <var data-var='time'>15:33</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>19</var>, <var data-var='time'>08:57</var> UTC Investigating - We're Investigating the issue

Apr <var data-var='date'>19</var>, <var data-var='time'>06:30</var> UTC Resolved - We're experiencing an elevated level of data API errors and are currently looking into the issue.

Elevated API Errors

Sep <var data-var='date'>26</var>, <var data-var='time'>08:27</var> UTC Resolved - This incident has been resolved. Sep <var data-var='date'>26</var>, <var data-var='time'>08:04</var> UTC Investigating - We are currently investigating this issue.

Aug <var data-var='date'> 7</var>, <var data-var='time'>11:39</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'> 7</var>, <var data-var='time'>11:38</var> UTC Update - We are continuing to monitor for any further issues. Aug <var data-var='date'> 7</var>, <var data-var='time'>11:38</var> UTC Update - We are continuing to monitor for any further issues. Aug <var data-var='date'> 7</var>, <var data-var='time'>11:01</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Aug <var data-var='date'> 7</var>, <var data-var='time'>07:37</var> UTC Update - We are continuing to work on a fix for this issue. Aug <var data-var='date'> 7</var>, <var data-var='time'>07:35</var> UTC Identified - Anomaly detection and anomaly alerting in US region are currently unstable, we've identified the issue and are working to resolve it

US Region - Anomaly Detection and Anomaly Alerts unstable

Apr <var data-var='date'>30</var>, <var data-var='time'>20:30</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>30</var>, <var data-var='time'>20:16</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>30</var>, <var data-var='time'>18:39</var> UTC Investigating - We're currently investigating an issue where small portion of metrics (the same constant metrics) are experiencing sample loss. As a result false-positive no-data alerts can be triggered and the metrics data shall not be shown in UI.

US Region | Some Metrics Are Loosing Samples

Jan <var data-var='date'>17</var>, <var data-var='time'>16:33</var> UTC Resolved - This incident has been resolved. Jan <var data-var='date'>17</var>, <var data-var='time'>14:00</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jan <var data-var='date'>17</var>, <var data-var='time'>12:59</var> UTC Identified - During the last 24 hours some alerts may have been missed by the system, we're deploying a hotfix in an hour time

Some Alerts Went Missing During Last 24 Hours

Sep <var data-var='date'>24</var>, <var data-var='time'>14:07</var> UTC Resolved - This incident has been resolved. Sep <var data-var='date'>24</var>, <var data-var='time'>10:35</var> UTC Identified - We're experiencing connectivity issues since 5AM UTC time, this may result in degraded performance and reduced accuracy of alerts and composite calculations. There is also a minor chance of sample data loss. We have identified the problem and applied the necessary measures and are monitoring the system state.

US Cluster | Degraded operation of multiple services, probability of minor data loss

Aug <var data-var='date'> 9</var>, <var data-var='time'>18:05</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'> 9</var>, <var data-var='time'>17:41</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Aug <var data-var='date'> 9</var>, <var data-var='time'>16:37</var> UTC Investigating - Streams working with BigQuery does not retrieve new data, seems to be related to incompatible changes in BigQuery - Investigating. This is a cross region issue that impact all of our clusters.

BigQuery stream does not produce data

Jul <var data-var='date'> 1</var>, <var data-var='time'>06:46</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'> 1</var>, <var data-var='time'>06:21</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'> 1</var>, <var data-var='time'>06:21</var> UTC Investigating - Around 05:00 UTC we detected errors in samples receiving pipeline, issue is under investigation. Please be advised that at least some of the incoming metrics data was lost and false-positive alerts can be triggered

Feb <var data-var='date'> 2</var>, <var data-var='time'>21:30</var> UTC Resolved - This incident has been resolved. Feb <var data-var='date'> 2</var>, <var data-var='time'>20:47</var> UTC Investigating - We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.

Mumbai site is not accessible

Jan <var data-var='date'>12</var>, <var data-var='time'>19:40</var> UTC Resolved - This incident has been resolved. Jan <var data-var='date'>12</var>, <var data-var='time'>18:34</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jan <var data-var='date'>12</var>, <var data-var='time'>13:51</var> UTC Identified - We've identified the issue and working on a fix. Please note that alerts console is fully functional and no events are lost.

Alert Notifications via Slack Channels Are Not Being Sent

Jan <var data-var='date'> 6</var>, <var data-var='time'>13:28</var> UTC Resolved - The incident was resolved. Alerts dispatching got delayed but no data was lost. Jan <var data-var='date'> 6</var>, <var data-var='time'>13:16</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jan <var data-var='date'> 6</var>, <var data-var='time'>11:07</var> UTC Identified - The issue has been identified and a fix is being implemented.

Processing of Anomaly Alerts is currently impared for customers of "in.anodot.com" cluster (Mumbai AZ)

Jan <var data-var='date'> 3</var>, <var data-var='time'>18:48</var> UTC Resolved - This incident has been resolved. Jan <var data-var='date'> 3</var>, <var data-var='time'>13:10</var> UTC Update - We are continuing to monitor for any further issues. Jan <var data-var='date'> 3</var>, <var data-var='time'>10:00</var> UTC Monitoring - Alerts is now back to normal and we're investigating the root cause Jan <var data-var='date'> 3</var>, <var data-var='time'>09:50</var> UTC Identified - Our alert data processing infrastructure is running behind which is causing delays in sending alerts. No data has been lost and the system should be caught up shortly.

Alert Processing Delays In US-EAST Region (app.anodot.com)

Sep <var data-var='date'>21</var>, <var data-var='time'>09:00</var> UTC Resolved - This incident has been resolved. Sep <var data-var='date'>20</var>, <var data-var='time'>22:12</var> UTC Update - We are continuing to monitor for any further issues. Sep <var data-var='date'>20</var>, <var data-var='time'>20:00</var> UTC Monitoring - A fix has been implemented and applied globally, additional monitoring has been applied and we'll be monitoring it over the next few days Sep <var data-var='date'>20</var>, <var data-var='time'>18:35</var> UTC Update - Mitigation identified and is being rolled out to production clusters Sep <var data-var='date'>20</var>, <var data-var='time'>17:41</var> UTC Identified - We are working on a mitigation with an ETA of 1-2 hours.

Alerts dispatch through channels does not operate in Frankfurt, Mumbai, Sydney and Oregon data-centers

Aug <var data-var='date'>24</var>, <var data-var='time'>13:24</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'>24</var>, <var data-var='time'>09:47</var> UTC Identified - The issue has been identified and a fix is being implemented.

US Availability Zone - Some Segment Collector samples are lost

Aug <var data-var='date'> 9</var>, <var data-var='time'>17:43</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'> 9</var>, <var data-var='time'>15:03</var> UTC Identified - The Alerts Management page does not show the configured alerts, this is UI-only issue, the configurations are intact and alerts are operational

Alerts Management Page Shows No Alerts - Only Display Is Impacted, Alerts are Intact

Aug <var data-var='date'> 4</var>, <var data-var='time'>16:36</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'> 4</var>, <var data-var='time'>02:52</var> UTC Update - We are continuing to investigate this issue. Aug <var data-var='date'> 4</var>, <var data-var='time'>02:47</var> UTC Update - We are continuing to investigate this issue. Aug <var data-var='date'> 4</var>, <var data-var='time'>02:46</var> UTC Investigating - We're experiencing an issue with creation and initialization of new streams. The existing streams are operating correctly

Business Collectors - US Region - Creation of New Streams May Fail

May <var data-var='date'>18</var>, <var data-var='time'>13:30</var> UTC Resolved - This incident has been resolved. May <var data-var='date'>18</var>, <var data-var='time'>10:13</var> UTC Identified - We identified a lag in kinesis and mparticle streams, investigating. May <var data-var='date'>18</var>, <var data-var='time'>09:49</var> UTC Update - Some customers may have received false system notification on missing data There was no data loss or any impact on the service. May <var data-var='date'>18</var>, <var data-var='time'>09:45</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'>18</var>, <var data-var='time'>09:29</var> UTC Investigating - We are currently investigating this issue.

Data collectors issue in data center us-east-1

May <var data-var='date'>11</var>, <var data-var='time'>10:07</var> UTC Resolved - This incident has been resolved. May <var data-var='date'>11</var>, <var data-var='time'>08:32</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'>11</var>, <var data-var='time'>08:30</var> UTC Identified - The issue has been identified and a fix is being implemented. May <var data-var='date'>11</var>, <var data-var='time'>08:22</var> UTC Investigating - We are currently investigating this issue.

Is Anodot Revenue down?

Anodot Revenue status is UP

Lastest published events

Data collectors / Detection Issues

EU (Frankfurt) Region - Anomaly Detection and Anomaly Alerts not working

UI intermittently unavailable in US zone

Anodot Revenue Services, Components and Regions

Recent Anodot Revenue events

Data collectors / Detection Issues

EU (Frankfurt) Region - Anomaly Detection and Anomaly Alerts not working

UI intermittently unavailable in US zone

Infrastructure maintenance

Partial Data collection issue

US Region - Errors in Processing Incoming Metrics - Data Loss with High Probability