Oct <var data-var='date'> 3</var>, <var data-var='time'>04:16</var> UTC Resolved - Data ingest lag from Microsoft has been resolved. Oct <var data-var='date'> 2</var>, <var data-var='time'>14:12</var> UTC Update - We have observed a significant improvement in the data lag coming from Microsoft. We will continue to monitor to ensure this trend continues, and will share additional information if any becomes available. Oct <var data-var='date'> 1</var>, <var data-var='time'>20:50</var> UTC Update - Microsoft has identified the cause of the lag and expect the latency to drop over the next 24 hours. We will continue to monitor and share information as we receive it. Oct <var data-var='date'> 1</var>, <var data-var='time'>00:38</var> UTC Monitoring - We've identified an intermittent lag in events from Microsoft. We are working with our partners at Microsoft and providing them with information on any affected customers. Sep <var data-var='date'>30</var>, <var data-var='time'>22:29</var> UTC Investigating - We have noticed a delay in events being sent to us by Microsoft and are investigating the issue.

ITDR Event Ingestion Delay

Oct <var data-var='date'> 1</var>, <var data-var='time'>09:05</var> UTC Resolved - This incident has been resolved. Oct <var data-var='date'> 1</var>, <var data-var='time'>05:43</var> UTC Update - We are continuing to monitor for any further issues. Oct <var data-var='date'> 1</var>, <var data-var='time'>05:41</var> UTC Update - We are continuing to monitor for any further issues. Oct <var data-var='date'> 1</var>, <var data-var='time'>05:40</var> UTC Monitoring - A fix has been implemented and signal generation + detections are returning to normal. SOC response continues to be delayed. Oct <var data-var='date'> 1</var>, <var data-var='time'>05:35</var> UTC Identified - The issue has been identified and a fix is being implemented. Oct <var data-var='date'> 1</var>, <var data-var='time'>05:02</var> UTC Investigating - We are experiencing delayed signal generation affecting our SOC response time and are currently investigating the issue.

Signal generation and SOC response delay

Sep <var data-var='date'>20</var>, <var data-var='time'>21:18</var> UTC Resolved - Microsoft has deployed additional fixes to their infrastructure, and our internal telemetry is back to showing normal ingest timing for events. This incident is now resolved. Sep <var data-var='date'>17</var>, <var data-var='time'>15:18</var> UTC Update - Our partners at Microsoft have deployed a fix for the affected customers which will roll out over the remainder of this week and into the weekend. We will continue working with them to verify the fix is working as intended. Sep <var data-var='date'>16</var>, <var data-var='time'>14:53</var> UTC Update - Our partners at Microsoft are in the process of rolling out a fix across the affected tenants. We are monitoring the effect and providing information to them on the effectiveness of this solution. Sep <var data-var='date'>14</var>, <var data-var='time'>18:49</var> UTC Update - We are continuing to work with Microsoft to resolve the intermittent delays for a small set of affected customers. Sep <var data-var='date'>11</var>, <var data-var='time'>15:55</var> UTC Update - We are continuing to provide information about any affected customers to our partners at Microsoft. Intermittent delays continue to show up for a small set of affected customers. Sep <var data-var='date'> 1</var>, <var data-var='time'>22:09</var> UTC Monitoring - We're continuing to work with our partners at Microsoft, who have identified the cause of the delay and are in the process of implementing a fix. We are still seeing intermittent delays of up to 4 hours for some customers. We will continue to provide updates as soon as more information becomes available. Aug <var data-var='date'>31</var>, <var data-var='time'>17:37</var> UTC Identified - We've identified that the cause of our ingestion delay is from a degradation in Microsoft's M365 APIs. No GWS customer are affected. We're working through our partner channels to get this resolved, and will update this page as soon as we know more information. Aug <var data-var='date'>31</var>, <var data-var='time'>16:53</var> UTC Investigating - We have noticed a slight delay in our ITDR ingestion pipeline, and are investigating the cause.

ITDR Ingestion Delay

Aug <var data-var='date'>20</var>, <var data-var='time'>12:30</var> UTC Resolved - We're no longer experiencing delays from our upstream provider. This incident has been resolved. Aug <var data-var='date'>20</var>, <var data-var='time'>01:24</var> UTC Update - While we have seen an improvement in overall ingestion timing, we are still seeing delayed events from our upstream provider. We're currently working with them to resolve the issue. We will continue monitoring, and posting updates as we have them. Aug <var data-var='date'>19</var>, <var data-var='time'>21:14</var> UTC Update - We are seeing improvements in our ingestion timing and continuing to monitor for a resolution from our upstream provider. We will continue to publish updates as we have them. Aug <var data-var='date'>19</var>, <var data-var='time'>20:10</var> UTC Update - We are still seeing delays from our upstream provider leading to ITDR ingestion delay and are continuing to monitor the situation. Aug <var data-var='date'>19</var>, <var data-var='time'>19:02</var> UTC Monitoring - We've identified an issue with an upstream provider that is resulting in ITDR ingestion latency and are actively monitoring the situation.

Aug <var data-var='date'>11</var>, <var data-var='time'>15:52</var> UTC Resolved - This incident has been resolved. Aug <var data-var='date'>10</var>, <var data-var='time'>20:28</var> UTC Update - We've identified an issue with an upstream provider that is resulting in ITDR ingestion latency and errors in the Huntress portal and are working on mitigations. Aug <var data-var='date'>10</var>, <var data-var='time'>20:21</var> UTC Identified - The issue has been identified and a fix is being implemented. Aug <var data-var='date'>10</var>, <var data-var='time'>15:41</var> UTC Investigating - We are currently investigating this issue.

Aug <var data-var='date'> 5</var>, <var data-var='time'>22:18</var> UTC Resolved - The upstream latency issues are resolving. Huntress ingestion is operating normally. Aug <var data-var='date'> 5</var>, <var data-var='time'>15:27</var> UTC Monitoring - This is a Microsoft latency issue that is outside of our control. We are continuing to monitor the situation and will update when the feed recovers. Aug <var data-var='date'> 5</var>, <var data-var='time'>14:17</var> UTC Investigating - We are currently investigating this issue.

Jul <var data-var='date'>23</var>, <var data-var='time'>20:03</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'>23</var>, <var data-var='time'>18:58</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.

Elevated ITDR Signal Latency

Jul <var data-var='date'> 2</var>, <var data-var='time'>02:28</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'> 2</var>, <var data-var='time'>01:55</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'> 2</var>, <var data-var='time'>01:39</var> UTC Update - We are continuing to investigate this issue. Jul <var data-var='date'> 2</var>, <var data-var='time'>00:57</var> UTC Investigating - We are currently investigating an issue affecting the management portal for users located in Australia, New Zealand, and the surrounding Oceania region

Page Load Failures in Australia, New Zealand

Jun <var data-var='date'> 4</var>, <var data-var='time'>02:51</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'> 4</var>, <var data-var='time'>02:39</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jun <var data-var='date'> 4</var>, <var data-var='time'>02:10</var> UTC Identified - The issue has been identified and a fix is being implemented. Jun <var data-var='date'> 4</var>, <var data-var='time'>01:38</var> UTC Investigating - We are investigating delays in signal processing for ITDR, EDR and SIEM.

Signal processing delays

May <var data-var='date'>29</var>, <var data-var='time'>02:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'>29</var>, <var data-var='time'>00:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'>27</var>, <var data-var='time'>14:31</var> UTC Scheduled - During this maintenance window, support@curricula.com, support@mycurricula.com and support@getcurricula.com will be unavailable. If you need to reach our support team, please contact us at support@huntresslabs.com or visit https://support.huntress.io/hc/en-us

Curricula Support Email Maintenance

May <var data-var='date'>21</var>, <var data-var='time'>17:16</var> UTC Completed - This maintenance window has been postponed to Wednesday next week. May <var data-var='date'>19</var>, <var data-var='time'>19:15</var> UTC Scheduled - We will be undergoing scheduled maintenance during this time.

Database Maintenance

May <var data-var='date'>15</var>, <var data-var='time'>01:43</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'>15</var>, <var data-var='time'>00:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'>14</var>, <var data-var='time'>17:07</var> UTC Scheduled - We will be undergoing scheduled maintenance during this time.

May <var data-var='date'>14</var>, <var data-var='time'>15:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'>14</var>, <var data-var='time'>12:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'>12</var>, <var data-var='time'>23:15</var> UTC Scheduled - We will be undergoing scheduled maintenance during this time.

May <var data-var='date'> 8</var>, <var data-var='time'>02:27</var> UTC Resolved - We have shifted resources off of the affected AWS zone and are not seeing any further issues. May <var data-var='date'> 8</var>, <var data-var='time'>02:10</var> UTC Monitoring - We have taken action to mitigate the effects of the AWS outage and are continuing to monitor. May <var data-var='date'> 8</var>, <var data-var='time'>02:09</var> UTC Identified - The issue has been identified and a fix is being implemented. May <var data-var='date'> 8</var>, <var data-var='time'>01:24</var> UTC Update - Amazon Web Services is experiencing an outage which is impacting our ability to remediate. Investigation is ongoing. May <var data-var='date'> 8</var>, <var data-var='time'>00:49</var> UTC Investigating - We are currently investigating this issue.

Portal Web Errors and Agent Communication Delays

May <var data-var='date'> 3</var>, <var data-var='time'>01:38</var> UTC Resolved - This incident has been resolved. May <var data-var='date'> 2</var>, <var data-var='time'>23:20</var> UTC Update - We're currently experiencing an issue with our outbound calling provider, which is temporarily preventing our automated incident notification phone calls and voice-based verification/MFA codes from going out. SMS notifications are still being delivered as expected. A workaround has been implemented while we continue to work with the vendor on resolution. May <var data-var='date'> 2</var>, <var data-var='time'>21:36</var> UTC Update - We are continuing to work on a fix for this issue. May <var data-var='date'> 2</var>, <var data-var='time'>21:22</var> UTC Identified - The issue has been identified and a fix is being implemented.

Automated Phone Notification Outage

Apr <var data-var='date'>25</var>, <var data-var='time'>12:01</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>25</var>, <var data-var='time'>10:20</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>25</var>, <var data-var='time'>08:42</var> UTC Update - We are continuing to work on a fix for this issue. Apr <var data-var='date'>25</var>, <var data-var='time'>03:52</var> UTC Update - We are continuing to work on a fix for this issue, and we're seeing positive changes. Apr <var data-var='date'>25</var>, <var data-var='time'>02:22</var> UTC Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'>25</var>, <var data-var='time'>02:03</var> UTC Investigating - We are currently investigating an issue causing occasional timeouts across services.

Portal Degradation

Apr <var data-var='date'>21</var>, <var data-var='time'>19:51</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>21</var>, <var data-var='time'>17:01</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>21</var>, <var data-var='time'>16:45</var> UTC Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'>21</var>, <var data-var='time'>16:43</var> UTC Investigating - We are currently investigating this issue.

Event processing delay

Apr <var data-var='date'>16</var>, <var data-var='time'>21:15</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>16</var>, <var data-var='time'>17:33</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>16</var>, <var data-var='time'>17:28</var> UTC Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'>16</var>, <var data-var='time'>17:28</var> UTC Investigating - We are currently investigating this issue.

Services degraded

Apr <var data-var='date'>10</var>, <var data-var='time'>20:46</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>10</var>, <var data-var='time'>20:14</var> UTC Update - We are continuing to monitor for any further issues. Apr <var data-var='date'>10</var>, <var data-var='time'>20:11</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>10</var>, <var data-var='time'>20:06</var> UTC Identified - The issue has been identified and a fix is being implemented.

Web Portal Not Loading

Apr <var data-var='date'> 1</var>, <var data-var='time'>17:34</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'> 1</var>, <var data-var='time'>17:17</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'> 1</var>, <var data-var='time'>17:15</var> UTC Update - We are continuing to work on a fix for this issue. Apr <var data-var='date'> 1</var>, <var data-var='time'>17:14</var> UTC Identified - The issue has been identified and a fix is being implemented.

Delay in Notification including Emails / PSA

Mar <var data-var='date'>28</var>, <var data-var='time'>04:03</var> UTC Completed - The scheduled maintenance has been completed. Mar <var data-var='date'>28</var>, <var data-var='time'>03:30</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Mar <var data-var='date'>28</var>, <var data-var='time'>02:50</var> UTC Scheduled - We will be undergoing scheduled maintenance at this time. Users may be logged out of the web portal several times during this window.

Database maintenance

Mar <var data-var='date'>19</var>, <var data-var='time'>16:57</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'>19</var>, <var data-var='time'>16:46</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Mar <var data-var='date'>19</var>, <var data-var='time'>16:45</var> UTC Identified - The issue has been identified and a fix is being implemented. Mar <var data-var='date'>19</var>, <var data-var='time'>16:28</var> UTC Update - We are continuing to investigate this issue. Mar <var data-var='date'>19</var>, <var data-var='time'>16:24</var> UTC Investigating - We are currently investigating this issue.

Portal Web Errors

Mar <var data-var='date'>19</var>, <var data-var='time'>03:36</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'>19</var>, <var data-var='time'>02:24</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Mar <var data-var='date'>19</var>, <var data-var='time'>02:08</var> UTC Identified - The issue has been identified and a fix is being implemented.

Rio Processing Delay

Mar <var data-var='date'>19</var>, <var data-var='time'>00:46</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'>18</var>, <var data-var='time'>19:32</var> UTC Monitoring - A fix has been implemented. We are working on data replay.

ITDR Detections Are Catching Up

Mar <var data-var='date'>17</var>, <var data-var='time'>14:26</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'>17</var>, <var data-var='time'>13:59</var> UTC Update - We are continuing to monitor for any further issues. Mar <var data-var='date'>17</var>, <var data-var='time'>13:48</var> UTC Update - Negative side effects of the mitigation are being addressed. Mar <var data-var='date'>17</var>, <var data-var='time'>13:17</var> UTC Monitoring - An issue has been identified within our infrastructure preventing new services from being created. A mitigation is being deploying now.

Is Huntress down?

Huntress status is UP

Lastest published events

ITDR Event Ingestion Delay

Signal generation and SOC response delay

ITDR Ingestion Delay

Huntress Services, Components and Regions

Recent Huntress events

ITDR Event Ingestion Delay

Signal generation and SOC response delay

ITDR Ingestion Delay

ITDR Ingestion Delay

ITDR Ingestion Delay

ITDR Ingestion Delay