Jul <var data-var='date'> 7</var>, <var data-var='time'>22:34</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'> 7</var>, <var data-var='time'>20:31</var> UTC Monitoring - The connectivity issue with the Bogota Point of Presence (PoP) has been restored. We will continue to monitor to ensure no other issues. Jul <var data-var='date'> 7</var>, <var data-var='time'>19:51</var> UTC Identified - The connectivity issue was caused by a power outage from a third party provider. The provider is aware and currently working to restore power as soon as possible. Jul <var data-var='date'> 7</var>, <var data-var='time'>19:28</var> UTC Investigating - We are currently investigating connectivity issues with the Bogota Point of Presence (PoP). Devices will automatically route to the next nearest and fastest PoP and Static IP traffic will egress from the configured secondary location. Will update here as we the investigation continues.

Investigating Bogota Point of Presence (PoP) Connectivity Issue

Jul <var data-var='date'> 2</var>, <var data-var='time'>03:43</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'> 2</var>, <var data-var='time'>02:37</var> UTC Monitoring - It appears the 3rd party issues have been restored. Will continue monitoring until we are clear that it is fully resolved. Jul <var data-var='date'> 2</var>, <var data-var='time'>01:15</var> UTC Investigating - This issue is currently under investigation, but possibly related to a known Microsoft outage in the region - https://downdetector.com.au/status/microsoft-365/ We will post more information here as soon as we have better insight.

Investigating partial connectivity issues impacting Oceania region

Jun <var data-var='date'>24</var>, <var data-var='time'>00:43</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'>23</var>, <var data-var='time'>23:21</var> UTC Monitoring - The connectivity issue with the Miami Point of Presence (PoP) has been restored. We will continue to monitor to ensure no other issues, but SASE, tunnels, and Static IP traffic should be connecting now. Jun <var data-var='date'>23</var>, <var data-var='time'>23:00</var> UTC Identified - The connectivity issue was caused by a power outage from a third party provider. The provider is aware and currently working to restore power as soon as possible. Jun <var data-var='date'>23</var>, <var data-var='time'>22:43</var> UTC Investigating - We are currently investigating connectivity issues with the Miami Point of Presence (PoP). Devices will automatically route to the next nearest and fastest PoP and Static IP traffic will egress from the configured secondary location. Will update here as we the investigation continues.

Investigating Miami Point of Presence (PoP) Connectivity Issue

Jun <var data-var='date'>22</var>, <var data-var='time'>21:01</var> UTC Resolved - This incident is considered resolved as we see no impact on Todyl portal anymore, despite Cloudflare still continuing to monitor the issue. Jun <var data-var='date'>22</var>, <var data-var='time'>17:03</var> UTC Update - Cloudflare has successfully mitigated the majority of congestion and packet drops. We are continuing to monitor any possible impacts to the Todyl portal. For more information, please also refer to the Cloudflare status page - https://www.cloudflarestatus.com/. Jun <var data-var='date'>22</var>, <var data-var='time'>16:48</var> UTC Monitoring - majority of congestion and packet drops has been mitigated. Services are stable, with only minor residual impact remaining. A small number of intermittent errors may still be observed for services with origins in North America. We are putting this into a monitoring state. Jun <var data-var='date'>22</var>, <var data-var='time'>16:48</var> UTC Identified - Cloudflare has successfully mitigated the majority of congestion and packet drops. We are continuing to monitor any possible impacts to the Todyl portal. For more information, please also refer to the Cloudflare status page - https://www.cloudflarestatus.com/. Jun <var data-var='date'>22</var>, <var data-var='time'>16:45</var> UTC Monitoring - Cloudflare has successfully mitigated the majority of congestion and packet drops. We are continuing to monitor any possible impacts to the Todyl portal. For more information, please also refer to the Cloudflare status page - https://www.cloudflarestatus.com/. Jun <var data-var='date'>22</var>, <var data-var='time'>14:56</var> UTC Identified - The issue has been identified and awaiting on a Cloudflare resolution. For more information, please also refer to the Cloudflare status page - https://www.cloudflarestatus.com/. Jun <var data-var='date'>22</var>, <var data-var='time'>14:52</var> UTC Investigating - Some partners may be experiencing intermittent latency when using the Todyl portal.  Currently there is a partial Cloudflare system error being reported, leading to the intermittent errors on the Todyl portal.  We are monitoring the situation closely and getting updates from Cloudflare.  We will be be providing regular updates. For more information, please also refer to the Cloudflare status page - https://www.cloudflarestatus.com/.

Cloudflare error causing intermittent portal delays

Jun <var data-var='date'>16</var>, <var data-var='time'>04:29</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'>15</var>, <var data-var='time'>20:53</var> UTC Monitoring - Performance has returned to normal operation for the Secure Global Network (SASE) - Houston, HOU region. The system has self-recovered and affected users should no longer be experiencing elevated latency or reduced throughput. We are continuing to monitor the environment to ensure sustained stability. Jun <var data-var='date'>15</var>, <var data-var='time'>20:18</var> UTC Investigating - We are currently investigating performance degradation affecting the Secure Global Network (SASE) - Houston, HOU region. A subset of users connecting through this region may be experiencing elevated latency or reduced throughput. Our engineering team is actively investigating the root cause.

Secure Global Network (SASE) - Houston, HOU

Jun <var data-var='date'> 4</var>, <var data-var='time'>12:43</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'> 4</var>, <var data-var='time'>12:38</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Jun <var data-var='date'> 4</var>, <var data-var='time'>12:08</var> UTC Investigating - We are currently investigating an issue affecting Management Portal sign ins. A subset of users may experience intermittent failures when attempting to load the portal. All other Todyl services are operating normally. Our engineering team is actively investigating the root cause.

Management Portal Sign In Issues

Jun <var data-var='date'> 3</var>, <var data-var='time'>04:00</var> UTC Completed - The scheduled maintenance has been completed. Jun <var data-var='date'> 3</var>, <var data-var='time'>03:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jun <var data-var='date'> 2</var>, <var data-var='time'>22:47</var> UTC Scheduled - All SGN tunnels configured in the Chicago region will be offline during the maintenance window

Maintenance on SGN Tunnels in Chicago

May <var data-var='date'>22</var>, <var data-var='time'>05:34</var> UTC Resolved - This incident has been resolved. May <var data-var='date'>22</var>, <var data-var='time'>05:18</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'>22</var>, <var data-var='time'>05:14</var> UTC Identified - The issue has been identified and a fix is being implemented. May <var data-var='date'>22</var>, <var data-var='time'>02:47</var> UTC Investigating - We're investigating a delay in EDR policy updates affecting a small subset of endpoints. Impacted agents remain fully connected and continue enforcing current policies. Detection, prevention, and response functionality are unaffected.

Delayed EDR Policy Updates on Some Endpoints

May <var data-var='date'> 9</var>, <var data-var='time'>16:25</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 9</var>, <var data-var='time'>15:50</var> UTC Verifying - Verification is currently underway for the maintenance items. May <var data-var='date'> 9</var>, <var data-var='time'>12:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'> 7</var>, <var data-var='time'>02:44</var> UTC Scheduled - We will be performing scheduled maintenance on our cloud environment. During this window, the management portal and the authentication service may be intermittently unreachable.

Todyl Cloud Environment Scheduled Maintenance

May <var data-var='date'> 8</var>, <var data-var='time'>22:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 8</var>, <var data-var='time'>21:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>21:14</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - London - Amsterdam - Frankfurt Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

SASE Scheduled Maintenance: Tunnels

May <var data-var='date'> 8</var>, <var data-var='time'>13:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 8</var>, <var data-var='time'>12:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>21:12</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Sydney Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

May <var data-var='date'> 8</var>, <var data-var='time'>05:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 8</var>, <var data-var='time'>03:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>21:09</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Atlanta - Miami - Dallas - Houston - Denver Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

May <var data-var='date'> 7</var>, <var data-var='time'>21:00</var> UTC Resolved - Intermittent loading errors were seen on portal between 3PM MT and 4PM MT. The errors affected SIEM search, SIEM dashboards, Cases alert data and SOAR playbook execution. The errors have been addressed since.

Intermittent data load errors in portal

May <var data-var='date'> 7</var>, <var data-var='time'>16:36</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 7</var>, <var data-var='time'>14:36</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'> 7</var>, <var data-var='time'>14:35</var> UTC Scheduled - We are performing scheduled maintenance on the Mexico Point of Presence (PoP). During this window, agents will not be able to connect to this PoP and will be redirected to nearest available PoP. Rest of the PoPs are operational.

SASE Scheduled Maintenance: Mexico PoP

May <var data-var='date'> 7</var>, <var data-var='time'>04:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 7</var>, <var data-var='time'>02:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>21:02</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Newark - Washington DC - Chicago - Boston - Toronto Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

May <var data-var='date'> 5</var>, <var data-var='time'>06:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'> 5</var>, <var data-var='time'>05:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>20:58</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Los Angeles - Seattle - San Hose - Vancouver Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

May <var data-var='date'> 1</var>, <var data-var='time'>22:14</var> UTC Resolved - This incident has been resolved. May <var data-var='date'> 1</var>, <var data-var='time'>21:58</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'> 1</var>, <var data-var='time'>20:41</var> UTC Investigating - We are currently investigating an anomalous spike in alert volume affecting SIEM Processing & Alerting. Beginning at roughly 19:30 UTC a large number of cases were generated across a portion of partner tenants, associated with endpoint detection activity. The volume of alerts generated is inconsistent with expected baseline behavior. Our engineering team is actively investigating the issue.

SIEM Processing & Alerting — Alert Volume Anomaly

Apr <var data-var='date'>30</var>, <var data-var='time'>04:00</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>30</var>, <var data-var='time'>03:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>20:54</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Queretaro, Mexico Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

Apr <var data-var='date'>29</var>, <var data-var='time'>16:00</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>29</var>, <var data-var='time'>15:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>19:56</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Auckland - Singapore Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

Apr <var data-var='date'>28</var>, <var data-var='time'>15:16</var> UTC Resolved - Delays in case Creation and reporting issue is resolved.    System and MXDR team is fully up to speed in reviewing the delayed cases.  We are continuing to monitor but we consider the issue resolved. Apr <var data-var='date'>28</var>, <var data-var='time'>13:46</var> UTC Update - We are continuing to investigate this issue. Apr <var data-var='date'>28</var>, <var data-var='time'>13:33</var> UTC Update - We are continuing to investigate this issue. Apr <var data-var='date'>28</var>, <var data-var='time'>13:29</var> UTC Investigating - We identified and resolved an issue that caused delays in case creation and scheduled reporting. No alerts were lost, and MXDR is actively reviewing and addressing all open cases. We are maintaining heightened monitoring until all delayed cases have been fully processed, reviewed, and addressed, and all scheduled reports have been delivered.

Case creation

Apr <var data-var='date'>26</var>, <var data-var='time'>21:00</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>26</var>, <var data-var='time'>20:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>19:47</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Madris - Milan - Stockholm - Bucharest Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

Apr <var data-var='date'>26</var>, <var data-var='time'>04:00</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>26</var>, <var data-var='time'>03:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>19:43</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Bogota - New York - Sao Paulo Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

Apr <var data-var='date'>25</var>, <var data-var='time'>21:00</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>25</var>, <var data-var='time'>19:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>19:31</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Oslo - Tel Aviv - Istanbul - Paris - Johannesburg Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window. Thank you for your understanding and continued support.

Apr <var data-var='date'>25</var>, <var data-var='time'>15:16</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>25</var>, <var data-var='time'>15:02</var> UTC Verifying - Verification is currently underway for the maintenance items. Apr <var data-var='date'>25</var>, <var data-var='time'>14:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>21:23</var> UTC Scheduled - We will be performing scheduled maintenance on our cloud environment. During this window, the management portal and the authentication service may be intermittently unreachable.

Apr <var data-var='date'>25</var>, <var data-var='time'>15:00</var> UTC Completed - The scheduled maintenance has been completed. Apr <var data-var='date'>25</var>, <var data-var='time'>14:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Apr <var data-var='date'>24</var>, <var data-var='time'>19:40</var> UTC Scheduled - We will be performing scheduled maintenance on following Point of Presence (PoP). During this window, tunnels configured in these locations will be temporarily unavailable. - Tokyo - Hong Kong Customers with redundant configurations should not experience any disruption. Non-redundant tunnels terminating in these PoPs may experience downtime for part or all of the maintenance window.

Is Todyl down?

Todyl status is UP

Lastest published events

Investigating Bogota Point of Presence (PoP) Connectivity Issue

Investigating partial connectivity issues impacting Oceania region

Investigating Miami Point of Presence (PoP) Connectivity Issue

Todyl Services, Components and Regions

Recent Todyl events

Investigating Bogota Point of Presence (PoP) Connectivity Issue

Investigating partial connectivity issues impacting Oceania region

Investigating Miami Point of Presence (PoP) Connectivity Issue

Cloudflare error causing intermittent portal delays

Secure Global Network (SASE) - Houston, HOU

Management Portal Sign In Issues

Get your FREE account now,
5 monitors included!

Is Todyl down?

Todyl status is UP

Lastest published events

Investigating Bogota Point of Presence (PoP) Connectivity Issue

Investigating partial connectivity issues impacting Oceania region

Investigating Miami Point of Presence (PoP) Connectivity Issue

Todyl Services, Components and Regions

Recent Todyl events

Investigating Bogota Point of Presence (PoP) Connectivity Issue

Investigating partial connectivity issues impacting Oceania region

Investigating Miami Point of Presence (PoP) Connectivity Issue

Cloudflare error causing intermittent portal delays

Secure Global Network (SASE) - Houston, HOU

Management Portal Sign In Issues

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!