Jun <var data-var='date'>25</var>, <var data-var='time'>01:12</var> EDT Resolved - The remaining out-of-sync user records are now back in sync. Jun <var data-var='date'>24</var>, <var data-var='time'>23:29</var> EDT Update - Synchronization has continued to catch up. We are continuing to monitor the few out of sync changes that are remaining. Jun <var data-var='date'>24</var>, <var data-var='time'>20:09</var> EDT Monitoring - Provisioning has caught up for user creation, and new directory activity is syncing normally. We are still working through and monitoring the remaining synchronization catch-up, which is not yet fully complete. Another update to follow. Jun <var data-var='date'>24</var>, <var data-var='time'>13:37</var> EDT Update - We have corrected the affected directories, and they are back to a provisioning status. Jun <var data-var='date'>24</var>, <var data-var='time'>12:47</var> EDT Identified - The underlying issue that has lead to a delay in provisioning has been identified, and a fix is being prepared Jun <var data-var='date'>22</var>, <var data-var='time'>17:36</var> EDT Investigating - We are seeing a delay in Directory Provisioning for Authkit, for any directories created before June 22, 21:36:17 UTC. Any directories created after this time are working as expected.

Degraded performance for Directory Provisioning for Authkit

Jun <var data-var='date'>22</var>, <var data-var='time'>14:01</var> EDT Resolved - From 13:35 UTC to 14:24 UTC users connecting through our Chicago (ORD) edge node may have experienced elevated API error rates. We have resolved the issue and are monitoring. Jun <var data-var='date'>22</var>, <var data-var='time'>10:42</var> EDT Monitoring - We detected elevated errors from our Chicago (ORD) edge node. Errors have since recovered and we're continuing to monitor. Jun <var data-var='date'>22</var>, <var data-var='time'>10:03</var> EDT Investigating - We are investigating elevated error rates across our API. We apologize for the inconvenience and will share an update once we have more information.

Elevated API Errors

Jun <var data-var='date'> 9</var>, <var data-var='time'>05:00</var> EDT Resolved - From 9:43 to 10:02 UTC SSO endpoints experienced increased latency and intermittent timeouts. We have resolved the issue and are monitoring.

SSO Request Latency

Jun <var data-var='date'> 4</var>, <var data-var='time'>20:49</var> EDT Resolved - The backlog of messages have been sent and system performance has returned to its expected state. Sending delays occurred between 2:50pm – 5:18pm PT. Jun <var data-var='date'> 4</var>, <var data-var='time'>20:34</var> EDT Monitoring - Email delivery has recovered. We're continuing to monitor. Jun <var data-var='date'> 4</var>, <var data-var='time'>20:08</var> EDT Update - Our email provider is continuing to work through the backlog and has scaled up to speed up the process. Jun <var data-var='date'> 4</var>, <var data-var='time'>19:22</var> EDT Identified - Email delivery for customers without custom domains is delayed due to an issue with our upstream service provider.

Email delivery delays

May <var data-var='date'>28</var>, <var data-var='time'>22:46</var> EDT Resolved - The issue affecting the Events API GET /events endpoint has been resolved. The fix has been deployed, and affected requests are now processing normally. May <var data-var='date'>28</var>, <var data-var='time'>22:30</var> EDT Identified - We have identified the cause of the elevated errors affecting the GET /events API and are working on a resolution. We’ll share another update soon.

Elevated Events API Errors

May <var data-var='date'>22</var>, <var data-var='time'>14:58</var> EDT Resolved - This incident has been resolved. May <var data-var='date'>22</var>, <var data-var='time'>14:55</var> EDT Monitoring - A fix has been deployed for the issue affecting static asset loading on the WorkOS Dashboard, Admin Portal, Docs, and hosted result pages. We are continuing to monitor for full recovery. May <var data-var='date'>22</var>, <var data-var='time'>14:32</var> EDT Update - We're aware of a styling issue currently affecting the WorkOS Dashboard. CSS assets are failing to load, leaving the Dashboard in an unstyled and difficult-to-use state. Underlying APIs, authentication, and data are not impacted - this is limited to the Dashboard UI. We're investigating and will update shortly. May <var data-var='date'>22</var>, <var data-var='time'>14:29</var> EDT Investigating - We are investigating an issue with our Dashboard. We will share an update as soon as possible.

Dashboard and Docs unavailable

May <var data-var='date'>19</var>, <var data-var='time'>18:00</var> EDT Resolved - The webhooks queue backlog has been delivered. Thank you for your patience. May <var data-var='date'>19</var>, <var data-var='time'>17:43</var> EDT Identified - We are experiencing delays delivering a subset of webhooks. We have identified the issue and working on a remediation.

Webhook delivery delays

May <var data-var='date'>14</var>, <var data-var='time'>11:12</var> EDT Resolved - This issue is now resolved. May <var data-var='date'>14</var>, <var data-var='time'>11:10</var> EDT Monitoring - We experienced errors in our OIDC connections. The issue is now resolved.

OIDC Errors

May <var data-var='date'>12</var>, <var data-var='time'>16:29</var> EDT Resolved - Newly-sent emails from WorkOS are now displaying properly in Gmail again. May <var data-var='date'>12</var>, <var data-var='time'>15:43</var> EDT Identified - We've identified the issue and are applying a fix. May <var data-var='date'>12</var>, <var data-var='time'>15:37</var> EDT Update - We are investigating reports of AuthKit emails rendering with no copy in some email clients. May <var data-var='date'>12</var>, <var data-var='time'>15:14</var> EDT Investigating - We are investigating reports of some AuthKit emails, such as invitations, being delivered with empty copy.

Emails being delivered with empty copy

May <var data-var='date'>12</var>, <var data-var='time'>14:41</var> EDT Resolved - This has been resolved. May <var data-var='date'>12</var>, <var data-var='time'>14:21</var> EDT Monitoring - Our team is seeing webhook delivery recover and we are continuing to monitor the situation. May <var data-var='date'>12</var>, <var data-var='time'>14:04</var> EDT Identified - Our team has identified the issue and is implementing mitigations. May <var data-var='date'>12</var>, <var data-var='time'>13:59</var> EDT Investigating - We are seeing delays delivering webhooks. Our team is currently investigating.

Elevated Webhook Latency

May <var data-var='date'> 8</var>, <var data-var='time'>20:45</var> EDT Resolved - This incident has been resolved. May <var data-var='date'> 8</var>, <var data-var='time'>20:35</var> EDT Identified - We are investigating reports of 504 Gateway Timeout errors affecting logins to the WorkOS Dashboard. Users may be unable to sign in at dashboard.workos.com via SSO or Google sign-in. The WorkOS API and AuthKit are unaffected. We will share another update shortly.

Dashboard login errors

May <var data-var='date'> 5</var>, <var data-var='time'>15:48</var> EDT Resolved - We have recovered from any delays delivering webhooks. May <var data-var='date'> 5</var>, <var data-var='time'>14:51</var> EDT Monitoring - Webhook delivery for Authkit is delayed but recovering. We are monitoring the situation. May <var data-var='date'> 5</var>, <var data-var='time'>12:45</var> EDT Identified - We are investigating a delay in sending Authkit webhooks. We will share an update once we have more information.

Webhook delivery delayed

May <var data-var='date'> 1</var>, <var data-var='time'>16:51</var> EDT Resolved - The issue has been resolved. The WorkOS Dashboard is fully operational. May <var data-var='date'> 1</var>, <var data-var='time'>16:40</var> EDT Monitoring - We've identified and addressed the issue. We'll continue to monitor. May <var data-var='date'> 1</var>, <var data-var='time'>16:28</var> EDT Investigating - We are investigating an issue with our Dashboard. We will share an update as soon as possible.

Elevated Dashboard Errors

Apr <var data-var='date'>29</var>, <var data-var='time'>22:23</var> EDT Resolved - This has been resolved. Apr <var data-var='date'>29</var>, <var data-var='time'>21:56</var> EDT Monitoring - Email delivery is returning to normal. Our team is continuing to monitor. Apr <var data-var='date'>29</var>, <var data-var='time'>21:30</var> EDT Identified - Our team has identified the issue and is actively working on mitigations to restore email delivery for the affected subset of users. Apr <var data-var='date'>29</var>, <var data-var='time'>21:00</var> EDT Investigating - We’re seeing email delivery delays affecting some users. Our team is investigating.

Email Delivery Delays

Apr <var data-var='date'>20</var>, <var data-var='time'>15:17</var> EDT Resolved - The issue has been resolved, and SSO login performance has returned to normal. We will continue to monitor closely to ensure stability. Apr <var data-var='date'>20</var>, <var data-var='time'>15:16</var> EDT Identified - Between 14:40 and 14:53 UTC, SSO endpoints experienced increased latency and intermittent timeouts. The issue was caused by a temporary reduction in available capacity due to an underlying infrastructure event, which led to a brief period of degraded performance

SSO API Latency Degradation Caused by Transient Capacity Constraints

Apr <var data-var='date'>16</var>, <var data-var='time'>19:10</var> EDT Resolved - The issue has been resolved. 429 error rates have returned to normal levels for the affected endpoint. Apr <var data-var='date'>16</var>, <var data-var='time'>16:59</var> EDT Monitoring - We’ve deployed a fix and are seeing early signs of recovery, with 429 error rates beginning to decline. We’re continuing to monitor closely to ensure the endpoint fully stabilizes and will provide further updates as needed. Apr <var data-var='date'>16</var>, <var data-var='time'>16:54</var> EDT Investigating - We’re currently investigating an elevated rate of HTTP 429 (Too Many Requests) responses impacting the /directory_users endpoint. Mitigations are in progress, and we’ll provide an update shortly.

Increased 429 Errors on /directory_users endpoint

Mar <var data-var='date'>27</var>, <var data-var='time'>19:15</var> EDT Resolved - We think the impact from the issue is over. Mar <var data-var='date'>27</var>, <var data-var='time'>18:03</var> EDT Monitoring - The custom branding code editor and JWT template editor functionality has been restored. Mar <var data-var='date'>27</var>, <var data-var='time'>17:57</var> EDT Investigating - We are investigating an issue with our Dashboard. The custom branding editor's custom CSS code editor and the JWT editor are currently not available. We will share an update as soon as possible.

Mar <var data-var='date'>17</var>, <var data-var='time'>10:30</var> EDT Resolved - This incident has been resolved. Mar <var data-var='date'>17</var>, <var data-var='time'>10:27</var> EDT Monitoring - Services have returned to normal operation Mar <var data-var='date'>17</var>, <var data-var='time'>10:12</var> EDT Investigating - We’re investigating elevated API errors and latency.

Feb <var data-var='date'>24</var>, <var data-var='time'>06:16</var> EST Resolved - All services have returned to normal. This incident is resolved. Feb <var data-var='date'>24</var>, <var data-var='time'>05:39</var> EST Identified - Our team has identified the issue and is implementing a fix. Feb <var data-var='date'>24</var>, <var data-var='time'>05:11</var> EST Investigating - We’re investigating elevated API errors and latency.

Jan <var data-var='date'>27</var>, <var data-var='time'>14:42</var> EST Resolved - This incident has been resolved. Jan <var data-var='date'>27</var>, <var data-var='time'>14:35</var> EST Update - Error rates and latencies returned to normal at 11:02am PT, but we are continuing to monitor until the upstream provider resolves the incident on their side. Jan <var data-var='date'>27</var>, <var data-var='time'>14:11</var> EST Monitoring - Our upstream provider has identified the problem and working on a fix. We continue to monitor. Jan <var data-var='date'>27</var>, <var data-var='time'>13:57</var> EST Investigating - We are seeing elevated error rates across our services. We will share an update as soon as possible.

Elevated Error Rates

Jan <var data-var='date'>23</var>, <var data-var='time'>10:16</var> EST Resolved - The issue impacting outbound email delivery to Outlook and Microsoft 365 inboxes has been resolved. Delivery is now operating normally across all providers. Jan <var data-var='date'>22</var>, <var data-var='time'>18:44</var> EST Monitoring - We are experiencing degraded delivery of outbound emails from WorkOS to Outlook and Microsoft 365 inboxes. Some messages may be delayed or not delivered. Delivery to non-Outlook providers is unaffected. We are monitoring the situation and will share updates as they become available.

Degraded Email Delivery to Microsoft Outlook Inboxes

Jan <var data-var='date'> 3</var>, <var data-var='time'>09:20</var> EST Resolved - Webhook delivery latencies have returned to normal levels. Please contact support if you are still experiencing delays. Jan <var data-var='date'> 2</var>, <var data-var='time'>21:10</var> EST Monitoring - Our team is actively monitoring webhook processing and we're seeing steady progress as our increased processing capacity takes effect. The backlog of pending webhooks is decreasing, and we expect continued improvement over the next several hours. Jan <var data-var='date'> 2</var>, <var data-var='time'>20:03</var> EST Identified - We've identified the issue causing webhook delivery delays and have increased processing capacity. Jan <var data-var='date'> 2</var>, <var data-var='time'>15:48</var> EST Investigating - We are investigating delays in webhook event delivery affecting some webhook event types. Directory Sync and Magic Auth code functionality are operating normally and are not impacted.

Delayed Webhook Deliveries

Dec <var data-var='date'>29</var>, <var data-var='time'>09:13</var> EST Resolved - Our DDOS protection mitigated an attack but also resulted in false positives causing end users to see intermittent 404s. We have adjusted the sensitivity of these rules to prevent further false positives. Dec <var data-var='date'>29</var>, <var data-var='time'>07:26</var> EST Monitoring - We are seeing degraded performance of AuthKit resulting in intermittent errors. The error rate has improved and we are monitoring.

Degraded AuthKit Behavior

Dec <var data-var='date'> 5</var>, <var data-var='time'>04:32</var> EST Resolved - This incident has been resolved. Dec <var data-var='date'> 5</var>, <var data-var='time'>04:21</var> EST Monitoring - Our upstream provider has recovered. We continue to monitor. Dec <var data-var='date'> 5</var>, <var data-var='time'>03:57</var> EST Investigating - We are seeing elevated error rates across our services. We will share an update as soon as possible.

Dec <var data-var='date'> 3</var>, <var data-var='time'>14:55</var> EST Resolved - A fix has been tested, deployed and verified. Dec <var data-var='date'> 3</var>, <var data-var='time'>13:51</var> EST Monitoring - We have reverted a deployed change that affected the impersonation reason field, and service has been restored. We will monitor closely. Dec <var data-var='date'> 3</var>, <var data-var='time'>12:57</var> EST Identified - And issue in the dashboard has prevented users from using the impersonation feature. The issue has been identified, and a fix is being deployed

Is WorkOS down?

WorkOS status is UP

Lastest published events

Degraded performance for Directory Provisioning for Authkit

Elevated API Errors

SSO Request Latency

WorkOS Services, Components and Regions

Recent WorkOS events

Degraded performance for Directory Provisioning for Authkit

Elevated API Errors

SSO Request Latency

Email delivery delays

Elevated Events API Errors

Dashboard and Docs unavailable

Get your FREE account now,
5 monitors included!

Is WorkOS down?

WorkOS status is UP

Lastest published events

Degraded performance for Directory Provisioning for Authkit

Elevated API Errors

SSO Request Latency

WorkOS Services, Components and Regions

Recent WorkOS events

Degraded performance for Directory Provisioning for Authkit

Elevated API Errors

SSO Request Latency

Email delivery delays

Elevated Events API Errors

Dashboard and Docs unavailable

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!