THIS IS A SCHEDULED EVENT Dec <var data-var='date'>31</var>, <var data-var='time'>01:00</var> - <var data-var='time'>09:00</var> PST Aug <var data-var='date'>25</var>, <var data-var='time'>11:43</var> PDT Scheduled - As part of our commitment to reliability and security, LaunchDarkly will retire all API versions except 20240415 on December 31, 2026. After this date, all requests made to retired versions will stop working. To avoid disruption, customers must update integrations and workflows to use version 20240415 before the retirement date. We previously announced the end of life (EOL) for older versions as of April 30, 2025. They have remained available during a grace period, which will end on December 31, 2026 For full details, see our API version changelog: https://launchdarkly.com/docs/api#api-version-changelog Upgrading now ensures uninterrupted access and provides immediate improvements in performance and functionality. We recognize this may be a large undertaking, which is why we are providing extended notice. If you need guidance, please reach out to your account team or contact our support team.

Planned Upcoming API Versions Retirement

Jul <var data-var='date'> 7</var>, <var data-var='time'>08:10</var> PDT Resolved - Event ingestion is operating normally. Experimentation results, Guarded Rollouts, and in-product data charts were delayed ~5 min between 7:13–7:53 AM PT. No data was lost. We apologize for any inconvenience.

Delays in Experimentation and Guarded Rollout results

Jul <var data-var='date'> 6</var>, <var data-var='time'>14:21</var> PDT Resolved - All error rates have normalized. Jul <var data-var='date'> 6</var>, <var data-var='time'>14:09</var> PDT Monitoring - We have rolled out a fix and error rates are returning to normal Jul <var data-var='date'> 6</var>, <var data-var='time'>14:08</var> PDT Update - We are continuing to investigate this issue. Jul <var data-var='date'> 6</var>, <var data-var='time'>15:30</var> PDT Investigating - At 13:21 Pacific Time, we experienced a high volume of error alerts affecting users' ability to interact with the LaunchDarkly application including logging in and UI / API responsiveness. We quickly identified the root cause relating to flag delivery. We are expecting errors decreasing now.

High volume of error alerts related to flag delivery network.

Jul <var data-var='date'> 6</var>, <var data-var='time'>08:52</var> PDT Resolved - This incident has been resolved. Jul <var data-var='date'> 6</var>, <var data-var='time'>07:45</var> PDT Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'> 6</var>, <var data-var='time'>07:33</var> PDT Investigating - Some customers are not receiving account verification emails. Our team is actively working to mitigate this issue.

Verification emails not being sent

Jul <var data-var='date'> 3</var>, <var data-var='time'>09:06</var> PDT Resolved - This incident has been resolved. Jul <var data-var='date'> 3</var>, <var data-var='time'>08:13</var> PDT Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'> 3</var>, <var data-var='time'>07:29</var> PDT Investigating - We are investigating elevated error rates on our event data ingestion endpoints affecting production customers, between approximately 7:00 AM PDT and 7:15 AM PDT. During this window, a portion of feature flag, experimentation, and AI/agent event data may have been rejected or delayed. Flag delivery and evaluation are not affected. SDKs continue to serve flags normally. Observability ingest is not affected.

Elevated error rates on event data ingestion endpoints

Jun <var data-var='date'>17</var>, <var data-var='time'>18:30</var> PDT Resolved - We're now seeing restored access in the affected regions. Jun <var data-var='date'>17</var>, <var data-var='time'>18:23</var> PDT Update - We've identified the source of the problem for areas that were still seeing issues with the LaunchDarkly.com and docs and have a solution in place. We are now monitoring to ensure everything is resolved. Jun <var data-var='date'>17</var>, <var data-var='time'>16:56</var> PDT Update - Incident seems to be resolved for the majority of users, however, we are continuing to see some issues for in the San Jose and Los Angeles region. We're working to remediate for these regions. Jun <var data-var='date'>17</var>, <var data-var='time'>12:59</var> PDT Monitoring - A fix has been implemented and monitoring the results. Jun <var data-var='date'>17</var>, <var data-var='time'>12:43</var> PDT Identified - LaunchDarkly web application and flag delivery unaffected, but documentation and LaunchDarkly.com are unavailable for some users. A fix is on the way.

LaunchDarkly.com and LaunchDarkly.com/docs unavailable for some

Jun <var data-var='date'>15</var>, <var data-var='time'>10:38</var> PDT Resolved - The issue with Observability data ingest for OpenTelemetry data has been resolved. Impacted services have returned to normal operation. Jun <var data-var='date'>15</var>, <var data-var='time'>10:33</var> PDT Monitoring - The issue with Observability data ingest for OpenTelemetry data has been identified and a fix has been implemented. We are continuing to monitor the performance of impacted services. We will continue to update this page until it is resolved. Observability signals (errors / logs / traces / OTel metrics) are currently delayed by <1 minutes on average. Jun <var data-var='date'>15</var>, <var data-var='time'>10:02</var> PDT Identified - Some customers are experiencing delayed Observability data ingest for OpenTelemetry data. We have identified the issue and are continuing our work to resolve it. Observability signals (errors / logs / traces / OTel metrics) are currently delayed by ~6 minutes on average. Data ingest delays began at 10:02am and are improving as of 10:16am PT.

Observability OpenTelemetry Ingest Delay

Jun <var data-var='date'>13</var>, <var data-var='time'>06:52</var> PDT Resolved - This incident has been resolved. Jun <var data-var='date'>12</var>, <var data-var='time'>15:15</var> PDT Update - We have made major progress. We identified that a small percentage of event data was lost during two windows: June 9, 11:41 - 11:59 UTC and June 10, 23:55 UTC - June 11, 00:21 UTC. Some event-based features may show incomplete data while backfills and recovery continue. We are continuing to monitor and will provide another update when recovery is complete. Jun <var data-var='date'>11</var>, <var data-var='time'>08:37</var> PDT Update - We have made major progress. We identified that a small percentage of event data was lost during two windows: June 9, 11:41 - 11:59 UTC and June 10, 23:55 UTC - June 11, 00:21 UTC. Some event-based features may show incomplete data while backfills and recovery continue. We are continuing to monitor and will provide another update when recovery is complete. Jun <var data-var='date'>10</var>, <var data-var='time'>22:14</var> PDT Update - We have made partial progress. We identified that a small percentage of event data was lost during two windows: June 9, 11:41 - 11:59 UTC and June 10, 23:55 UTC - June 11, 00:21 UTC. Some event-based features may show incomplete data while backfills and recovery continue. We are continuing to monitor and will provide another update when recovery is complete. Jun <var data-var='date'>10</var>, <var data-var='time'>21:12</var> PDT Update - We have made partial progress. We have identified a brief period of event data loss, and some event-based features may show incomplete data while we continue recovery. We are continuing to monitor. Jun <var data-var='date'>10</var>, <var data-var='time'>21:07</var> PDT Update - We have made partial progress. We have identified a brief period of event data loss, and some event-based features may show incomplete data while we continue recovery. We are continuing to monitor. Jun <var data-var='date'>10</var>, <var data-var='time'>21:07</var> PDT Update - We are continuing to monitor for any further issues. Jun <var data-var='date'>10</var>, <var data-var='time'>18:31</var> PDT Update - We have made partial progress. We have identified a brief period of event data loss, and some event-based features may show incomplete data while we continue recovery. We are continuing to monitor. Jun <var data-var='date'>10</var>, <var data-var='time'>11:54</var> PDT Update - We have made partial progress. Some features may show data gaps while we complete recovery. We are continuing to monitor. Jun <var data-var='date'> 9</var>, <var data-var='time'>05:08</var> PDT Update - We have made partial progress. Some features may show data gaps while we complete recovery. We are continuing to monitor. Jun <var data-var='date'> 8</var>, <var data-var='time'>19:23</var> PDT Update - We are continuing to monitor for any further issues. Jun <var data-var='date'> 8</var>, <var data-var='time'>19:03</var> PDT Update - We are monitoring the status of our services. Jun <var data-var='date'> 8</var>, <var data-var='time'>14:24</var> PDT Monitoring - A fix has been implemented and we are monitoring the results. Jun <var data-var='date'> 8</var>, <var data-var='time'>13:46</var> PDT Identified - We are currently experiencing delays in event ingestion impacting multiple product areas starting at 12:40pm PT. We have identified the issue and are working on a fix. No data has been lost.

Delays in event ingestion

Jun <var data-var='date'>11</var>, <var data-var='time'>21:58</var> PDT Resolved - Monitoring has confirmed no further impact outside of the initially reported 7:30-7:37pm PT timeframe Jun <var data-var='date'>11</var>, <var data-var='time'>21:23</var> PDT Monitoring - We identified a brief period of elevated errors affecting some LaunchDarkly application requests between 7:30-7:37pm PT. The issue has recovered, and we are continuing to monitor.

Elevated errors affecting LaunchDarkly application requests

Jun <var data-var='date'>10</var>, <var data-var='time'>09:00</var> PDT Completed - The scheduled maintenance has been completed. Jun <var data-var='date'>10</var>, <var data-var='time'>08:00</var> PDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'>27</var>, <var data-var='time'>08:13</var> PDT Scheduled - We are removing 535 legacy individual IP addresses for stream.launchdarkly.com from our public IP list. The public IP list API will be updated when this maintenance window expires. The following IP addresses will be removed: 3.0.119.37, 3.1.42.27, 3.1.97.23, 3.81.156.201, 3.91.135.85, 3.94.227.253, 3.208.63.99, 3.209.142.240, 3.209.231.150, 3.210.8.83, 3.210.111.117, 3.210.143.243, 3.210.162.180, 3.210.179.65, 3.211.0.174, 3.211.40.100, 3.211.78.169, 3.211.171.106, 3.212.14.208, 3.212.26.71, 3.212.29.247, 3.212.63.158, 3.212.69.145, 3.212.153.172, 3.212.215.241, 3.213.150.152, 3.213.152.159, 3.215.17.57, 3.215.85.74, 3.215.132.92, 3.215.154.205, 3.217.33.194, 3.217.93.49, 3.217.156.217, 3.221.55.92, 3.221.129.247, 3.222.86.128, 3.222.172.85, 3.223.193.186, 3.223.232.72, 3.224.166.159, 3.224.215.35, 3.225.49.136, 3.225.83.251, 3.225.123.155, 3.225.212.129, 3.226.201.70, 3.228.176.198, 3.229.200.95, 3.230.174.47, 3.232.113.99, 3.232.151.250, 3.232.245.239, 3.233.41.188, 3.233.231.70, 13.213.179.130, 13.214.45.79, 13.215.24.141, 13.215.32.248, 13.215.214.52, 13.216.151.182, 13.219.10.149, 13.228.2.1, 13.228.142.109, 13.228.146.169, 13.251.92.215, 13.251.181.59, 13.251.193.75, 18.136.21.56, 18.136.37.225, 18.136.69.184, 18.138.159.9, 18.138.181.38, 18.139.120.111, 18.139.139.40, 18.139.240.134, 18.141.20.225, 18.141.99.255, 18.141.115.110, 18.142.8.158, 18.200.175.192, 18.200.180.85, 18.202.92.19, 18.203.170.201, 18.203.183.133, 18.204.101.179, 18.204.156.199, 18.204.254.23, 18.206.18.130, 18.207.77.1, 18.209.25.132, 18.209.84.15, 18.209.88.214, 18.210.54.18, 18.210.67.12, 18.210.108.119, 18.210.122.219, 18.210.240.81, 18.210.251.21, 18.211.110.191, 18.211.133.130, 18.211.136.198, 18.211.150.137, 18.211.150.173, 18.211.172.26, 18.211.224.119, 18.213.70.53, 18.213.97.92, 18.214.59.159, 18.215.38.116, 18.233.237.81, 18.235.89.20, 23.22.17.50, 23.22.141.179, 34.192.2.36, 34.192.59.174, 34.192.76.59, 34.192.182.83, 34.192.228.56, 34.192.248.67, 34.193.139.161, 34.193.141.46, 34.193.205.73, 34.193.236.64, 34.194.6.199, 34.194.18.88, 34.194.82.127, 34.194.82.222, 34.194.162.18, 34.194.181.217, 34.194.195.110, 34.195.29.226, 34.195.40.39, 34.195.114.163, 34.195.178.243, 34.195.196.184, 34.195.235.129, 34.196.53.78, 34.196.57.238, 34.196.90.241, 34.196.162.28, 34.196.185.135, 34.196.185.152, 34.196.226.108, 34.196.231.1, 34.197.132.159, 34.197.153.179, 34.197.176.8, 34.197.220.63, 34.197.229.208, 34.198.5.248, 34.198.16.214, 34.198.103.118, 34.198.127.108, 34.199.155.251, 34.199.166.40, 34.201.196.14, 34.202.189.71, 34.203.82.48, 34.203.160.31, 34.204.215.144, 34.204.250.230, 34.205.180.137, 34.205.184.16, 34.206.142.57, 34.206.235.240, 34.206.243.165, 34.207.21.226, 34.224.206.32, 34.225.44.159, 34.225.52.183, 34.225.210.63, 34.226.53.28, 34.230.142.141, 34.230.201.255, 34.231.143.151, 34.232.120.176, 34.233.152.188, 34.234.29.92, 34.235.101.237, 34.236.195.69, 34.237.7.234, 34.237.149.109, 34.239.17.91, 34.240.90.246, 34.240.198.227, 34.248.94.177, 34.249.84.253, 34.249.165.146, 34.249.183.194, 34.250.129.69, 34.250.139.204, 34.251.47.151, 34.251.52.49, 34.251.220.170, 34.251.227.185, 34.252.74.89, 34.252.128.109, 34.254.129.50, 34.255.14.118, 34.255.186.89, 34.255.232.13, 35.153.62.144, 35.168.119.170, 35.169.61.114, 35.169.87.104, 35.169.162.188, 35.170.221.57, 35.171.38.152, 35.171.42.112, 35.171.69.133, 35.172.28.29, 35.175.51.91, 35.175.62.9, 44.193.41.212, 44.193.145.213, 44.193.160.19, 44.193.176.64, 44.193.192.114, 44.194.134.227, 44.195.178.165, 44.195.240.70, 44.196.237.249, 44.196.240.113, 44.197.94.28, 44.198.65.246, 44.198.216.81, 44.205.130.196, 44.205.142.202, 44.205.162.192, 44.205.194.1, 44.205.242.41, 44.206.106.7, 44.206.161.201, 44.207.32.19, 44.208.215.105, 44.208.247.20, 44.209.6.233, 44.209.27.102, 44.209.98.64, 44.209.163.213, 44.210.2.163, 44.212.201.65, 44.213.219.16, 44.214.25.250, 44.214.85.107, 44.214.199.141, 44.214.230.241, 44.219.171.95, 44.221.72.252, 44.223.189.67, 46.137.234.74, 47.130.27.211, 50.16.48.228, 50.16.53.115, 50.16.101.82, 50.16.209.122, 50.16.212.232, 50.17.142.209, 52.0.9.218, 52.0.20.18, 52.1.23.87, 52.1.64.188, 52.1.126.54, 52.2.69.118, 52.3.131.52, 52.3.164.32, 52.3.203.3, 52.4.17.19, 52.6.134.5, 52.6.144.34, 52.6.242.226, 52.7.81.224, 52.7.118.204, 52.16.12.215, 52.17.69.182, 52.17.108.103, 52.17.255.115, 52.18.122.137, 52.19.166.117, 52.20.44.107, 52.20.244.244, 52.21.16.31, 52.21.24.75, 52.21.105.64, 52.21.118.82, 52.22.11.124, 52.22.34.71, 52.22.75.64, 52.22.120.141, 52.22.199.197, 52.23.10.33, 52.23.189.51, 52.30.11.14, 52.30.81.136, 52.30.93.40, 52.30.206.184, 52.31.104.23, 52.31.141.156, 52.44.140.109, 52.44.175.163, 52.44.226.138, 52.45.14.195, 52.49.228.174, 52.51.161.20, 52.51.189.194, 52.51.248.2, 52.51.255.151, 52.55.197.136, 52.70.143.213, 52.71.94.11, 52.71.97.4, 52.72.133.223, 52.72.189.47, 52.72.196.236, 52.73.32.133, 52.74.25.166, 52.74.29.119, 52.74.99.63, 52.74.228.174, 52.76.0.153, 52.76.134.15, 52.76.172.180, 52.76.202.54, 52.77.84.211, 52.77.86.161, 52.77.128.254, 52.77.131.86, 52.77.147.155, 52.77.148.2, 52.77.150.234, 52.77.172.15, 52.86.102.109, 52.86.148.140, 52.200.10.183, 52.200.155.176, 52.200.171.107, 52.201.19.0, 52.201.30.87, 52.202.18.147, 52.202.164.129, 52.203.44.182, 52.203.112.214, 52.203.145.124, 52.203.169.8, 52.203.187.144, 52.204.35.155, 52.205.29.16, 52.205.60.71, 52.205.74.149, 52.205.140.231, 52.205.199.141, 52.206.123.108, 52.206.193.249, 52.207.13.114, 52.207.48.173, 52.209.147.212, 52.211.6.90, 52.211.63.57, 52.211.120.5, 52.212.183.84, 52.213.138.197, 52.220.58.39, 52.220.74.31, 52.220.114.176, 52.220.155.208, 52.220.225.57, 52.221.74.28, 52.221.163.147, 54.72.235.236, 54.74.99.219, 54.74.103.246, 54.76.123.134, 54.77.34.178, 54.77.78.215, 54.77.141.140, 54.77.209.136, 54.78.38.39, 54.80.1.84, 54.80.39.21, 54.81.117.83, 54.81.213.212, 54.82.23.136, 54.82.115.76, 54.82.179.12, 54.83.61.253, 54.83.86.62, 54.83.149.69, 54.84.21.101, 54.84.93.37, 54.84.132.233, 54.84.132.245, 54.84.144.50, 54.84.189.119, 54.84.245.230, 54.85.9.44, 54.85.13.148, 54.85.15.157, 54.85.33.15, 54.85.161.78, 54.85.161.163, 54.86.14.199, 54.86.118.177, 54.86.156.169, 54.86.180.134, 54.86.238.142, 54.86.244.151, 54.87.94.27, 54.87.214.216, 54.88.160.160, 54.88.189.116, 54.88.225.200, 54.88.232.219, 54.89.56.128, 54.144.52.234, 54.144.244.40, 54.145.75.12, 54.145.161.1, 54.145.215.104, 54.147.67.241, 54.152.40.39, 54.154.125.37, 54.154.217.226, 54.154.242.172, 54.156.155.61, 54.158.1.193, 54.158.114.255, 54.158.201.166, 54.159.128.43, 54.160.1.221, 54.160.65.173, 54.160.102.192, 54.162.53.174, 54.162.82.157, 54.164.104.43, 54.164.149.203, 54.167.133.6, 54.167.202.203, 54.169.20.142, 54.169.225.167, 54.171.251.15, 54.173.55.139, 54.173.173.176, 54.174.5.251, 54.175.84.251, 54.179.41.143, 54.179.145.26, 54.179.180.239, 54.179.215.92, 54.194.61.132, 54.194.100.11, 54.194.128.33, 54.195.91.253, 54.197.46.108, 54.197.143.76, 54.204.192.64, 54.209.132.65, 54.209.192.81, 54.209.235.134, 54.209.245.121, 54.210.33.232, 54.210.125.182, 54.210.159.138, 54.210.224.100, 54.211.58.167, 54.211.63.220, 54.211.104.168, 54.211.161.195, 54.211.233.7, 54.224.162.1, 54.225.42.164, 54.225.71.167, 54.227.204.38, 54.235.4.229, 54.236.77.131, 54.236.87.64, 54.236.121.82, 54.236.158.150, 54.236.171.5, 54.236.192.107, 54.243.34.157, 54.243.39.147, 54.243.54.147, 54.243.58.248, 54.243.79.193, 54.243.121.35, 54.243.165.178, 54.243.220.97, 54.243.227.67, 54.243.238.143, 54.243.249.198, 54.246.141.209, 54.251.134.26, 54.251.160.36, 54.251.177.85, 54.251.237.126, 54.254.167.36, 54.254.196.191, 63.32.12.239, 63.33.91.235, 63.33.245.218, 63.34.202.232, 72.44.54.239, 75.101.212.163, 79.125.50.68, 79.125.65.61, 98.82.52.30, 98.82.55.107, 98.85.24.70, 98.85.64.100, 98.86.87.228, 98.86.214.67, 98.87.97.132, 98.88.212.98, 98.90.74.184, 99.80.210.140, 99.81.3.208, 99.81.124.173, 100.24.171.201, 100.26.67.253, 100.29.106.41, 107.20.48.118, 107.22.84.205, 107.22.136.94, 107.23.47.163, 107.23.110.189, 107.23.139.117, 107.23.223.23, 108.128.57.57, 174.129.25.155, 174.129.120.2, 175.41.136.1, 184.72.99.32, 184.72.125.67, 184.73.82.37 For the current public IP list, see: https://docs.launchdarkly.com/home/advanced/public-ip-list.

Removing legacy IP addresses for stream.launchdarkly.com

Jun <var data-var='date'> 3</var>, <var data-var='time'>13:26</var> PDT Resolved - The incident has been resolved and the flag dashboard is successfully loading. Jun <var data-var='date'> 3</var>, <var data-var='time'>13:06</var> PDT Monitoring - A fix has been implemented and were monitoring the results Jun <var data-var='date'> 3</var>, <var data-var='time'>12:54</var> PDT Identified - The issue has been identified and a fix is in progress. Jun <var data-var='date'> 3</var>, <var data-var='time'>12:31</var> PDT Investigating - Currently investigating an issue causing the flag dashboard to fail to load.

Flag dashboard is failing to load

May <var data-var='date'>22</var>, <var data-var='time'>12:41</var> PDT Resolved - We have confirmed Slack notifications are now coming through for segment approvals again. May <var data-var='date'>22</var>, <var data-var='time'>12:33</var> PDT Monitoring - A fix has been implemented and we are monitoring notifications. May <var data-var='date'>22</var>, <var data-var='time'>11:40</var> PDT Investigating - We are experiencing issues with sending Slack notifications for segment approvals. Email and in-app notifications still work as expected. We are working on restoring this functionality as soon as possible.

Issue with Slack notifications for Segment approvals

May <var data-var='date'>19</var>, <var data-var='time'>07:54</var> PDT Resolved - This incident has been resolved. Event ingestion is operating normally and all Experimentation results and Guarded Rollouts are fully restored. No data was lost. We apologize for any inconvenience caused. Results were delayed from 6:58am PDT to 7:52am PDT May <var data-var='date'>19</var>, <var data-var='time'>07:49</var> PDT Identified - We are currently experiencing delays in event ingestion, which is impacting Experimentation results and Guarded Rollouts. No data has been lost — all results and events are intact and will be fully accessible once a fix has been deployed. We have identified the root cause and a fix is being released now.

Event Ingestion Delays

May <var data-var='date'>15</var>, <var data-var='time'>18:13</var> PDT Resolved - All queued events have been processed and data ingestion is operating normally. No data was lost during this incident. May <var data-var='date'>15</var>, <var data-var='time'>17:14</var> PDT Identified - We have identified the issue and a fix is in progress. The backlog is actively decreasing. No data has been lost — all queued events will be processed. May <var data-var='date'>15</var>, <var data-var='time'>17:03</var> PDT Investigating - We are currently experiencing a delay in processing Session Replay and Error event data. Incoming data is being queued and will be processed — no data has been lost. Customers may observe a temporary lag in session and error data appearing in the LaunchDarkly UI.

Delayed processing of Session Replay and Error data

May <var data-var='date'> 5</var>, <var data-var='time'>21:26</var> PDT Resolved - We've implemented a fix and have mitigated impact as of 9:25pm PT. Streaming SDKs should be able to connect successfully at this time. May <var data-var='date'> 5</var>, <var data-var='time'>21:00</var> PDT Identified - We're identified an infrastructure issue causing elevated error rates and are working on a fix. May <var data-var='date'> 5</var>, <var data-var='time'>20:58</var> PDT Investigating - We are investigating elevated error rates for streaming initialization in the APAC region. Some customers using streaming server-side SDKs may have experienced longer than normal initialization times or timeouts when connecting to LaunchDarkly.

Elevated Streaming Errors in APAC region

May <var data-var='date'> 4</var>, <var data-var='time'>13:25</var> PDT Resolved - All experiment and multi-armed bandit result data is caught up. No data was lost. May <var data-var='date'> 4</var>, <var data-var='time'>13:19</var> PDT Monitoring - We have implemented a fix and are catching up on backlogged data. We estimate that we will be fully caught up in about 1 hour. May <var data-var='date'> 4</var>, <var data-var='time'>12:50</var> PDT Identified - We have identified the issue and are continuing our work to resolve it. May <var data-var='date'> 4</var>, <var data-var='time'>12:47</var> PDT Investigating - We are currently investigating delays in experiment and multi-armed bandit results. Data is currently delayed by 1-2 hours. No data loss is expected.

Experiment and multi-armed bandit results delayed

May <var data-var='date'> 4</var>, <var data-var='time'>09:23</var> PDT Resolved - This incident has been resolved. May <var data-var='date'> 4</var>, <var data-var='time'>08:47</var> PDT Monitoring - We've implemented a fix and have mitigated impact as of 8:35am PT. Streaming SDKs should be able to connect successfully at this time. May <var data-var='date'> 4</var>, <var data-var='time'>07:30</var> PDT Update - We're continuing to address the root cause of the elevated error rate. Server SDKs may require additional time to initialize while requests are retried. May <var data-var='date'> 4</var>, <var data-var='time'>05:51</var> PDT Identified - We're identified an infrastructure issue causing elevated error rates and are working on a fix. May <var data-var='date'> 4</var>, <var data-var='time'>03:14</var> PDT Investigating - We are investigating elevated error rates for streaming initialization in the Europe region. Some customers using streaming server-side SDKs may have experienced longer than normal initialization times or timeouts when connecting to LaunchDarkly.

Elevated Streaming Errors in Europe

May <var data-var='date'> 2</var>, <var data-var='time'>12:15</var> PDT Resolved - Experimentation results processing and data warehouse export was caught up as of 12:28 pm PT. May <var data-var='date'> 2</var>, <var data-var='time'>06:51</var> PDT Monitoring - We have implemented a fix and are catching up on backlogged experimentation results and data export. We estimate that we will be fully caught up in about 8 hours. May <var data-var='date'> 2</var>, <var data-var='time'>06:17</var> PDT Identified - We've identified the root cause of the experimentation results processing and data export delays and are working on implementing a fix. May <var data-var='date'> 2</var>, <var data-var='time'>05:55</var> PDT Investigating - We are investigating an issue causing delayed experimentation results processing and delayed experimentation data warehouse export.

Delay in Experimentation Results and Experimentation Warehouse Data Export

Apr <var data-var='date'>30</var>, <var data-var='time'>19:52</var> PDT Resolved - We've identified elevated error rates with server-side streaming initialization requests in the APAC region between 19:52 and 19:56 PT. Some customers may have experienced initialization timeouts despite SDK retries.

Elevated Server SDK Initialization Errors in APAC region

Apr <var data-var='date'>30</var>, <var data-var='time'>17:09</var> PDT Resolved - We have validated a fix to resolve all remaining experiments with inaccurate sample ratio mismatches (SRMs) or empty experiment results. The vast majority of experiments and experimentation customers were not impacted. Apr <var data-var='date'>29</var>, <var data-var='time'>17:04</var> PDT Monitoring - We've released a fix to prevent this issue in new experiments and iterations. We're working to resolve this issue for experiment data recently processed. Apr <var data-var='date'>29</var>, <var data-var='time'>13:49</var> PDT Identified - We've identified the issue and are implementing a fix for this behavior. Apr <var data-var='date'>29</var>, <var data-var='time'>13:19</var> PDT Investigating - We are aware of an issue affecting some customers that may result in sample ratio mismatches (SRMs) or empty experiment results if flag variations are edited between experiment iterations.

Issue with Experiment Results in case of Flag Variation edits

Apr <var data-var='date'>30</var>, <var data-var='time'>04:58</var> PDT Resolved - We've caught up on the streaming data export backlog. Apr <var data-var='date'>30</var>, <var data-var='time'>04:45</var> PDT Monitoring - We've identified latency in Pub/Sub export that were causing streaming data export delays that has since resolved. We're catching up on the export backlog now. Apr <var data-var='date'>30</var>, <var data-var='time'>04:14</var> PDT Investigating - We are investigating a delay with streaming data export delivery. No data is lost, but customers using streaming data export may notice a multi-minute lag in receiving data.

Streaming Data Export Delay

Apr <var data-var='date'>29</var>, <var data-var='time'>16:45</var> PDT Resolved - We've identified an infrastructure issue that caused some 503 responses in feature flagging streaming client initialization on the US LaunchDarkly instance between 16:45 and 17:05 PT. SDKs should automatically retry initialization in most cases.

Errors on streaming SDK initialization

Apr <var data-var='date'>28</var>, <var data-var='time'>12:25</var> PDT Resolved - Observability alerts are functioning and delayed alerts have been redelivered. Apr <var data-var='date'>28</var>, <var data-var='time'>12:10</var> PDT Monitoring - A fix has been deployed for Observability alerting and we are working to redeliver delayed notifications. Apr <var data-var='date'>28</var>, <var data-var='time'>11:38</var> PDT Identified - We've identified the root cause delaying Observability alert notifications and are deploying a fix. All other notification types are operational. Apr <var data-var='date'>28</var>, <var data-var='time'>11:34</var> PDT Investigating - We are currently investigating an issue prevent Observability alert notifications from being delivered.

Issues with Observability Alert Notification

Apr <var data-var='date'>27</var>, <var data-var='time'>10:37</var> PDT Resolved - We've fully caught up on the metrics data backlog. Apr <var data-var='date'>27</var>, <var data-var='time'>09:43</var> PDT Monitoring - We're catching up on backlogged metric data in EU and are monitoring progress. No metric data has been lost. Apr <var data-var='date'>27</var>, <var data-var='time'>08:45</var> PDT Identified - We've identified that metrics ingest was delayed between 11:11 - 11:27am ET. No metric data has been lost. We're working on catching up on the delayed data. Apr <var data-var='date'>27</var>, <var data-var='time'>08:39</var> PDT Investigating - We are investigating an issue causing under-reporting of customer metrics in the EU region.

Delayed Metrics ingest in EU

Apr <var data-var='date'>23</var>, <var data-var='time'>12:00</var> PDT Resolved - We've addressed an issue that prevented warehouse data export from 12pm to 6pm PT on April 23. All missing data has been retroactively pushed to customer warehouses.

Is LaunchDarkly down?

LaunchDarkly status is UP

Lastest published events

Delays in Experimentation and Guarded Rollout results

High volume of error alerts related to flag delivery network.

Verification emails not being sent

LaunchDarkly Services, Components and Regions

Recent LaunchDarkly events

Planned Upcoming API Versions Retirement

Delays in Experimentation and Guarded Rollout results

High volume of error alerts related to flag delivery network.

Verification emails not being sent

Elevated error rates on event data ingestion endpoints

LaunchDarkly.com and LaunchDarkly.com/docs unavailable for some

Get your FREE account now,
5 monitors included!

Is LaunchDarkly down?

LaunchDarkly status is UP

Lastest published events

Delays in Experimentation and Guarded Rollout results

High volume of error alerts related to flag delivery network.

Verification emails not being sent

LaunchDarkly Services, Components and Regions

Recent LaunchDarkly events

Planned Upcoming API Versions Retirement

Delays in Experimentation and Guarded Rollout results

High volume of error alerts related to flag delivery network.

Verification emails not being sent

Elevated error rates on event data ingestion endpoints

LaunchDarkly.com and LaunchDarkly.com/docs unavailable for some

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!