Jun <var data-var='date'>19</var>, <var data-var='time'>17:56</var> UTC Resolved - The issue affecting the Commerce Studio Rules interface has been resolved, and all previously affected services are now operating normally. Commerce Studio users can interact with the Rules interface and page through large datasets without experiencing page load errors. There remains no direct end-user impact on search experiences. The root cause of the incident was database contention triggered by interactions involving a large number of rules. To resolve the issue, we scaled up the database, which normalized performance and stabilized the system. Jun <var data-var='date'>19</var>, <var data-var='time'>17:48</var> UTC Monitoring - We have scaled up the database to alleviate the performance issues. Performance has now normalized, and Commerce Studio Rules interactions have stabilized. Platform users may still observe minor latency when attempting to page through 100 rules at a time within the interface, but these requests no longer experience complete timeouts or produce errors. There is no direct end-user impact. We are continuing to monitor database response times and page load times, and we will provide a final update once full resolution is confirmed. Jun <var data-var='date'>19</var>, <var data-var='time'>17:21</var> UTC Identified - Commerce Studio is experiencing reduced performance within the Rules API due to apparent underlying database contention. Platform users attempting to interact with the Rules section of the interface, particularly when selecting the option to display 100 rules per page, will encounter timeouts. There is no direct end-user impact to search experiences, as this issue is confined to administrative functions within the management console. We believe the issue to be caused by database contention specifically triggered by simultaneous interactions involving a large number of rules. We are currently scaling up the database to alleviate the contention. While this scaling process is underway, some page loads may be interrupted. We are continuing to work toward full resolution and will provide an update once the system has stabilized. Jun <var data-var='date'>19</var>, <var data-var='time'>16:23</var> UTC Investigating - Lucidworks Commerce Studio is currently experiencing intermittent latency, specifically affecting the Rules component. Customers attempting to interact with the Rules interface will experience timeouts when trying to load their data. There is no direct end-user impact, as this issue only affects internal administrative tasks within the Lucidworks Platform. We are actively investigating the cause of this disruption and will provide further updates as more information becomes available.

Commerce Studio Rules Interface Disruption

May <var data-var='date'>28</var>, <var data-var='time'>18:08</var> UTC Resolved - The service disruption affecting certain Lucidworks AI hosted models has been fully resolved, and all previously affected services are operating normally. End-user functionality is fully restored, with all dependent services operating as expected. The disruption originated during routine Kubernetes maintenance involving node upgrades, during which a cloud provider capacity shortfall prevented the instances hosting these models from re-launching. Stability was restored after successfully launching new nodes and pulling the required images. We will share a postmortem report containing the full root cause analysis within three business days. May <var data-var='date'>28</var>, <var data-var='time'>18:01</var> UTC Monitoring - We have successfully launched new nodes with the required LLM images, and redirected Lucidworks AI traffic to them. All LWAI hosted models are fully functional again, and stability has been confirmed through successful 200 responses from the prediction endpoint. End-user functionality is fully restored, with queries no longer returning errors. We are continuing to observe monitoring metrics to ensure system stability remains consistent before providing a final resolution update. May <var data-var='date'>28</var>, <var data-var='time'>17:39</var> UTC Identified - The service disruption affecting certain Lucidworks AI hosted models remains ongoing, and the models are currently unavailable. The disruption occurred during routine Kubernetes maintenance involving node upgrades, during which a cloud provider capacity shortfall prevented the instances hosting these models from re-launching. We have secured the needed capacity and have engaged our cloud provider to resolve the remaining delays in bringing the models back up. We will provide further updates as additional information becomes available. May <var data-var='date'>28</var>, <var data-var='time'>17:14</var> UTC Investigating - Certain Lucidworks AI hosted models (llama-3-8b-instruct, llama-3v2-3b-instruct, and phi-4-multimodal-instruct) in the us-southcarolina region are experiencing a service disruption, causing them to be currently unavailable. Customers attempting to use these Lucidworks-hosted models are receiving 500 or 429 errors for queries that rely on them. End users utilizing services that depend on these specific models are encountering errors, though passthrough LLM calls remain fully functional. We are currently investigating the issue to determine the cause and are actively working to restore full availability. We will provide further updates as new information becomes available.

Lucidworks AI Hosted LLM Service Disruption

May <var data-var='date'>13</var>, <var data-var='time'>00:01</var> UTC Resolved - Lucidworks has confirmed the full resolution of this incident. The SaaS Platform interface is once again accessible, and the errors previously encountered at platform.lucidworks.com have been cleared. The Lucidworks Platform is operating normally for all users. Lucidworks AI and Beacon services remained stable and were not affected throughout this event. There is no lingering impact to platform users or end-user experiences. A postmortem report will be shared once the internal review is complete. May <var data-var='date'>12</var>, <var data-var='time'>23:41</var> UTC Monitoring - Engineering has successfully deployed a fix to restore the missing data and has been merged into the production environment. We are now observing a period of stability following these mitigation actions. The SaaS Platform interface is now accessible, and the previously reported errors have been cleared. Lucidworks AI and Beacon services remain stable and were unaffected throughout the event. No widespread user impact is currently being observed, though we are performing final data validation to ensure all configuration values have been fully restored across the environment. We are continuing to monitor platform stability and system logs to confirm that all services maintain normal performance levels. We will provide a final update once the monitoring period is complete and the incident is fully resolved. May <var data-var='date'>12</var>, <var data-var='time'>23:23</var> UTC Identified - Lucidworks has identified that a recent configuration update caused a critical component to be removed, preventing the platform interface from loading correctly. This is the source of the access errors users are currently experiencing. Services hosted on the platform remain operational, and Lucidworks AI and Beacon services continue to function normally. We are deploying a fix to restore the missing component and are repopulating the necessary data from a secure backup to ensure all settings are accurately recovered. Users should refrain from reconfiguring platform settings or applications until access is fully restored. We will provide a further update as soon as more information is available. May <var data-var='date'>12</var>, <var data-var='time'>21:19</var> UTC Investigating - We are currently investigating an issue affecting the Lucidworks Platform user interface. While services hosted on the Platform remain operational and the Beacon continues to receive signals from client sites, all SaaS users are unable to access the SaaS Platform interface. Users attempting to load platform.lucidworks.com encounter an error stating that applications failed to load and menu items may be unavailable. There is no direct impact to the client’s end-user experience at this time. We are currently working to identify the cause of this disruption and will provide further updates as more information becomes available.

SaaS Platform Access Disruption

Apr <var data-var='date'>24</var>, <var data-var='time'>16:06</var> UTC Resolved - The Lucidworks SaaS Platform experienced an issue that caused a total disruption of service routing, rendering the platform inaccessible. Users were unable to log in or manage any platform configurations during this period. End users were unable to access search results or interact with any deployed services. The incident was caused by an unintended deployment of a configuration change at 4:06 UTC that should not have gone to production. We resolved the issue by performing a full rollback of the faulty change, which was completed at 4:13 UTC. A postmortem report will be posted with the full incident details.

SaaS Platform Service Routing Disruption

Mar <var data-var='date'> 6</var>, <var data-var='time'>06:13</var> UTC Resolved - We've confirmed that all platform.lucidworks.com functionality has been restored after reverting the most recently-deployed change. This incident only involved customer access to the SaaS Platform user interface; no search or inference capabilities were affected, and all Platform services remained operational. A detailed analysis will be posted here as soon as it is available. Mar <var data-var='date'> 6</var>, <var data-var='time'>05:55</var> UTC Monitoring - The recent change has been rolled back and UI functionality has been restored. We are monitoring to confirm full recovery. Mar <var data-var='date'> 6</var>, <var data-var='time'>05:48</var> UTC Identified - We believe this issue to have been caused by a recently-deployed change for a specific customer UI feature that is currently in development. Mar <var data-var='date'> 6</var>, <var data-var='time'>05:05</var> UTC Investigating - We are currently experiencing a disruption of the SaaS Platform user interface. Users are receiving 404 errors when attempting to access platform.lucidworks.com. All other SaaS Platform services remain operational. There is no direct impact on end-user services or search experiences at this time, but the ability to provision or configure SaaS Platform-based services is affected. We are actively investigating the issue and taking steps to restore access. We will provide further updates as more information becomes available.

platform.lucidworks.com user interface unavailable

Dec <var data-var='date'> 3</var>, <var data-var='time'>22:05</var> UTC Resolved - The Lucidworks SaaS Platform is now fully restored and operational. The third-party network issue has been successfully fixed and full stability has been confirmed during our monitoring period. We thank you for your patience. A detailed postmortem will be posted here as soon as it is available. Dec <var data-var='date'> 3</var>, <var data-var='time'>21:01</var> UTC Monitoring - The core issue has been successfully mitigated by our third-party vendor. The Lucidworks SaaS Platform is now entering a critical monitoring phase as our teams verify full stability and functionality across all services (Lucidworks AI, Commerce Studio, etc.). We anticipate a full resolution announcement shortly. Dec <var data-var='date'> 3</var>, <var data-var='time'>20:48</var> UTC Update - The Lucidworks SaaS Platform remains unavailable due to the external third-party issue. The vendor has successfully identified the root cause within their system and their engineering teams are now actively working on the corrective fix. We are closely monitoring their progress and maintaining direct communication. We will post our next status update within 30 minutes Dec <var data-var='date'> 3</var>, <var data-var='time'>20:16</var> UTC Update - The Lucidworks SaaS Platform remains unavailable as our teams are still actively engaged with the third-party service vendor. We have successfully identified the root cause as a network infrastructure issue on the vendor's side, and we are closely collaborating with their engineering team as they prioritize and execute the necessary corrective actions to restore service stability and platform connectivity. Dec <var data-var='date'> 3</var>, <var data-var='time'>19:28</var> UTC Update - The Lucidworks Platform remains unavailable due to an outage with a critical third-party service provider. Our teams are working directly with the vendor's engineers to restore connectivity and service stability as quickly as possible. Dec <var data-var='date'> 3</var>, <var data-var='time'>18:30</var> UTC Identified - The Lucidworks SaaS Platform is currently experiencing an unexpected outage. This impacts all services, including Lucidworks AI, Commerce Studio, Analytics Studio, and Connected Search. We are actively investigating the root cause and deploying resources to restore full functionality as quickly as possible. Next Update: We will post a full status update within 30 minutes. We apologize for the disruption and appreciate your patience. Dec <var data-var='date'> 3</var>, <var data-var='time'>19:30</var> UTC Investigating - We are currently investigating this issue.

SaaS Platform unavailable

Nov <var data-var='date'>13</var>, <var data-var='time'>17:47</var> UTC Resolved - We have confirmed that rolling back the erroneous change has resolved this incident. A postmortem will be posted here as soon as possible. Nov <var data-var='date'>13</var>, <var data-var='time'>17:15</var> UTC Monitoring - We have identified a recent job that ran with an incorrect parameter, which we believe to be the cause of this incident. We have reverted that change and services are now becoming available again. We are continuing to monitor to ensure full recovery. Nov <var data-var='date'>13</var>, <var data-var='time'>17:07</var> UTC Investigating - The Lucidworks SaaS Platform, which powers Lucidworks AI, Commerce Studio, Analytics Studio, and Connected Search is currently unavailable. We are actively working to investigate and mitigate this issue and will post updates here.

Oct <var data-var='date'>30</var>, <var data-var='time'>17:12</var> UTC Resolved - Rolling back the recent change has resolved this incident, and we have confirmed that we are no longer seeing query errors. Individual Connected Search customers who we know to have been affected have already had a proactive ticket raised by Lucidworks Support. Any additional inquiries or follow-ups can be submitted by visiting our Support Portal: https://support.lucidworks.com/hc/en-us We will post a full postmortem here within 48 hours. Oct <var data-var='date'>30</var>, <var data-var='time'>17:01</var> UTC Monitoring - The change has been rolled back and we are starting to see recovery. Search queries for Connected Search endpoints are once again responding successfully. Our team is continuing to monitor to ensure all errors resolve. Oct <var data-var='date'>30</var>, <var data-var='time'>16:10</var> UTC Identified - We have identified a recent change that seems to be the source of this issue, and are preparing to roll that change back now. Oct <var data-var='date'>30</var>, <var data-var='time'>15:40</var> UTC Investigating - We are experiencing an elevated rate of search errors for Connected Search customers. Many queries are currently returning 504 responses and the calls are unable to complete. Lucidworks personnel are actively investigating.

Elevated search errors

Sep <var data-var='date'> 8</var>, <var data-var='time'>13:15</var> UTC Resolved - Recovery of the 3rd-party service provider's system led to recovery of our own backend search service, and Connected Search query responses have been fully functional since that time. Sep <var data-var='date'> 8</var>, <var data-var='time'>12:43</var> UTC Monitoring - The issue was due to a performance degradation with one of our 3rd-party service providers. Once their service was fully recovered, we saw recovery of the elevated Connected Search error rate. We will continue to monitor this issue to ensure it is fully resolved. Sep <var data-var='date'> 8</var>, <var data-var='time'>12:41</var> UTC Update - We are continuing to investigate this issue. We are seeing elevated query latency related to a critical backend search service, and are focusing our investigation there. Sep <var data-var='date'> 8</var>, <var data-var='time'>11:54</var> UTC Investigating - We are seeing an elevated error rate on search requests for our Connected Search customers. Lucidworks Engineering is actively investigating the issue, and updates will be posted here as soon as we have more information to share.

Intermittent search failures of Connected Search sites

Jun <var data-var='date'>12</var>, <var data-var='time'>20:43</var> UTC Resolved - We have confirmed that all Lucidworks Platform products and services are once again fully functional, including LWAI and all Neural Hybrid Search functionality for Lucidworks Search customers. Jun <var data-var='date'>12</var>, <var data-var='time'>20:27</var> UTC Monitoring - Our cloud provider has resolved the underlying issue, and Lucidworks Platform services are recovering. Lucidworks engineers are monitoring the recovery of our services and will provide another update once we've confirmed that everything is fully functional again. Our Platform UI is once again accessible. Connected Search functionality has fully recovered. Jun <var data-var='date'>12</var>, <var data-var='time'>19:54</var> UTC Update - Our cloud provider is actively mitigating the issue. We are monitoring the recovery and will keep impacted customers informed of the progress. Jun <var data-var='date'>12</var>, <var data-var='time'>19:14</var> UTC Update - Our cloud provider has indicated that they are in the process of mitigating the underlying issue affecting us. We are continuing to monitor and to notify impacted customers. Jun <var data-var='date'>12</var>, <var data-var='time'>18:46</var> UTC Investigating - Due to a widespread outage with our public cloud provider, the SaaS Platform user console is currently unavailable. This incident is affecting Lucidworks customers' ability to use LWAI, as well as Connected Search's ability to serve queries. We are actively working with our provider to reach a resolution as soon as possible, and will provide regular updates here.

May <var data-var='date'>26</var>, <var data-var='time'>20:28</var> UTC Resolved - We’ve confirmed that all Lucidworks Platform functionality related to this incident has been fully restored. We will share more details in a postmortem update as soon as our analysis is complete. May <var data-var='date'>26</var>, <var data-var='time'>20:23</var> UTC Monitoring - We have identified a misconfiguration related to the TLS certificate configured for the applications.lucidworks.com domain. This configuration has been reverted and the SSL handshake errors we had been seeing have stopped. We are monitoring to confirm stability. May <var data-var='date'>26</var>, <var data-var='time'>19:30</var> UTC Identified - The specific errors we’re seeing are “SSL handshake fails for connections to *.applications.lucidworks.com.” Our team is pursuing all avenues of investigation with a focus on TLS certificate configurations for endpoints in the applications.lucidworks.com DNS domain. May <var data-var='date'>26</var>, <var data-var='time'>18:16</var> UTC Investigating - We have been alerted to Lucidworks SaaS Platform errors being returned in response to ingestion and query API calls for Connected Search clients, as well as for LWAI calls coming via Managed Fusion Integrations. The Platform UI continues to function, and configuration changes can still be made. The Lucidworks team is investigating.

Applications and Integration Errors

Apr <var data-var='date'>10</var>, <var data-var='time'>16:19</var> UTC Resolved - A change related to enabling a new product feature was rolled out that had the unintended effect of making our Platform UI unavailable, which was not caught in our development and testing environment. That change has been rolled back, and the UI is once again available. No other product functionality was impacted by this incident. We will share more details in a postmortem update as soon as our analysis is complete. Apr <var data-var='date'>10</var>, <var data-var='time'>15:50</var> UTC Investigating - The Lucidworks Platform UI at platform.lucidworks.com is currently unavailable. This has no effect on product functionality other than users' ability to change configurations. Our teams are actively working to resolve the issue.

Platform UI unavailable

Jul <var data-var='date'>24</var>, <var data-var='time'>18:57</var> UTC Resolved - The fix has been verified and we've confirmed that full functionality has been restored. No Signals Beacon configurations were lost during this incident. Jul <var data-var='date'>24</var>, <var data-var='time'>18:40</var> UTC Monitoring - We identified the source of the error as being a database configuration update that was inadvertently applied to our production site, rather than in our development environment as intended. We have rolled back that change, and the Beacon is able to successfully load its configuration again. We're continuing to monitor to ensure there are no ongoing effects of this incident. Jul <var data-var='date'>24</var>, <var data-var='time'>17:30</var> UTC Investigating - The Lucidworks Signals Beacon is currently unable to load its configuration due to a server-side error on the Lucidworks Platform. As a result, while existing site visits will continue to send signals to Lucidworks Analytics, new page loads will not be able to initialize the Beacon and some signals will not be collected. We are working to restore full functionality as soon as possible, and will provide timely updates as this incident is addressed.

Beacon configs failing to load

Sep <var data-var='date'>29</var>, <var data-var='time'>18:24</var> UTC Resolved - We have confirmed that reverting our earlier change has corrected the root cause of this incident, and platform.lucidworks.com is once again fully functional. Sep <var data-var='date'>29</var>, <var data-var='time'>17:55</var> UTC Update - The platform.lucidworks.com interface is once again available and functional. No indexing or search query outage was experienced during this incident. We're continuing to monitor to ensure there are no lingering issues. Sep <var data-var='date'>29</var>, <var data-var='time'>17:29</var> UTC Update - We have identified the source of the issue and have reverted the recent change that caused it. The fix is actively rolling out. Some users will see recovery and be able to log in to platform.lucidworks.com already, while for others it may not show full recovery until the rollout has fully completed. Sep <var data-var='date'>29</var>, <var data-var='time'>17:14</var> UTC Investigating - The configuration UI at platform.lucidworks.com is currently unavailable and returning errors when users attempt to access it. Lucidworks engineers are actively investigating the problem. This incident does NOT affect any indexing or search functionality; it is only preventing configuration changes to existing Connected Search applications. We will post updates as soon as we have more details to share.

platform.lucidworks.com unavailable

Apr <var data-var='date'>18</var>, <var data-var='time'>15:57</var> UTC Resolved - Our upstream vendor has deployed a fix for the bug that was impacting some Connected Search customers, and Lucidworks has verified that the issue is now fully resolved. Apr <var data-var='date'>13</var>, <var data-var='time'>07:56</var> UTC Update - Our upstream vendor has identified the cause of the bug and is working on an urgent fix. In the meantime, we are keeping our mitigation in place and continuing to monitor to ensure continued operations and functionality of Connected Search for all customers. We will provide further updates as soon as we have them. Apr <var data-var='date'>11</var>, <var data-var='time'>20:41</var> UTC Monitoring - We have confirmed the underlying cause of this issue, and it has been mitigated at this time. A particularly resource-intensive query was found to be impacting the Connected Search backend for all customers by triggering a bug in the underlying data store service. Disabling this query has returned service to normal, and all Connected Search and Springboard UI functionality should be fully restored at this time. Lucidworks will follow up on a root-cause analysis with the upstream vendor in question, and will update our own findings here as we have more details to share. Apr <var data-var='date'>11</var>, <var data-var='time'>18:54</var> UTC Identified - We continue to work to fully resolve the source of the query timeouts, while minimizing the impact as much as possible. We believe we have identified the root cause and are now pursuing corrective actions. We'll continue to provide updates as soon as more information is available. Apr <var data-var='date'>11</var>, <var data-var='time'>16:19</var> UTC Update - We have identified the source of intermittent query failures and occasional timeouts, and are working with an upstream vendor to resolve them. We do not have an ETA for full resolution at this time, but believe that our initial troubleshooting has already lessened both the frequency and severity of these query issues. We will continue to provide updates as the situation develops. Apr <var data-var='date'>11</var>, <var data-var='time'>12:30</var> UTC Investigating - Some Connected Search requests to the Query API are intermittently timing out, which results in them returning either empty or inconsistent results. This issue is affecting all Connected Search customers, as well as the Experience Optimizer section of the Springboard UI (platform.lucidworks.com). Lucidworks engineers are investigating the cause of this issue and updates will provided here as soon as we have them.

Intermittent search request timeouts

Jan <var data-var='date'> 6</var>, <var data-var='time'>16:15</var> UTC Resolved - An internal routing component of Connected Search had gotten into a stuck state where it could not validate a newly-rotated TLS certificate used to route traffic to the search backend. By restarting this component, we were able to mitigate the issue. We have updated our tooling to force a restart of this component every time the TLS certificate is rotated, in order to avoid a recurrence. Jan <var data-var='date'> 6</var>, <var data-var='time'>15:55</var> UTC Update - We have mitigated the issue, and all search functionality (including both API-based search requests, as well as the ability to use Experience Optimizer in our platform UI) has been restored. Jan <var data-var='date'> 6</var>, <var data-var='time'>10:36</var> UTC Investigating - The Lucidworks Connected Search platform is experiencing an issue with routing traffic that is resulting in failed search query requests. This issue is also affecting the ability to use Experience Optimizer from within the platform web interface.

Is Lucidworks Platform down?

Lucidworks Platform status is UP

Lastest published events

Commerce Studio Rules Interface Disruption

Lucidworks AI Hosted LLM Service Disruption

SaaS Platform Access Disruption

Lucidworks Platform Services, Components and Regions

Recent Lucidworks Platform events

Commerce Studio Rules Interface Disruption

Lucidworks AI Hosted LLM Service Disruption

SaaS Platform Access Disruption

SaaS Platform Service Routing Disruption

platform.lucidworks.com user interface unavailable

SaaS Platform unavailable

Get your FREE account now,
5 monitors included!

Is Lucidworks Platform down?

Lucidworks Platform status is UP

Lastest published events

Commerce Studio Rules Interface Disruption

Lucidworks AI Hosted LLM Service Disruption

SaaS Platform Access Disruption

Lucidworks Platform Services, Components and Regions

Recent Lucidworks Platform events

Commerce Studio Rules Interface Disruption

Lucidworks AI Hosted LLM Service Disruption

SaaS Platform Access Disruption

SaaS Platform Service Routing Disruption

platform.lucidworks.com user interface unavailable

SaaS Platform unavailable

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!