Sep <var data-var='date'>26</var>, <var data-var='time'>13:13</var> CEST Resolved - Between 13:13 and 13:42 UTC on 26 September, some customers experienced errors or delays with order processing. Our team identified the issue and restored normal service at 13:42 UTC. We are reviewing the incident and taking steps to prevent it from happening again.

Order processing delays

Sep <var data-var='date'>15</var>, <var data-var='time'>19:00</var> CEST Resolved - Between 16:55 and 17:14 UTC on 15 September, some customers experienced errors or delays with order processing. Our team identified the issue and restored normal service at 17:14 UTC. We are reviewing the incident and taking steps to prevent it from happening again.

Order processing errors

Apr <var data-var='date'>23</var>, <var data-var='time'>20:10</var> CEST Resolved - This incident has been resolved. Apr <var data-var='date'>23</var>, <var data-var='time'>20:03</var> CEST Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>23</var>, <var data-var='time'>19:55</var> CEST Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'>23</var>, <var data-var='time'>19:32</var> CEST Update - We are continuing to investigate this issue. Apr <var data-var='date'>23</var>, <var data-var='time'>19:09</var> CEST Investigating - We are currently experiencing an issue where menu pushes are failing. Our team is aware and actively investigating the root cause. We will provide updates as more information becomes available.

Menu Pushes Failing

Dec <var data-var='date'> 5</var>, <var data-var='time'>10:20</var> CET Resolved - The incident has been resolved and our services and products are fully operational again. Dec <var data-var='date'> 5</var>, <var data-var='time'>10:15</var> CET Monitoring - A fix has been implemented by Cloudflare and our systems are recovering, we are monitoring closely. Dec <var data-var='date'> 5</var>, <var data-var='time'>10:09</var> CET Investigating - We are currently investigating a major, global outage reported by our cloud provider, Cloudflare. This outage is affecting their network infrastructure, which is severely impact the availability, performance, and accessibility of some of our products and services.

Global Cloudflare Outage

Nov <var data-var='date'>18</var>, <var data-var='time'>17:20</var> CET Resolved - We noticed a service interruption impacting our Retail Order Processing and related front-end UIs in several regions. The root cause was identified as a major global outage affecting our critical third-party provider, Cloudflare. The incident started at roughly 1:00 PM UTC. Importantly, our restaurant order processing channels were unaffected and continued to function normally throughout this period. Our engineering teams were immediately focused on developing and implementing a service workaround. As of 2:21 PM UTC, all Retail Order Processing services and front-end UIs were fully functional and stable. We apologize for the disruption caused by this external event.

Intermittent Retail Order Processing Interruption

Jun <var data-var='date'>12</var>, <var data-var='time'>23:40</var> CEST Resolved - This incident has been resolved. Jun <var data-var='date'>12</var>, <var data-var='time'>20:48</var> CEST Monitoring - A fix has been implemented and we are monitoring the results. Jun <var data-var='date'>12</var>, <var data-var='time'>20:20</var> CEST Investigating - We are currently experiencing a service outage due to an incident with our cloud provider. This disruption is impacting multiple systems, and our team is actively working to assess the situation and restore services as quickly as possible

Outage Due to Cloud Provider Issue

Mar <var data-var='date'>16</var>, <var data-var='time'>22:33</var> CET Resolved - This incident has been resolved Mar <var data-var='date'>16</var>, <var data-var='time'>21:02</var> CET Monitoring - A fix has been implemented and we are monitoring the results. Mar <var data-var='date'>16</var>, <var data-var='time'>19:45</var> CET Investigating - We are currently investigating an issue within our API that is causing delays in order processing. Our engineering team is actively working to identify the root cause and resolve the issue as quickly as possible.

Slow order processing

Nov <var data-var='date'> 1</var>, <var data-var='time'>21:00</var> CET Resolved - Deliverect experienced degraded performance between 20:08 UTC and 20:53 UTC. Most customers should not have been impacted, but a small number may have experienced dropped orders, delayed orders, or other issues with Deliverect functionalities. The issue has been resolved and we are working to prevent future occurrences.

Degraded Performance

Jul <var data-var='date'>18</var>, <var data-var='time'>19:53</var> CEST Resolved - The overall degradation in API response times has been solved. The fix we implemented has fixed the issue. Jul <var data-var='date'>18</var>, <var data-var='time'>19:30</var> CEST Monitoring - A fix has been implemented and we are monitoring the results. Jul <var data-var='date'>18</var>, <var data-var='time'>19:20</var> CEST Investigating - We are currently investigating the issue.

Degraded API Performance

Apr <var data-var='date'>30</var>, <var data-var='time'>06:30</var> CEST Resolved - Start Time: April 30, 2024, 06:41 AM CEST (04:41 AM UTC) End Time: April 30, 2024, 06:48 AM CEST (04:48 AM UTC) We are backfilling an incident that occurred earlier this morning where the Deliverect API experienced degraded performance for a short period of time (approximatively 7 minutes). This resulted in slower response times and potential delayed order processing. Deliverect API is now functioning normally and we apologize for any inconvenience this may have caused.

Deliverect API Degraded Performance

Mar <var data-var='date'>13</var>, <var data-var='time'>19:30</var> CET Resolved - Customers might have experienced delay on order processing from 18:12 to 18:36 UTC+0 The incident is now resolved and we are back to normal order processing activities.

Order processing delay

Jan <var data-var='date'>19</var>, <var data-var='time'>00:18</var> CET Resolved - Between 00:18 and 00:27 UTC+1, our system degraded and performance slowly declined. The incident was caused by a third-party provider experiencing issues. We sincerely apologize for the inconvenience.

Processing Delays

Oct <var data-var='date'> 4</var>, <var data-var='time'>21:45</var> CEST Resolved - This incident has been resolved. Oct <var data-var='date'> 4</var>, <var data-var='time'>21:11</var> CEST Monitoring - We've identified the cause of the incident and applied a fix for it. We'll keep monitoring. Oct <var data-var='date'> 4</var>, <var data-var='time'>19:42</var> CEST Identified - We experienced order processing delays, but we can confirm that now all processing activities are back to normal. As a way to improve order processing performance, the menu-push feature has been paused until we have further information about its impact on the order flow. Oct <var data-var='date'> 4</var>, <var data-var='time'>19:15</var> CEST Investigating - We are currently investigating this issue.

Sep <var data-var='date'>18</var>, <var data-var='time'>09:00</var> CEST Completed - The scheduled maintenance has been completed. Sep <var data-var='date'>18</var>, <var data-var='time'>08:00</var> CEST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>15</var>, <var data-var='time'>13:23</var> CEST Scheduled - We will be performing scheduled maintenance in our Tunnel service on September 18th at 06:00 UTC. This update will only impact customers using our Tunnel service, there is a possibility of a slight increase in latency and disconnections during this maintenance window. Please make sure you monitor your Deliverect frontend orders and manually input any orders that fail to integrate. We apologize for any inconvenience and thank you for your support as we continue to improve our integrations.

Tunnel Service Maintenance Work

Aug <var data-var='date'>29</var>, <var data-var='time'>08:00</var> CEST Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>29</var>, <var data-var='time'>07:02</var> CEST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>29</var>, <var data-var='time'>01:12</var> CEST Scheduled - We will be undergoing maintenance during this time to improve performance. Hence there is a possibility of a slight increase in latency during this maintenance window for end-users. But no downtime is expected.

Maintenance

Aug <var data-var='date'>24</var>, <var data-var='time'>11:00</var> CEST Resolved - This incident has been resolved. Aug <var data-var='date'>24</var>, <var data-var='time'>10:33</var> CEST Update - We are continuing to monitor for any further issues. Aug <var data-var='date'>24</var>, <var data-var='time'>09:44</var> CEST Monitoring - A fix has been implemented and we are monitoring the results Aug <var data-var='date'>24</var>, <var data-var='time'>08:23</var> CEST Identified - The issue has been identified and a fix is being implemented. Aug <var data-var='date'>24</var>, <var data-var='time'>07:35</var> CEST Investigating - We are currently investigating this issue.

Aug <var data-var='date'>16</var>, <var data-var='time'>17:18</var> CEST Resolved - Operations were restored completely for affected customers. We are currently actively monitoring our systems. Aug <var data-var='date'>16</var>, <var data-var='time'>16:34</var> CEST Monitoring - A fix has been deployed and we are monitoring the results Aug <var data-var='date'>16</var>, <var data-var='time'>16:18</var> CEST Identified - After a thorough investigation the issue has been identified and a fix is being implemented. Aug <var data-var='date'>16</var>, <var data-var='time'>16:05</var> CEST Update - We are continuing to investigate this issue. The impact was that our queueing systems was struggling to process orders to the POS systems, and restarting them only partially helped recovery. Aug <var data-var='date'>16</var>, <var data-var='time'>16:04</var> CEST Investigating - Our monitoring system detected a sudden traffic decrease on one of our systems, which was impacting order processing. Requests are not correctly arriving to our systems we are actively investigating and setting all efforts to quickly resolve it.

Service Unavailability

Jul <var data-var='date'>31</var>, <var data-var='time'>11:17</var> CEST Resolved - This incident has been resolved. Jul <var data-var='date'>28</var>, <var data-var='time'>14:45</var> CEST Update - We are continuing to monitor for any further issues. Jul <var data-var='date'>28</var>, <var data-var='time'>14:44</var> CEST Update - We are monitoring the fix that was applied, and are implementing changes to avoid this from occurring in the future. Jul <var data-var='date'>28</var>, <var data-var='time'>14:44</var> CEST Monitoring - Starting at 10:28 (UTC+0), we faced potential delays over a period of 45min. All orders were eventually processed and the incident has been addressed and fixed.

Degraded performance for DMA and KDS solutions

May <var data-var='date'>23</var>, <var data-var='time'>09:54</var> CEST Resolved - This incident has been resolved May <var data-var='date'>22</var>, <var data-var='time'>15:33</var> CEST Update - We are continuing to monitor for any further issues. May <var data-var='date'>22</var>, <var data-var='time'>11:30</var> CEST Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'>21</var>, <var data-var='time'>22:00</var> CEST Identified - You may notice that our Frontend is slower than usual at certain times, when it's the case our API takes more time than usual to respond to requests. This can lead to some orders being rejected or duplicated. Our team is actively working on resolving the problem and restoring normal operations as quickly as possible. May <var data-var='date'>21</var>, <var data-var='time'>19:00</var> CEST Investigating - Starting May 21, 19:00 CET time we have been experiencing degraded performance with our API, which is impacting the speed and responsiveness of our frontend.

Customers and partners may experience API and Frontend slowness

May <var data-var='date'>19</var>, <var data-var='time'>20:00</var> CEST Resolved - Customers and partners might have experienced slowness on the Deliverect Frontend and non-critical API calls between 8:00 PM CET and 9:45 PM CET. A fix has been implemented and the incident is now resolved.

Customers using Frontend and Deliverect API might have experienced slowness

Feb <var data-var='date'> 2</var>, <var data-var='time'>10:00</var> CET Resolved - An issue with a TLS certificate impacted customers using the Tunnel but did not affect customers using DMA. We experienced system instability for around 50 minutes. The issue is related to the expiry of a TLS certificate, which is normally automatically renewed by our processes. Our Infrastructure and Support teams were notified of the incident and immediately began investigating the cause. The automation responsible for renewing this specific certificate was identified as the root cause of the incident. The team worked to replace manually the expired certificate with a new one so customers can use the tunnel again without disruption. The code responsible for the certificate renewal is now being reviewed to avoid this bug from happening again.

Tunnel service outage

Nov <var data-var='date'> 7</var>, <var data-var='time'>19:45</var> CET Resolved - We’ve noticed some order updates did not reach the DMA. This issue was noticed between 15:19:45 (UTC+0) and 18:40:00 (UTC+0). We’ve identified the issue and applied a fix to it. We will keep monitoring.

Updates failed to reach DMA

Oct <var data-var='date'>19</var>, <var data-var='time'>11:50</var> CEST Resolved - This incident has been resolved. Users may have experienced slowness and duplicate orders from 10:30 to 10:58 UTC+2 Oct <var data-var='date'>19</var>, <var data-var='time'>10:57</var> CEST Monitoring - A fix has been implemented and we are monitoring the results. Oct <var data-var='date'>19</var>, <var data-var='time'>10:30</var> CEST Investigating - We have identified a performance issue causing slowness for some frontend users. As a workaround, users can do a hard refresh.

Degraded performance

Jun <var data-var='date'>21</var>, <var data-var='time'>08:30</var> CEST Resolved - A major cloud services provider experienced a global outage for approximately 30 minutes. Even if Deliverect critical systems were up and running during the incident, customers may have experienced service unavailability if one of their POS or Delivery Channel providers was impacted by the incident.

Connectivity issues

Jun <var data-var='date'>15</var>, <var data-var='time'>13:28</var> CEST Resolved - We've experienced degraded performance with order processing from 09:47 AM (GMT+0) to 11:08 AM (GMT+0). The only product impacted was Online Ordering, where some stores have faced service interruption.

Is Deliverect down?

Deliverect status is UP

Lastest published events

Order processing delays

Order processing errors

Menu Pushes Failing

Deliverect Services, Components and Regions

Recent Deliverect events

Order processing delays

Order processing errors

Menu Pushes Failing

Global Cloudflare Outage

Intermittent Retail Order Processing Interruption

Outage Due to Cloud Provider Issue