Apr <var data-var='date'> 6</var>, <var data-var='time'>23:12</var> EDT Resolved - Webhook delivery delays have been fully resolved, and all queues have been successfully cleared. Event processing has returned to normal levels, and all pending events have been delivered, no data was lost. Thank you for your patience while we worked through this.Apr <var data-var='date'> 6</var>, <var data-var='time'>21:15</var> EDT Investigating - We are currently experiencing delays in webhook event delivery. Our team is actively investigating the issue and working to identify the root cause. No events are expected to be lost, but delivery may be delayed. We will provide an update as soon as we have more information or once mitigation steps are in place. Thank you for your patience.

Webhook Delivery Delays

Mar <var data-var='date'>11</var>, <var data-var='time'>15:00</var> EDT Resolved - The bulk of the delayed subscriptions have been identified and are queued up for delivery. We will provide additional details as they become available.Mar <var data-var='date'>11</var>, <var data-var='time'>10:30</var> EDT Update - We are continuing to work on delayed events and will provide another update once all the events have gone through.Mar <var data-var='date'>11</var>, <var data-var='time'>08:43</var> EDT Identified - We have resumed normal operation of sending real-time subscription events into Ordergroove and are now working on delayed events. Another update will be provided within 1 hour.Mar <var data-var='date'>11</var>, <var data-var='time'>07:25</var> EDT Investigating - We are currently investigating a delay with sending subscription data into Ordergroove for some of our Shopify stores. We will provide another update within the next 1 hour.

Subscription creation delayed

Jan <var data-var='date'>21</var>, <var data-var='time'>15:18</var> EST Resolved - The issue is resolved, and all services are operating normally. We apologize for the inconvenience and will publish a post-mortem within 24 hours which will be available via our CS team.Jan <var data-var='date'>21</var>, <var data-var='time'>14:45</var> EST Monitoring - We have identified a secondary cause of latency and addressed the issue. We are monitoring the system and will post an update shortly.Jan <var data-var='date'>21</var>, <var data-var='time'>14:05</var> EST Investigating - There is an ongoing spike in failed API responses. We are investigating again.Jan <var data-var='date'>21</var>, <var data-var='time'>13:32</var> EST Monitoring - We have identified and mitigated the effects of the latency. We are monitoring the performance of the system.Jan <var data-var='date'>21</var>, <var data-var='time'>13:22</var> EST Investigating - We are currently investigating response latency and will have an update in about 30 minutes.

We are experiencing a performance degradation in the API response times

Dec <var data-var='date'> 5</var>, <var data-var='time'>20:26</var> EST Resolved - We have replayed previously rejected orders.Dec <var data-var='date'> 5</var>, <var data-var='time'>15:14</var> EST Monitoring - On December 5th, 3:45 AM EST Shopify GraphQL APIs experienced an elevated error rate. The issue was resolved by 4:15 AM EST. During the duration of this issue recurring subscription orders failed to place and went into Rejected status. The root cause of this outage within Shopify was due to a CloudFlare issue which impacted a lot of other service providers that rely on this service. The steps for remediation is to safely retry these rejected orders from our side. We are continuing to run this recovery efforts on all impacted orders and will provide another update as soon as all recurring orders have been retried by our system.

Shopify Order Placement

Dec <var data-var='date'> 2</var>, <var data-var='time'>16:27</var> EST Resolved - This incident has been resolved.Dec <var data-var='date'> 2</var>, <var data-var='time'>16:26</var> EST Update - We are continuing to monitor for any further issues.Dec <var data-var='date'> 1</var>, <var data-var='time'>17:31</var> EST Monitoring - We have identified the issue, applied mitigating steps, and are monitoringDec <var data-var='date'> 1</var>, <var data-var='time'>15:48</var> EST Investigating - We’re currently experiencing a partial outage affecting SFTP report generation. Some scheduled reports are not being generated or delivered as expected. Our team is actively investigating and working to restore functionality as quickly as possible.

Partial Outage: SFTP Report Generation Delays

Nov <var data-var='date'> 5</var>, <var data-var='time'>17:49</var> EST Resolved - The incident has been resolved, and analytics is loading correctly for all accounts now.Nov <var data-var='date'> 5</var>, <var data-var='time'>12:16</var> EST Investigating - We are currently investigation the issue. For some people the Analytics page is not currently loading on the Ordergroove dashboard

Analytics page not loading consistently on Ordergroove Dashboard

Nov <var data-var='date'> 5</var>, <var data-var='time'>01:54</var> EST Resolved - The short-term fix has been rolled out to all affected merchants and product status is actively being updated in our system back to in stock for products where inventory is not tracked within Shopify.Nov <var data-var='date'> 4</var>, <var data-var='time'>18:44</var> EST Monitoring - On November 1st, there was an update to the data being sent over from Shopify which negatively impacted our ability to mark products as in stock in our system. The issue only affects products where inventory is not tracked within Shopify. We have been working since then to implement a short-term solutions and have been rolling it out gradually to all merchants who we found to have been impacted by this change. This short-term fix requires our platform to pull additional product details from Shopify so we have been very gradually rolling it out and measuring system performance. We are going to provide another update as soon as this short-term fix has been rolled out to all stores.

Products showing as out of stock in Ordergroove

Oct <var data-var='date'>29</var>, <var data-var='time'>15:55</var> EDT Resolved - The incident was caused by elevated I/O latency in GCE Persistent Disk within a single region, which impacted our database performance and led to increased application response times. We mitigated the issue by relocating our database instances out of the affected zone. Our team is continuing to work with Google to ensure the underlying issue is fully resolved, but this is no longer impacting Ordergroove services. The incident began at 2:20 PM EST, mitigation was completed by 2:59 PM EST, and full recovery was achieved at 3:18 PM EST.Oct <var data-var='date'>29</var>, <var data-var='time'>15:22</var> EDT Update - We’ve identified and resolved the root cause of the issue. Latency levels are steadily improving, and we’re closely monitoring system performance to ensure continued stability. We’ll provide another update by 4 PM EST.Oct <var data-var='date'>29</var>, <var data-var='time'>14:47</var> EDT Investigating - We're investigating the issue affecting multiple services. We're seeing increased latencies and some requests fail to complete. We'll provide an update by 3:15PM EST.

Intermittent performance degradation

Sep <var data-var='date'>16</var>, <var data-var='time'>23:08</var> EDT Resolved - The remediation has been successfully completed. All missed Order Reminder events for the September 15 run have been replayed, with safeguards in place to prevent any stale actions. The September 16 run has executed successfully as well, and we expect future runs to proceed smoothly starting tomorrow. We will conduct a post mortem to strengthen monitoring for this process going forward. Thank you for your patience as we resolved this issue.Sep <var data-var='date'>16</var>, <var data-var='time'>15:40</var> EDT Monitoring - Order Reminder Events 9/15/2025: Yesterday, on 9/15/2025, Order Reminder events were not sent to Klaviyo or delivered through Ordergroove Webhook system What happened We upgraded our queueing system for the application that produces Order Reminder events. During the upgrade, a subset of consumers did not update correctly and were listening to the wrong system, which caused events to be missed. What we’re doing We are carefully replaying all missed Order Reminder events today. To avoid outdated actions, reminders will only be sent if the related order has not already been attempted. We expect the remediation will be fully completed today Next steps Once remediation is complete, we will run an internal post mortem to strengthen monitoring and ensure full coverage of the process going forward.

Order Reminder Events were missed on 9/15/2025

Aug <var data-var='date'>21</var>, <var data-var='time'>07:00</var> EDT Resolved - Between 7:04 AM - 9:05 AM ET we noticed degraded performance in our async workflow for creating Subscriptions. All subscriptions were created successfully. We have identified the root cause and are working on solutions to prevent this from happening again.

Degraded Subscription Creation Performance

Aug <var data-var='date'> 4</var>, <var data-var='time'>02:41</var> EDT Resolved - Our APIs were experiencing slow response time between 1:50am EST to 2:28am EST

API response time degradation

Jun <var data-var='date'>12</var>, <var data-var='time'>17:07</var> EDT Resolved - Google Cloud Platform has confirmed that the incident is now resolved and we're starting to see recovery across all affected systems. GCP incident: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsWJun <var data-var='date'>12</var>, <var data-var='time'>15:18</var> EDT Update - Although we do not have an exact ETA on recovery, it appears that Google Cloud Platform is starting to recover.Jun <var data-var='date'>12</var>, <var data-var='time'>15:10</var> EDT Update - Although we do not have an exact ETA on recovery, it appears that Google Cloud Platform is starting to recover.Jun <var data-var='date'>12</var>, <var data-var='time'>14:21</var> EDT Investigating - We are experiencing some connectivity issues related to our cloud hosting provider. We are investigating this incident and will provide further updates as soon as they become available. This issue is impacting some internal systems that are responsible for migrating data into Ordergroove and delivering events. This issue also has implications on our Shopify integration. More details regarding this outage can be found here: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

The google cloud platform is experiencing issues

Apr <var data-var='date'>20</var>, <var data-var='time'>05:30</var> EDT Completed - The scheduled maintenance has been completed.Apr <var data-var='date'>20</var>, <var data-var='time'>04:30</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Apr <var data-var='date'> 8</var>, <var data-var='time'>14:03</var> EDT Scheduled - Scheduled maintenance will occur on April 20th between 4:30 AM and 5:30 AM EST to enhance platform scalability and security. We do not expect downtime, and no user action is required. Please note: a short period of approximately 10–15 seconds may occur during which some write operations to the api, restapi, data, and sc API endpoints might experience higher than normal error rates. For merchants using the sync subscription creation method, a small fraction of requests may return a 400 error code, but these are expected to retry automatically without intervention.

Platform maintenance - April 20 2025

Mar <var data-var='date'>24</var>, <var data-var='time'>07:00</var> EDT Completed - The scheduled maintenance has been completed.Mar <var data-var='date'>24</var>, <var data-var='time'>06:00</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Feb <var data-var='date'> 4</var>, <var data-var='time'>13:42</var> EST Scheduled - Please note that the TLS ciphers TLS_RSA_WITH_AES_128_GCM_SHA256 and TLS_RSA_WITH_AES_256_GCM_SHA384 will be decommissioned in 45 days as part of our ongoing efforts to enhance the security and stability of our platform. These ciphers are being phased out to align with modern cryptographic best practices and ensure stronger protection for data in transit. The ciphers will be removed from the following domains: - rc3.ordergroove.com - sc.ordergroove.com - api.ordergroove.com - restapi.ordergroove.com - om.ordergroove.com - offers.ordergroove.com We recommend updating your configurations to use more secure cipher suites to avoid any service disruptions. The above ciphers will be disabled on March 24, 2025. Please make all necessary changes prior to that date. If you have any questions or require assistance, please reach out to our support team.

TLS cipher suite updates

Mar <var data-var='date'> 8</var>, <var data-var='time'>21:12</var> EST Resolved - We have completed the main recovery efforts related to this incident. The small amount of previously affected recurring orders are starting to flow through the placement process.Mar <var data-var='date'> 8</var>, <var data-var='time'>05:42</var> EST Update - All valid Shopify events have been replayed successfully. We are now entering the next and final phase of the recovery effort. There is a small number of recurring orders that were impacted by this issue which will be retried over the next 24-48 hour period. We are taking extra precautionary measures to ensure that we only retry orders which were not properly processed by Shopify. No action needs to be taken and we will provide a final update once the task is completed by our team.Mar <var data-var='date'> 7</var>, <var data-var='time'>17:53</var> EST Update - All of the services have recovered and message delivery is back to normal. We are in the process of replaying any missed transactions. No action needs to be taken, our team will make sure all events will be reprocessed. We will provide further updates as progress is made.Mar <var data-var='date'> 7</var>, <var data-var='time'>11:11</var> EST Monitoring - We have identified an issue related to internal systems that process Shopify events from 03/06/2025 4PM EST and until 03/07/2025 9AM EST. The underlying issue has been resolved and we are actively investigating all impacted areas that require manual intervention. No action needs to be taken, our team will make sure all events will be reprocessed. Areas impacted: - Increase in recurring order placement retry attempts due to connection errors - Delayed delivery of events and updates tied to certain customer actions such as change of frequency or payment updates Next Steps: - We are working diligently to ensure that all delayed events are being processed - We are reviewing all potentially impacted recurring orders to ensure proper placement We will provide further updates throughout the day

Partial degradation of service with merchants on the Shopify platform

Oct <var data-var='date'>27</var>, <var data-var='time'>05:30</var> EDT Completed - The scheduled maintenance has been completed.Oct <var data-var='date'>27</var>, <var data-var='time'>04:30</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Oct <var data-var='date'>14</var>, <var data-var='time'>15:02</var> EDT Scheduled - As part of our ongoing efforts to keep the Ordergroove platform scalable and secure, we are planning to conduct our next routine maintenance on October 27th between 4:30 AM and 5:30 AM EST. This maintenance will result in a 10–15-second window during which a subset of requests made to Ordergroove’s API endpoints may experience elevated error rates. Subscription creation requests may return error code 400, but they are expected to be retried automatically. No action is required on your side.

Platform maintenance - October 27 2024

Aug <var data-var='date'> 6</var>, <var data-var='time'>23:00</var> EDT Completed - The scheduled maintenance has been completed.Aug <var data-var='date'> 6</var>, <var data-var='time'>22:15</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Jul <var data-var='date'>16</var>, <var data-var='time'>18:11</var> EDT Scheduled - As part of our ongoing efforts to keep the Ordergroove platform scalable and secure we are planning our next routine maintenance to be conducted on Aug 6th between 10:15PM and 11:00PM EST. During the maintenance window the SFTP server will not be accessible - this means product feed processing and batch order placement will be paused. Any scheduled reports will be uploaded to the SFTP server once the maintenance completes. Merchants with uploads scheduled during the maintenance window will need to be retried.

Platform maintenance - August 6 2024

Jul <var data-var='date'>12</var>, <var data-var='time'>18:41</var> EDT Resolved - The incident has been resolved and the backlog has been entirely processed. We apologize for the delay in processing subscription creations, we have identified multiple steps to prevent this incident from happening again and we will prioritize those items to prevent any reoccurrence.Jul <var data-var='date'>12</var>, <var data-var='time'>11:17</var> EDT Identified - The issue has been identified and a fix is being implemented.Jul <var data-var='date'>12</var>, <var data-var='time'>08:00</var> EDT Investigating - We have identified a delay in our subscription creation processing due to higher volume than usual. We are looking into this and scaling up our infrastructure in order to process subscriptions in the queue faster.

Delay in Subscription Creation processing

Jul <var data-var='date'> 7</var>, <var data-var='time'>05:30</var> EDT Completed - The scheduled maintenance has been completed.Jul <var data-var='date'> 7</var>, <var data-var='time'>04:30</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Jun <var data-var='date'>24</var>, <var data-var='time'>17:56</var> EDT Scheduled - As part of our ongoing efforts to keep the Ordergroove platform scalable and secure we are planning our next routine maintenance to be conducted on July 7th between 4:30AM and 5:30AM EST. The impact will be a 10 - 15 second window when a subset of requests made to Ordergrove's API endpoints will experience elevated error rates. Subscription creation requests may return with error code 400, but they are expected to be retried automatically. There is no action required on the merchants' side.

Platform maintenance - July 7 2024

May <var data-var='date'>14</var>, <var data-var='time'>15:27</var> EDT Resolved - This incident has been resolved.May <var data-var='date'>14</var>, <var data-var='time'>14:02</var> EDT Monitoring - The delay is now down to under 30 minutes and the system is churning through the backlog of requests. We are still actively monitoring this delay.May <var data-var='date'>14</var>, <var data-var='time'>13:00</var> EDT Investigating - We noticed a delay of up to 1 hour for tasks that were submitted between 12.30 PM and 1.30 PM ET. We are actively working on scaling up asynchronous worker to process the backlog of subscription creation requests.

Slight delay in Subscription Creation processing

May <var data-var='date'> 5</var>, <var data-var='time'>05:30</var> EDT Completed - The scheduled maintenance has been completed.May <var data-var='date'> 5</var>, <var data-var='time'>04:30</var> EDT In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Apr <var data-var='date'>22</var>, <var data-var='time'>15:08</var> EDT Scheduled - As part of our ongoing efforts to keep the Ordergroove platform scalable and secure we are planning our next routine maintenance to be conducted on May 5th between 4:30AM and 5:30AM EST. The impact will be a 10 - 15 second window when a subset of requests made to Ordergrove's API endpoints will experience elevated error rates. Subscription creation requests may return with error code 400, but they are expected to be retried automatically. There is no action required on the merchants' side.

Platform maintenance - May 5 2024

Apr <var data-var='date'>30</var>, <var data-var='time'>19:15</var> EDT Resolved - The queue has been fully caught up and all subscriptions were processed with some delays.Apr <var data-var='date'>30</var>, <var data-var='time'>15:30</var> EDT Monitoring - We received an increased volume in the subscription creation queue in multiple waves throughout the day. All the requests are queued and are being processed continuously still. We are monitoring this very closely.

Slight Delay in Subscription Creation

Apr <var data-var='date'>12</var>, <var data-var='time'>13:43</var> EDT Resolved - We are pleased to inform you that the incident affecting the loading of analytics data has been successfully resolved. Analytics services are now operating as expected for all merchants. Users who are currently logged in to the Ordergroove Dashboard may need to log out and then log back in to ensure visibility of the analytics data. Should you encounter any further issues or have any questions, please do not hesitate to reach out to our support team (support@ordergroove.com)Apr <var data-var='date'>12</var>, <var data-var='time'>12:26</var> EDT Investigating - We are currently experiencing an issue impacting the loading of analytics data for some users on our Dashboard. Our team is taking immediate steps to restore normal functionality. We will provide further updates as soon as the issue is fully addressed.

Ordergroove Analytics Dashboard not showing data

Mar <var data-var='date'>13</var>, <var data-var='time'>15:43</var> EDT Resolved - The incident has been resolvedMar <var data-var='date'>13</var>, <var data-var='time'>15:19</var> EDT Identified - We are experiencing a slight delay in subscription creation. We are working through the current issue and should have it resolved shortly.

Slight delay in subscription creation

Feb <var data-var='date'>29</var>, <var data-var='time'>11:35</var> EST Resolved - The issue has been resolved. We expect the majority of subscription data to catch up shortly. There will be a small subset of subscription data that will come through within the course of the next few hours.Feb <var data-var='date'>29</var>, <var data-var='time'>10:44</var> EST Investigating - We are currently investigating a delay in subscription creation for merchants on the Shopify platform

Is Ordergroove down?

Ordergroove status is UP

Lastest published events

Webhook Delivery Delays

Subscription creation delayed

We are experiencing a performance degradation in the API response times

Ordergroove Services, Components and Regions

Recent Ordergroove events

Webhook Delivery Delays

Subscription creation delayed

We are experiencing a performance degradation in the API response times

Shopify Order Placement

Partial Outage: SFTP Report Generation Delays

Analytics page not loading consistently on Ordergroove Dashboard

Get your FREE account now,
5 monitors included!

Is Ordergroove down?

Ordergroove status is UP

Lastest published events

Webhook Delivery Delays

Subscription creation delayed

We are experiencing a performance degradation in the API response times

Ordergroove Services, Components and Regions

Recent Ordergroove events

Webhook Delivery Delays

Subscription creation delayed

We are experiencing a performance degradation in the API response times

Shopify Order Placement

Partial Outage: SFTP Report Generation Delays

Analytics page not loading consistently on Ordergroove Dashboard

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!