Mar <var data-var='date'>24</var>, <var data-var='time'>15:30</var> UTC Resolved - Marketplace API experienced elevated latency and error rates. The incident is resolved.

Elevated error rates in Marketplace API

Mar <var data-var='date'>20</var>, <var data-var='time'>17:05</var> UTC Resolved - From 16:00 to 16:50 UTC on March 20th we saw elevated error rates in Marketplace API. The incident is now solved and we are monitoring the situation.

Feb <var data-var='date'>11</var>, <var data-var='time'>16:00</var> UTC Resolved - We saw increased latency in the Marketplace API and addressed the issue. It went back to normal after 20 minutes. We keep monitoring the situation.

Elevated latency in Marketplace API

Jan <var data-var='date'>19</var>, <var data-var='time'>17:16</var> UTC Resolved - This incident has been resolved.Jan <var data-var='date'>19</var>, <var data-var='time'>16:48</var> UTC Update - We are continuing to monitor for any further issues.Jan <var data-var='date'>19</var>, <var data-var='time'>16:45</var> UTC Monitoring - We identified the issue and took action. We are seeing API latency coming down. We keep monitoring the situation.Jan <var data-var='date'>19</var>, <var data-var='time'>16:22</var> UTC Investigating - We're experiencing increased API latency and are currently looking into the issue.

Increased API latency

Nov <var data-var='date'>12</var>, <var data-var='time'>15:18</var> UTC Completed - The scheduled maintenance has been completed.Nov <var data-var='date'>12</var>, <var data-var='time'>14:49</var> UTC Verifying - Verification is currently underway for the maintenance items.Nov <var data-var='date'>12</var>, <var data-var='time'>14:24</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Nov <var data-var='date'>12</var>, <var data-var='time'>14:04</var> UTC Scheduled - We will be performing database maintenance. During this time, a partial outage is expected for the following components: Events, background processing, search, etc. In addition, attempts to perform database writes (e.g., POST API requests) will temporarily fail with a 503 status code. We expect this interruption to last less than a minute.

Database maintenance

Oct <var data-var='date'>21</var>, <var data-var='time'>05:50</var> UTC Resolved - This incident has been resolved.Oct <var data-var='date'>20</var>, <var data-var='time'>10:14</var> UTC Monitoring - There is ongoing incident in Amazon Web Services US East 1 region. While Sharetribe own services are not hosted there and are not directly affected, some 3rd party integrations may be affected, including Chargebee (our Sharetribe subscription management service) and Intercom (customer support platform). For details on the status of those services, see: * Chargebee - https://status.chargebee.com/ * Intercom - https://www.intercomstatus.com/

Partial degradation in Sharetribe subscription management and customer support system

Dec <var data-var='date'> 3</var>, <var data-var='time'>11:23</var> UTC Completed - Maintenance completed!Dec <var data-var='date'> 3</var>, <var data-var='time'>11:20</var> UTC Verifying - Maintenance complete, verifying the health of all systems.Dec <var data-var='date'> 3</var>, <var data-var='time'>11:15</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Dec <var data-var='date'> 3</var>, <var data-var='time'>11:14</var> UTC Scheduled - We are performing maintenance on the database that stores events. This may result in short period of increased latency or error rates for the Integration API /events/query endpoint.

Maintenance of events database

Oct <var data-var='date'>10</var>, <var data-var='time'>09:51</var> UTC Resolved - There was an increased delay in the Event processing pipeline between 9:25 - 9:40. The issue is now resolved and the Event processing pipeline has caught up.

Delay in Event processing

Oct <var data-var='date'> 6</var>, <var data-var='time'>16:15</var> UTC Resolved - We experienced an elevated level of API errors around 16:15-16:20 UTC.

Elevated API Errors

Sep <var data-var='date'>30</var>, <var data-var='time'>18:30</var> UTC Resolved - Around 18:20 to 18:25 UTC we encountered elevated latency in Marketplace and Integration APIs, caused by a database component failover.

Elevated latency in Marketplace and Integration APIs

Sep <var data-var='date'>20</var>, <var data-var='time'>21:00</var> UTC Resolved - From 21:15 to 21:25 UTC on September 20th we saw elevated error rates in Marketplace and Integration APIs. The incident was caused by faulty autoscaling of our search system.

Elevated error rates in Marketplace and Integration APIs

Sep <var data-var='date'> 6</var>, <var data-var='time'>00:30</var> UTC Resolved - The search engine was partially unavailable between 00:19 and 01:08 UTC. This affected all the API endpoints that use search, e.g., the Marketplace API `/listings/query` endpoint. During this time, the endpoints that use search returned 5xx responses. Our system couldn't recover from the partial outage without manual intervention. We keep investigating why the system didn't recover automatically and how we can improve it to prevent similar outages in the future.

Search temporary unavailable

Sep <var data-var='date'> 3</var>, <var data-var='time'>10:29</var> UTC Resolved - This incident has been resolved.Sep <var data-var='date'> 3</var>, <var data-var='time'>10:08</var> UTC Identified - The issue has been identified and a fix is being implemented.Sep <var data-var='date'> 3</var>, <var data-var='time'>09:49</var> UTC Investigating - We are currently investigating an issue causing delays in Event processing

Delays in Event processing

Aug <var data-var='date'> 1</var>, <var data-var='time'>18:45</var> UTC Resolved - This incident has been resolved.Aug <var data-var='date'> 1</var>, <var data-var='time'>12:31</var> UTC Monitoring - We rolled back some recent changes, which we suspect caused the issue. All systems are operating normally. We will keep monitoring the situation.Aug <var data-var='date'> 1</var>, <var data-var='time'>12:04</var> UTC Update - Increased delay in processing background jobsAug <var data-var='date'> 1</var>, <var data-var='time'>11:51</var> UTC Investigating - We are seeing elevated error rates in API responses. We are investigating the issue.

Elevated API error rates

May <var data-var='date'>23</var>, <var data-var='time'>20:30</var> UTC Resolved - Up to 30 minutes of delay in background job processing between 23:31 - 23:59 UTC.

Delay in background job processing

Mar <var data-var='date'>26</var>, <var data-var='time'>19:00</var> UTC Resolved - Up to 30 minutes of delay in background job processing between 19:07 - 19:39 UTC.

Mar <var data-var='date'> 4</var>, <var data-var='time'>19:20</var> UTC Resolved - This incident has been resolved.Mar <var data-var='date'> 4</var>, <var data-var='time'>17:05</var> UTC Monitoring - Our mitigation was effective, the event extraction service is catching up.Mar <var data-var='date'> 4</var>, <var data-var='time'>16:48</var> UTC Identified - There is a delay in processing new events. New events will become available in queries with some delay. We've deployed mitigation.

Delay in event processing

Jan <var data-var='date'>25</var>, <var data-var='time'>15:05</var> UTC Resolved - This incident has been resolved.Jan <var data-var='date'>25</var>, <var data-var='time'>15:04</var> UTC Monitoring - There is currently approximately 20 minutes delay in user data indexing and the indexing is catching up.Jan <var data-var='date'>25</var>, <var data-var='time'>14:56</var> UTC Identified - We've mitigated the issue and it only affects users data, but not listings. This, therefore, affects only the Integration API and Console.Jan <var data-var='date'>25</var>, <var data-var='time'>14:28</var> UTC Investigating - We're experiencing high delay in search indexing. Listing and user queries may match against stale data.

Delay in search indexing

Jan <var data-var='date'> 4</var>, <var data-var='time'>09:13</var> UTC Resolved - The attack has subsided. The mitigations we put in place were successful in keeping Sharetribe systems fully operational.Jan <var data-var='date'> 3</var>, <var data-var='time'>09:02</var> UTC Monitoring - We are under a DDoS attack but have managed to mitigate the attack and system is currently operating normally. We continue to monitor the issue.Jan <var data-var='date'> 3</var>, <var data-var='time'>08:52</var> UTC Update - We are continuing to investigate this issue.Jan <var data-var='date'> 3</var>, <var data-var='time'>08:23</var> UTC Update - We are continuing to investigate this issue.Jan <var data-var='date'> 3</var>, <var data-var='time'>08:15</var> UTC Update - The system is under a potential DDoS attack. We are investigating the situation and trying to bring the system back up.Jan <var data-var='date'> 3</var>, <var data-var='time'>08:06</var> UTC Update - We are continuing to investigate this issue.Jan <var data-var='date'> 3</var>, <var data-var='time'>07:53</var> UTC Update - We are continuing to investigate this issue.Jan <var data-var='date'> 3</var>, <var data-var='time'>07:39</var> UTC Update - We are continuing to investigate this issue.Jan <var data-var='date'> 3</var>, <var data-var='time'>07:31</var> UTC Investigating - We are investigating an issue making the system partly unresponsive

We are investigating an issue making the system partly unresponsive

Dec <var data-var='date'>29</var>, <var data-var='time'>08:11</var> UTC Resolved - This incident has been resolved.Dec <var data-var='date'>29</var>, <var data-var='time'>06:05</var> UTC Update - We are continuing to monitor for any further issues.Dec <var data-var='date'>29</var>, <var data-var='time'>05:30</var> UTC Update - We are continuing to monitor for any further issues.Dec <var data-var='date'>29</var>, <var data-var='time'>05:29</var> UTC Monitoring - The APIs are up and running again; we will continue investigating the root cause and monitoring the system.Dec <var data-var='date'>29</var>, <var data-var='time'>04:39</var> UTC Investigating - We are currently investigating an issue with Marketplace API and Integration APIs being unresponsive.

Markertplace APIs and Integration APIs are unresponsive

May <var data-var='date'>31</var>, <var data-var='time'>22:44</var> UTC Resolved - This incident has been resolved.May <var data-var='date'>31</var>, <var data-var='time'>22:23</var> UTC Monitoring - We have identified the issue and implemented a fix. We keep monitoring the situation.May <var data-var='date'>31</var>, <var data-var='time'>20:30</var> UTC Investigating - We are currently seeing slowness in search indexing. We are investigating the issue.

Slowness in search indexing

Apr <var data-var='date'>20</var>, <var data-var='time'>03:40</var> UTC Resolved - This incident has been resolved.Apr <var data-var='date'>20</var>, <var data-var='time'>03:32</var> UTC Update - The indexing process is catching up, there is currently approximately 30 minutes of unindexed data.Apr <var data-var='date'>20</var>, <var data-var='time'>03:22</var> UTC Monitoring - Fix is implemented and indexing process is catching up with the data. This process may still take a while.Apr <var data-var='date'>20</var>, <var data-var='time'>02:58</var> UTC Identified - The issue has been identified and a fix is being implemented.Apr <var data-var='date'>20</var>, <var data-var='time'>02:45</var> UTC Investigating - Search indexing is experiencing delays, listing queries may not match against most recent data.

Apr <var data-var='date'>17</var>, <var data-var='time'>17:28</var> UTC Resolved - This incident has been resolved.Apr <var data-var='date'>17</var>, <var data-var='time'>17:24</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Apr <var data-var='date'>17</var>, <var data-var='time'>17:12</var> UTC Update - We've developed a fix and will deploy it asap.Apr <var data-var='date'>17</var>, <var data-var='time'>16:42</var> UTC Identified - The issue has been identified and a fix is being implemented.Apr <var data-var='date'>17</var>, <var data-var='time'>16:31</var> UTC Investigating - We are investigating an issue preventing new events from being recorded.

Delay in processing events

Nov <var data-var='date'>19</var>, <var data-var='time'>07:59</var> UTC Resolved - All data has been indexed and is up-to-date.Nov <var data-var='date'>19</var>, <var data-var='time'>07:56</var> UTC Identified - Our data indexing pipeline is experiencing higher load than expected and there is delay in processing data. Listing and user search APIs may not return up-to-date matches.Nov <var data-var='date'>19</var>, <var data-var='time'>07:55</var> UTC Investigating - We are currently investigating this issue.

Aug <var data-var='date'>18</var>, <var data-var='time'>11:00</var> UTC Resolved - At 10:50:40 UTC, we saw elevated error rates when the system interacted with Stripe API. We assume this was caused by recent code changes, which are now rolled back. The rollback was completed at 12:24:25 UTC. We continue investigating the root cause.

Is Sharetribe down?

Sharetribe status is UP

Lastest published events

Elevated error rates in Marketplace API

Elevated error rates in Marketplace API

Elevated latency in Marketplace API

Sharetribe Services, Components and Regions

Recent Sharetribe events

Elevated error rates in Marketplace API

Elevated error rates in Marketplace API

Elevated latency in Marketplace API

Increased API latency

Database maintenance

Partial degradation in Sharetribe subscription management and customer support system

Get your FREE account now,
5 monitors included!

Is Sharetribe down?

Sharetribe status is UP

Lastest published events

Elevated error rates in Marketplace API

Elevated error rates in Marketplace API

Elevated latency in Marketplace API

Sharetribe Services, Components and Regions

Recent Sharetribe events

Elevated error rates in Marketplace API

Elevated error rates in Marketplace API

Elevated latency in Marketplace API

Increased API latency

Database maintenance

Partial degradation in Sharetribe subscription management and customer support system

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!