May <var data-var='date'>22</var>, <var data-var='time'>17:55</var> UTC Investigating - We are investigating an issue affecting the ability to submit and interact with support tickets through the Simon Web App. Customers can continue to respond to existing open tickets via email while we work to resolve the issue. We will provide further updates as more information becomes available.

Intermittent Issues Affecting Support Ticket Submission

May <var data-var='date'> 8</var>, <var data-var='time'>20:52</var> UTC Resolved - This incident has been resolved. May <var data-var='date'> 8</var>, <var data-var='time'>19:30</var> UTC Update - AWS has resolved the upstream Redshift issue, and we are observing recovery across affected workloads. Queue depth is decreasing and queries are completing, though some processing may remain slower than normal as the backlog clears. A subset of customers may have seen delays in data refreshes, syncs, journeys, and parts of the Simon Web App during the impact window; some customers were not affected. We are continuing to monitor for full stability and will post another update once recovery is complete. AWS status: https://health.aws.com/health/status May <var data-var='date'> 8</var>, <var data-var='time'>13:00</var> UTC Monitoring - We are tracking a service event at AWS in us-east-1 that is impacting Amazon Redshift availability. Some customers may see delayed or failing data refreshes. AWS has resolved the incident and we are working towards recovery.

AWS Redshift Outage

Feb <var data-var='date'>26</var>, <var data-var='time'>21:09</var> UTC Resolved - This incident has been resolved. Feb <var data-var='date'>26</var>, <var data-var='time'>18:05</var> UTC Monitoring - Data pipelines experienced an unexpected delay due to an AWS access issue. This has since been resolved and we are actively ensuring pipelines finish as quickly as possible.

AWS Access Issue

Feb <var data-var='date'>19</var>, <var data-var='time'>22:04</var> UTC Resolved - This incident has been resolved. Feb <var data-var='date'>19</var>, <var data-var='time'>19:09</var> UTC Monitoring - A fix has been implemented and we are monitoring. Feb <var data-var='date'>19</var>, <var data-var='time'>18:27</var> UTC Update - We are continuing to investigate this issue. Feb <var data-var='date'>19</var>, <var data-var='time'>18:27</var> UTC Investigating - We're aware of sporadic login issues and we're investigating

Sporadic login issues

Dec <var data-var='date'>22</var>, <var data-var='time'>18:04</var> UTC Resolved - Excited to announce that the Meta incident has been resolved. As a next step you must reauthenticate your Meta account to Simon. Including a Loom video below to walk through the process to re-auth your account in Simon https://www.loom.com/share/7af86a95cdf14d2c8a48553d08918166 Once this is done, please open up a Support Ticket in your Simon account so that we can resume your Meta syncs. Dec <var data-var='date'>21</var>, <var data-var='time'>14:48</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'>17</var>, <var data-var='time'>17:41</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'>12</var>, <var data-var='time'>17:37</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'>11</var>, <var data-var='time'>17:22</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'>10</var>, <var data-var='time'>17:54</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'> 9</var>, <var data-var='time'>21:46</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'> 5</var>, <var data-var='time'>20:34</var> UTC Update - Despite continued escalations and attempted workarounds to restore the app's connection, we continue to be in the same state. We recommend you continue with the manual upload workflow. Dec <var data-var='date'> 1</var>, <var data-var='time'>21:43</var> UTC Update - Simon remains dependent on Facebook approvals to reestablish the connection. While a new authentication app has been provisioned and set to live status, we are currently blocked awaiting the necessary approvals. Despite ongoing escalation efforts, we have not yet achieved a resolution. Additionally, we have explored and exhausted multiple workarounds aimed at restoring the previous app to good standing. At this time, we continue to recommend proceeding with the manual upload process to Facebook. Nov <var data-var='date'>27</var>, <var data-var='time'>13:26</var> UTC Update - Simon is continuing to wait for approvals from Facebook in order to reset the connection to Facebook. Simon has provisioned a new authentication app and set it to live, but we are blocked awaiting approvals. Our attempts to escalate this have been unsuccessful, though we will continue to do so. We have also exhausted several other workarounds to try to put the previous app back into good standing. We still advise the same manual upload process to Facebook. We will continue to closely monitor this throughout the Holiday weekend. Nov <var data-var='date'>26</var>, <var data-var='time'>01:15</var> UTC Update - Simon has provisioned a new authentication app and set it to live, but we are blocked awaiting approvals. Our attempts to escalate this have been unsuccessful. We have also exhausted several other workarounds to try to put the previous app back into good standing. We will continue to closely monitor this and attempt to expedite, but we cannot guarantee any remediation timeline. We advise you to continue to utilize manual segment exports used until resolved. Nov <var data-var='date'>25</var>, <var data-var='time'>03:34</var> UTC Update - Simon has lost access to the Facebook Marketing API, despite escalation from Simon leadership. The impact is that flows/journeys with Facebook actions are in an error state and cannot proceed until the connection has been restored. See the previous message for the workaround for time sensitive custom audiences. Otherwise, Simon will continue to push to restore access to the marketing API and resume normal syncing operations. Nov <var data-var='date'>24</var>, <var data-var='time'>20:28</var> UTC Investigating - Simon submitted Facebook (Meta) required integration updates weeks ago and is continuing to wait for confirmation that our app is completely updated. Due to this pending confirmation, Simon is at risk of losing and/or having downgraded performance with Facebook. Facebook has not given a specific date on when this would happen. While we continue to escalate this with Facebook, it is important to prepare should there be connection issues this week or through the weekend. To prepare for this impact, Simon is suggesting the following: 1. Create a segment export in Simon, adding the correct custom context fields needed (example: fn, ln, etc.) 2. Upload the list manually to Facebook - https://www.facebook.com/business/help/170456843145568?id=2469097953376494 https://www.facebook.com/business/help/2082575038703844?id=2469097953376494 3. For existing peak campaigns that include branches and/or variants to Facebook, we ask that you isolate all Facebook-related campaigns to their own individual flow/journey. This will help ensure Facebook integration issues do not impact any Email, SMS, Push, or other paid media syncs. To help you monitor our Facebook connection, we'll be providing updates through our status page to effectively communicate if and when Facebook shuts off our integration, at the moment it will show Facebook as operational. Simon leadership will continue to keep our team updated and will continue to escalate with Meta until this is resolved.

Monitoring Meta

Dec <var data-var='date'>16</var>, <var data-var='time'>14:23</var> UTC Resolved - At this time we are starting to see Snowflake processes recover. Dec <var data-var='date'>16</var>, <var data-var='time'>14:11</var> UTC Update - Snowflake continues to make progress here, there are some aspects of data sharing that are now operational. Dec <var data-var='date'>16</var>, <var data-var='time'>12:45</var> UTC Monitoring - Customers utilizing Snowflake services in the following regions are experiencing service disruption: Regions Impacted: AWS - US West (Oregon) AWS - Europe (Ireland)	 AWS - Asia Pacific (Mumbai) Users are unable to execute queries against Snowflake databases, tables, and views. Error message presented: "SQL execution internal error" or similar messages. Snowflake has identified the root cause and is actively developing a fix -- https://status.snowflake.com/

Snowflake Data Cloud - Degraded Service

Dec <var data-var='date'> 2</var>, <var data-var='time'>22:50</var> UTC Resolved - This incident has been resolved. Dec <var data-var='date'> 2</var>, <var data-var='time'>22:49</var> UTC Monitoring - Journeys service has been restored and is now operational. We are actively monitoring the service to ensure full recovery. Dec <var data-var='date'> 2</var>, <var data-var='time'>22:39</var> UTC Investigating - We are investigating an issue with Journeys that may cause service degradation for some users.

Journey Service Degradation

Nov <var data-var='date'>14</var>, <var data-var='time'>21:46</var> UTC Resolved - This incident has been resolved. Nov <var data-var='date'>13</var>, <var data-var='time'>03:06</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Nov <var data-var='date'>12</var>, <var data-var='time'>23:25</var> UTC Investigating - We're experiencing an degraded performance impacting Audience API and are currently looking into the issue.

Audience API performance degraded

Nov <var data-var='date'> 4</var>, <var data-var='time'>03:00</var> UTC Completed - The scheduled maintenance has been completed. Nov <var data-var='date'> 4</var>, <var data-var='time'>02:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Oct <var data-var='date'>13</var>, <var data-var='time'>17:31</var> UTC Update - We will be undergoing scheduled maintenance during this time. Oct <var data-var='date'>20</var>, <var data-var='time'>21:13</var> UTC Scheduled - Simon's Syncs infrastructure will be undergoing routine maintenance. During this time, campaigns may be delayed.

Simon's Syncs Maintenance

Oct <var data-var='date'>27</var>, <var data-var='time'>22:00</var> UTC Completed - The scheduled maintenance has been completed. Oct <var data-var='date'>27</var>, <var data-var='time'>21:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Oct <var data-var='date'>23</var>, <var data-var='time'>17:46</var> UTC Update - We will be undergoing scheduled maintenance during this time. Oct <var data-var='date'>27</var>, <var data-var='time'>22:10</var> UTC Scheduled - Simon AI's Journeys infrastructure will be undergoing routine maintenance. During this time, journey processing may be delayed.

Jenkins Journeys Maintenance

Oct <var data-var='date'>21</var>, <var data-var='time'>19:48</var> UTC Resolved - After our continued monitoring, Simon is considering this issue resolved. Oct <var data-var='date'>21</var>, <var data-var='time'>05:26</var> UTC Monitoring - All services are up and back to operational. AWS has resolved the last of the Redshift issues that have been impacting us. Our batch processing is continuing as scheduling and the team is monitoring progress. Oct <var data-var='date'>21</var>, <var data-var='time'>02:36</var> UTC Update - Simon continues to monitor the AWS recovery progress. At this time, the AWS Redshift cluster recovery is still outstanding with AWS engineers. Services continue to remain impacted. Oct <var data-var='date'>20</var>, <var data-var='time'>21:07</var> UTC Update - Some AWS services are beginning to recover from today's incident, but AWS Redshift continues to experience issues, which impacts Simon AI's segmentation and customer filtering features. We expect these issues to continue for several hours. Oct <var data-var='date'>20</var>, <var data-var='time'>13:32</var> UTC Identified - After additional investigation and monitoring from the previous incident, the following services are listed as degraded until the AWS incident is fully resolved.

Third Party Outage: AWS - Ongoing

Oct <var data-var='date'>20</var>, <var data-var='time'>10:26</var> UTC Resolved - We've successfully recovered from the AWS incident. Oct <var data-var='date'>20</var>, <var data-var='time'>10:00</var> UTC Monitoring - AWS had an outage that impacted a number of services that we rely on, including our ability to update our Status Page in a timely manner. The only impacted region was us-east-1. Journeys, Syncs, and Pipes have delayed processing that we are actively recovering from. Incoming events were delayed. More information on the outage. https://health.aws.amazon.com/health/status

Third Party Outage: AWS

Oct <var data-var='date'>16</var>, <var data-var='time'>08:30</var> UTC Resolved - This morning (Time frame: 3:55 am-4:26 am ET), Simon AI experienced an internal Elasticsearch issue that caused Audience API requests to fail. The issue has been fixed and we are continuing to monitor the health of the Audience API.

Audience API issue

Oct <var data-var='date'>13</var>, <var data-var='time'>22:00</var> UTC Completed - The scheduled maintenance has been completed. Oct <var data-var='date'>13</var>, <var data-var='time'>21:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Oct <var data-var='date'>13</var>, <var data-var='time'>18:02</var> UTC Update - We will be undergoing scheduled maintenance during this time. Oct <var data-var='date'>13</var>, <var data-var='time'>17:44</var> UTC Scheduled - Simon AI's Journeys infrastructure will be undergoing routine maintenance. During this time, journey processing may be delayed.

Jenkins Journey Maintenance

Oct <var data-var='date'> 6</var>, <var data-var='time'>02:00</var> UTC Completed - The scheduled maintenance has been completed. Oct <var data-var='date'> 6</var>, <var data-var='time'>01:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Oct <var data-var='date'> 2</var>, <var data-var='time'>20:42</var> UTC Scheduled - Simon AI will be undergoing routine database maintenance. During this time, the application and other scheduled processes may be delayed.

Scheduled Maintenance Window

Oct <var data-var='date'> 2</var>, <var data-var='time'>10:00</var> UTC Completed - The scheduled maintenance has been completed. Oct <var data-var='date'> 2</var>, <var data-var='time'>09:30</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>25</var>, <var data-var='time'>14:36</var> UTC Scheduled - Simon AI will be undergoing routine database maintenance. During this time, normal processes may be delayed.

Sep <var data-var='date'>22</var>, <var data-var='time'>22:00</var> UTC Completed - The scheduled maintenance has been completed. Sep <var data-var='date'>22</var>, <var data-var='time'>21:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Sep <var data-var='date'>15</var>, <var data-var='time'>21:13</var> UTC Update - We will be undergoing scheduled maintenance during this time. Sep <var data-var='date'>15</var>, <var data-var='time'>21:09</var> UTC Scheduled - Simon's Data Pipes infrastructure will be undergoing routine maintenance. During this time, data pipelines may be delayed.

Simon's Pipes Maintenance

Sep <var data-var='date'> 3</var>, <var data-var='time'>22:00</var> UTC Completed - The scheduled maintenance has been completed. Sep <var data-var='date'> 3</var>, <var data-var='time'>21:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>26</var>, <var data-var='time'>23:39</var> UTC Scheduled - Simon Journeys infrastructure will be undergoing routine maintenance. During this time, Journey processing may be delayed.

Scheduled Journey Maintenance

Aug <var data-var='date'>13</var>, <var data-var='time'>04:00</var> UTC Completed - The scheduled maintenance has been completed. Aug <var data-var='date'>13</var>, <var data-var='time'>03:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Aug <var data-var='date'>12</var>, <var data-var='time'>15:46</var> UTC Scheduled - Simon Data's Elasticsearch database will be undergoing routine maintenance and may experience minor disruptions during this time

Simon Data's Elasticsearch Routine Maintenance

Jul <var data-var='date'>29</var>, <var data-var='time'>05:00</var> UTC Completed - The scheduled maintenance has been completed. Jul <var data-var='date'>29</var>, <var data-var='time'>04:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jul <var data-var='date'>29</var>, <var data-var='time'>03:31</var> UTC Scheduled - The Audience API will be undergoing some routine maintenance and may experience minor disruptions at this time.

Audience API routine maintenance

Jul <var data-var='date'> 1</var>, <var data-var='time'>22:00</var> UTC Completed - The scheduled maintenance has been completed. Jul <var data-var='date'> 1</var>, <var data-var='time'>21:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Jun <var data-var='date'>30</var>, <var data-var='time'>16:16</var> UTC Scheduled - The Simon Data Engineering team will be conducting some database maintenance that requires a reboot. During this time you may experience degraded service in the platform.

Planned Database Maintenance

Mar <var data-var='date'>31</var>, <var data-var='time'>01:06</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'>31</var>, <var data-var='time'>00:26</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Mar <var data-var='date'>30</var>, <var data-var='time'>23:59</var> UTC Identified - We're experiencing an outage. Flows and journeys will be delayed as we repair the impacted system.

System outage

Mar <var data-var='date'>19</var>, <var data-var='time'>21:14</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'>19</var>, <var data-var='time'>18:22</var> UTC Identified - Simon is experiencing impacts to all data pipe refreshes, syncs, and journeys due to the Snowflake partial outage described here, https://status.snowflake.com/.

Snowflake Partial Outage

Mar <var data-var='date'>19</var>, <var data-var='time'>01:00</var> UTC Completed - The scheduled maintenance has been completed. Mar <var data-var='date'>19</var>, <var data-var='time'>00:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Mar <var data-var='date'>10</var>, <var data-var='time'>21:03</var> UTC Scheduled - Simon Data will be doing routine database maintenance on March 18th from 8:00PM - 9:00PM EST. This maintenance will only be for Journey syncs. You could expect up to 20 minutes of downtime during this period. During this planned maintenance there should be no impact to any campaigns outside of a brief delay in send due to the above mentioned downtime.

Mar <var data-var='date'> 5</var>, <var data-var='time'>22:14</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'> 5</var>, <var data-var='time'>21:44</var> UTC Identified - The issue has been identified and a fix is being implemented. Mar <var data-var='date'> 5</var>, <var data-var='time'>21:17</var> UTC Investigating - We are currently experiencing a service disruption with the Simon Web App and Audience API. We are investigating the degraded performance.

Is Simon Data down?

Simon Data status is UP

Lastest published events

Intermittent Issues Affecting Support Ticket Submission

AWS Redshift Outage

AWS Access Issue

Simon Data Services, Components and Regions

Recent Simon Data events

Intermittent Issues Affecting Support Ticket Submission

AWS Redshift Outage

AWS Access Issue

Sporadic login issues

Monitoring Meta

Snowflake Data Cloud - Degraded Service

Get your FREE account now,
5 monitors included!

Is Simon Data down?

Simon Data status is UP

Lastest published events

Intermittent Issues Affecting Support Ticket Submission

AWS Redshift Outage

AWS Access Issue

Simon Data Services, Components and Regions

Recent Simon Data events

Intermittent Issues Affecting Support Ticket Submission

AWS Redshift Outage

AWS Access Issue

Sporadic login issues

Monitoring Meta

Snowflake Data Cloud - Degraded Service

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!