Jul <var data-var='date'> 2</var>, <var data-var='time'>01:13</var> UTC Resolved - This incident has been resolved. Jul <var data-var='date'> 1</var>, <var data-var='time'>22:50</var> UTC Monitoring - A fix has been implemented and we are monitoring results. Delayed data was for a subset of customers between 22:12 UTC - 22:44 UTC. No data loss. Jul <var data-var='date'> 1</var>, <var data-var='time'>22:35</var> UTC Investigating - We are investigating an issue with collecting Mux data beacons. Beginning 22:12 UTC, view session data will be delayed.

Mux Data beacon collection issues

Jun <var data-var='date'>26</var>, <var data-var='time'>18:48</var> UTC Resolved - No further issues observed during monitoring period. Live streams are operational. Jun <var data-var='date'>26</var>, <var data-var='time'>16:50</var> UTC Monitoring - Between 16:15 and 16:20 UTC a small number of live streams were disconnected. Active live streams were able to reconnect and resume, and there is no long term impact to DVR or VOD content from affected live streams other than resuming on a new Asset ID. All systems are currently operational and we are monitoring the live ingest systems.

Live Stream Disconnects

Jun <var data-var='date'>25</var>, <var data-var='time'>16:48</var> UTC Resolved - A small number of live streams were disconnected around 13:10 UTC, ending the corresponding Asset. Active live streams were able to reconnect and resume, and there is no long term impact to DVR or VOD content from affected live streams other than resuming on a new Asset ID. All systems are currently green. Jun <var data-var='date'>25</var>, <var data-var='time'>13:56</var> UTC Investigating - We are investigating an incident which caused the disconnection of multiple live streams at 13:11 UTC.

Live stream disconnections

Jun <var data-var='date'>18</var>, <var data-var='time'>13:18</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'>18</var>, <var data-var='time'>11:31</var> UTC Monitoring - Video delivery for Live and VOD in the impacted region has returned to normal operations.  The impacted time frame was 9:30 - 10:30 UTC.  We are continuing to monitor. Jun <var data-var='date'>18</var>, <var data-var='time'>10:57</var> UTC Identified - We've identified an issue with a CDN provider in the impacted regions. We are seeing recovery, and continuing to monitor. Jun <var data-var='date'>18</var>, <var data-var='time'>10:16</var> UTC Investigating - We are experiencing increased errors with delivery across Live and VOD in Europe regions  and are actively investigating.

Increased error in Video Live and VOD

Jun <var data-var='date'>10</var>, <var data-var='time'>18:58</var> UTC Resolved - This incident has been resolved. Jun <var data-var='date'>10</var>, <var data-var='time'>16:11</var> UTC Monitoring - A fix has been implemented and we are monitoring. Jun <var data-var='date'>10</var>, <var data-var='time'>15:10</var> UTC Identified - We have identified the issue and are implementing a fix. Jun <var data-var='date'>10</var>, <var data-var='time'>14:04</var> UTC Investigating - We are investigating an issue causing an increase in errors in the Mux Data dashboard and API.

Elevated errors in Mux Data dashboard and API

May <var data-var='date'>19</var>, <var data-var='time'>23:56</var> UTC Resolved - From 21:02 UTC to 22:55 UTC some live streams may have experienced elevated rebuffering. We have identified and mitigated the issue.

Elevated rebuffering in live streams

May <var data-var='date'>19</var>, <var data-var='time'>21:00</var> UTC Completed - The scheduled maintenance has been completed. May <var data-var='date'>19</var>, <var data-var='time'>20:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May <var data-var='date'>12</var>, <var data-var='time'>22:10</var> UTC Scheduled - In order to improve the reliability of our realtime systems, we will be undergoing scheduled maintenance. During this time, customers from EU may see duplicate messages via our view stream exports to their Kinesis or PubSub endpoints. This will not affect views or metrics once this window is complete.

Mux Data Infrastructure Maintenance

May <var data-var='date'>14</var>, <var data-var='time'>20:57</var> UTC Resolved - This incident has been resolved. May <var data-var='date'>14</var>, <var data-var='time'>20:33</var> UTC Monitoring - We have identified the issue and have deployed a fix. We are continuing to monitor. May <var data-var='date'>14</var>, <var data-var='time'>20:31</var> UTC Investigating - We are seeing high webhook delivery latency. We are actively investigating

Webhook Delivery Stalled

May <var data-var='date'> 8</var>, <var data-var='time'>03:44</var> UTC Resolved - From 03:08UTC to 03:14UTC we saw increased request error rates and webhook delivery latency increases. The issue is mitigated and systems are operating normally.

Elevated API errors and webhook delivery latency

May <var data-var='date'> 7</var>, <var data-var='time'>23:41</var> UTC Resolved - This incident has been resolved May <var data-var='date'> 7</var>, <var data-var='time'>23:18</var> UTC Monitoring - Our team has implemented a fix and we are monitoring the results. We’ll post an update as soon as the issue is fully resolved. May <var data-var='date'> 7</var>, <var data-var='time'>22:39</var> UTC Identified - Our team has identified the cause of elevated Data Historical metrics errors and is actively working on a fix. We’ll continue to share updates as we have them. May <var data-var='date'> 7</var>, <var data-var='time'>22:32</var> UTC Investigating - We are currently investigating errors in Mux Data historical metrics beginning at approximately 22:08 UTC. We apologize for the inconvenience and will share an update once we have more information.

Elevated errors in Mux Data Historical metrics

May <var data-var='date'> 6</var>, <var data-var='time'>18:24</var> UTC Resolved - This incident has been resolved. May <var data-var='date'> 6</var>, <var data-var='time'>17:42</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'> 6</var>, <var data-var='time'>16:41</var> UTC Identified - The issue has been identified and a fix is being implemented. May <var data-var='date'> 6</var>, <var data-var='time'>16:36</var> UTC Investigating - We are currently investigating an issue where playback may experience elevated rebuffering rates and playback failures.

Elevated rebuffering and playback failures

May <var data-var='date'> 5</var>, <var data-var='time'>20:54</var> UTC Resolved - This incident has been resolved. May <var data-var='date'> 5</var>, <var data-var='time'>19:50</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. May <var data-var='date'> 5</var>, <var data-var='time'>19:18</var> UTC Identified - The issue has been identified and a fix is being implemented. May <var data-var='date'> 5</var>, <var data-var='time'>18:52</var> UTC Investigating - We are currently investigating the issue

Elevated rebuffering and startup times for some on-demand assets and live streams

Apr <var data-var='date'>30</var>, <var data-var='time'>03:30</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>30</var>, <var data-var='time'>02:26</var> UTC Monitoring - Our pipeline latency is resolved and we are caught up as of 1:50 UTC Apr <var data-var='date'>30</var>, <var data-var='time'>01:03</var> UTC Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'>30</var>, <var data-var='time'>00:22</var> UTC Investigating - Mux Data Views for EU customers are delayed beginning 22:53 UTC. We are investigating

Mux Data Delayed Views for EU customers

Apr <var data-var='date'>28</var>, <var data-var='time'>17:33</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>28</var>, <var data-var='time'>16:22</var> UTC Monitoring - We are monitoring for additional latency. EU saw elevated latency 15:50 UTC to 16:04 UTC. Monitoring data was inaccurate during that window and views were delayed. No data was lost. Apr <var data-var='date'>28</var>, <var data-var='time'>16:05</var> UTC Investigating - We are seeing some latency in both Monitoring and Historical views starting 15:53 UTC for EU customers and are currently investigating

Mux Data Pipeline Latency for EU Customers

Apr <var data-var='date'>28</var>, <var data-var='time'>03:50</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>27</var>, <var data-var='time'>20:09</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>27</var>, <var data-var='time'>20:08</var> UTC Investigating - Small percentage of static rendition downloads. We are currently investigating this issue.

Issues downloading MP4 assets

Apr <var data-var='date'>22</var>, <var data-var='time'>19:46</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>22</var>, <var data-var='time'>18:52</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>22</var>, <var data-var='time'>18:10</var> UTC Investigating - We are investigating increased startup times, re-buffering, and playback errors from 429 errors on some live streams and on-demand assets.

Increased re-buffering and errors for some assets and live streams

Apr <var data-var='date'>22</var>, <var data-var='time'>00:06</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'>21</var>, <var data-var='time'>18:20</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'>21</var>, <var data-var='time'>18:14</var> UTC Identified - The issue has been identified and a fix is being implemented. Apr <var data-var='date'>21</var>, <var data-var='time'>17:52</var> UTC Update - We are investigating an issue causing delays in processing for some assets. Additionally, the generation of static renditions, masters, and captions may be delayed. Apr <var data-var='date'>21</var>, <var data-var='time'>17:36</var> UTC Investigating - We are investigating an issue causing delays in processing for some assets.

Ingest delays for some assets

Apr <var data-var='date'> 9</var>, <var data-var='time'>15:49</var> UTC Resolved - This incident has been resolved. Apr <var data-var='date'> 9</var>, <var data-var='time'>14:27</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Apr <var data-var='date'> 9</var>, <var data-var='time'>14:20</var> UTC Investigating - We are currently investigating an issue where dashboard.mux.com is unavailable and may be showing an exception when loaded. All Data and Video services aside from dashboard.mux.com are unaffected.

Mux Dashboard unavailable

Mar <var data-var='date'> 7</var>, <var data-var='time'>01:57</var> UTC Resolved - Engineers have resolved the issue with clips not having audio. Any new clips will function as expected. Impacted clips created during the incident window are not retroactively corrected, but can be re-created from the original asset. Mar <var data-var='date'> 6</var>, <var data-var='time'>22:57</var> UTC Identified - A recent encoding change caused a mismatch in initialization metadata for some clips that were created in the last 24 hours. This would affect clips of assets created before Thursday, March 6 — 15:11 UTC that utilized "basic" or "premium" video_quality. A fix is currently being worked on.

video clips audio loss

Mar <var data-var='date'> 3</var>, <var data-var='time'>08:35</var> UTC Resolved - This incident has been resolved. Mar <var data-var='date'> 3</var>, <var data-var='time'>07:55</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Latency has returned to normal. Mar <var data-var='date'> 3</var>, <var data-var='time'>07:42</var> UTC Identified - The issue has been identified and we are implementing a fix. Latency in the system is reduced, but still remains for some customers. Mar <var data-var='date'> 3</var>, <var data-var='time'>06:36</var> UTC Update - We are continuing to investigate and will post as soon as we know more. Mar <var data-var='date'> 3</var>, <var data-var='time'>05:56</var> UTC Update - We are continuing to investigate and are seeing significant latency in our collection pipeline. Mar <var data-var='date'> 3</var>, <var data-var='time'>05:29</var> UTC Investigating - We are currently investigating an issue with our Mux Data Monitoring collection pipeline. Data may be delayed or missing beginning 05:14 UTC

Mux Data Monitoring Issues

Feb <var data-var='date'>25</var>, <var data-var='time'>00:32</var> UTC Resolved - After monitoring the fix we have confirmed full resolution. Feb <var data-var='date'>24</var>, <var data-var='time'>21:08</var> UTC Monitoring - Our teams have implemented a fix to resolve the delays. At approximately 19:53 UTC, customers may have experienced delays in video ingest time and webhook delivery. We will continue to monitor the recovery and will post an update when fully resolved. Feb <var data-var='date'>24</var>, <var data-var='time'>20:37</var> UTC Identified - Our teams identified the cause of the delays and are actively working on recovery. Feb <var data-var='date'>24</var>, <var data-var='time'>20:06</var> UTC Investigating - We are currently investigating delays in video ingestion and webhook delivery. Our teams are investigating and we will share an update once we have more information.

Delayed video ingest and webhook delivery for some customers

Feb <var data-var='date'>23</var>, <var data-var='time'>15:57</var> UTC Resolved - This incident has been resolved. Feb <var data-var='date'>23</var>, <var data-var='time'>14:47</var> UTC Monitoring - A fix has been implemented and we are monitoring the results. Some customers may have experienced a low number of playback failures during the incident. Feb <var data-var='date'>23</var>, <var data-var='time'>14:28</var> UTC Investigating - We are investigating an issue causing uploads to direct upload endpoints to fail for some customers. URL based ingest is not affected.

Direct upload failures and playback failures for some customers

Feb <var data-var='date'> 2</var>, <var data-var='time'>02:00</var> UTC Resolved - This incident has been resolved. Feb <var data-var='date'> 1</var>, <var data-var='time'>16:59</var> UTC Monitoring - We have completed the remediation steps for impacted assets and will be monitoring. Jan <var data-var='date'>30</var>, <var data-var='time'>18:59</var> UTC Update - A fix has been implemented to address the issue for new assets, and our team is working through any remaining affected assets. We will post further updates as we have them. Jan <var data-var='date'>29</var>, <var data-var='time'>20:40</var> UTC Identified - We have identified the cause of this issue and our team is working on a fix. Jan <var data-var='date'>29</var>, <var data-var='time'>18:07</var> UTC Investigating - We are investigating an issue that causes some audio or video segments to be corrupt, causing playback quality degradation on affected assets.

Corruption on some segments of some videos

Feb <var data-var='date'> 1</var>, <var data-var='time'>23:20</var> UTC Resolved - This incident has been resolved. Feb <var data-var='date'> 1</var>, <var data-var='time'>22:25</var> UTC Monitoring - We've taken corrective action and are monitoring the results. Tail latencies have fully recovered. Feb <var data-var='date'> 1</var>, <var data-var='time'>22:17</var> UTC Update - Elevated latency in the processing of incoming monitoring data began approximately 21:35 UTC. This impacted most customers that have access to monitoring data. Latency has starting improving, and we continue to investigate root cause. Feb <var data-var='date'> 1</var>, <var data-var='time'>21:56</var> UTC Investigating - We are investigating an issue where monitoring data is significantly delayed

Monitoring dashboard data delayed

Jan <var data-var='date'>25</var>, <var data-var='time'>13:09</var> UTC Resolved - This incident has been resolved. Jan <var data-var='date'>25</var>, <var data-var='time'>12:08</var> UTC Monitoring - Our team has implemented a fix to resolve the issue and is monitoring the situation. We'll post an update as soon as the issue is fully resolved. Jan <var data-var='date'>25</var>, <var data-var='time'>10:42</var> UTC Investigating - We are currently investigating errors in Mux Data, that are impacting some customers' metric collection, beginning at approximately 0935 UTC. We apologize for the inconvenience and will share an update once we have more information.

Is mux down?

mux status is UP

Lastest published events

Mux Data beacon collection issues

Live Stream Disconnects

Live stream disconnections

mux Services, Components and Regions

Recent mux events

Mux Data beacon collection issues

Live Stream Disconnects

Live stream disconnections

Increased error in Video Live and VOD

Elevated errors in Mux Data dashboard and API

Elevated rebuffering in live streams

Get your FREE account now,
5 monitors included!

Is mux down?

mux status is UP

Lastest published events

Mux Data beacon collection issues

Live Stream Disconnects

Live stream disconnections

mux Services, Components and Regions

Recent mux events

Mux Data beacon collection issues

Live Stream Disconnects

Live stream disconnections

Increased error in Video Live and VOD

Elevated errors in Mux Data dashboard and API

Elevated rebuffering in live streams

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!