THIS IS A SCHEDULED EVENT Apr <var data-var='date'>20</var>, <var data-var='time'>00:00</var> - <var data-var='time'>02:00</var> UTCMar <var data-var='date'>20</var>, <var data-var='time'>00:36</var> UTC Scheduled - We will be undergoing scheduled maintenance during this time.

balenaCloud infrastructure maintenance

Apr <var data-var='date'> 8</var>, <var data-var='time'>19:51</var> UTC Update - We are continuing to monitor for any further issues.Apr <var data-var='date'> 8</var>, <var data-var='time'>19:51</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Mar <var data-var='date'>31</var>, <var data-var='time'>12:55</var> UTC Identified - We're experiencing an elevated level of errors in our application builder infrastructure and are currently looking into the issue.

Elevated GIT/Application Builder Errors

Mar <var data-var='date'>25</var>, <var data-var='time'>14:07</var> UTC Resolved - We experienced degraded API performance due to an internal configuration change that unintentionally increased system load, resulting in slower response times and reduced request capacity. Our team identified the issue and rolled back the change to restore stability. A root cause fix has been implemented and deployed aswell. The system has now fully recovered, and services are operating normally.Mar <var data-var='date'>24</var>, <var data-var='time'>13:21</var> UTC Update - We are continuing to monitor for any further issues.Mar <var data-var='date'>24</var>, <var data-var='time'>12:58</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Mar <var data-var='date'>24</var>, <var data-var='time'>12:43</var> UTC Identified - We're experiencing an elevated level of API errors and are currently looking into the issue.

Elevated API Errors

Mar <var data-var='date'>25</var>, <var data-var='time'>13:18</var> UTC Resolved - This incident has been resolved.Mar <var data-var='date'>23</var>, <var data-var='time'>18:55</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Mar <var data-var='date'>23</var>, <var data-var='time'>16:57</var> UTC Investigating - We are seeing several builds intermitently failing with 404 errors - No such image during builds and are investigating.

Builder Degraded performance

Mar <var data-var='date'> 3</var>, <var data-var='time'>20:17</var> UTC Resolved - This incident has been resolved.Mar <var data-var='date'> 3</var>, <var data-var='time'>19:45</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Mar <var data-var='date'> 3</var>, <var data-var='time'>17:44</var> UTC Investigating - We're experiencing an issue where the Dashboard may redirect to an unexpected page on initial load, which can prevent access to certain account and billing pages.

Elevated Dashboard Errors

Feb <var data-var='date'>27</var>, <var data-var='time'>03:39</var> UTC Resolved - This incident has been resolved.Feb <var data-var='date'>27</var>, <var data-var='time'>03:02</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Feb <var data-var='date'>27</var>, <var data-var='time'>02:41</var> UTC Identified - The issue has been identified and a fix is being implemented.Feb <var data-var='date'>26</var>, <var data-var='time'>21:13</var> UTC Update - We are continuing to investigate this issue. It only affects the direct SSH to device connection for example established via the balena-cli with balena device ssh <uuid>. Web Terminal via the dashboard and standard tunnelling is not affected.Feb <var data-var='date'>26</var>, <var data-var='time'>19:20</var> UTC Investigating - We're experiencing an elevated level of device SSH errors and are currently looking into the issue.

Elevated Device SSH Errors

Feb <var data-var='date'>24</var>, <var data-var='time'>20:54</var> UTC Resolved - This incident has been resolved.Feb <var data-var='date'>24</var>, <var data-var='time'>18:41</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Feb <var data-var='date'>24</var>, <var data-var='time'>18:29</var> UTC Identified - The issue has been identified and a fix is being implemented.Feb <var data-var='date'>24</var>, <var data-var='time'>17:29</var> UTC Update - We are continuing to investigate this issue.Feb <var data-var='date'>24</var>, <var data-var='time'>17:27</var> UTC Investigating - We're experiencing an elevated level of device SSH errors and are currently looking into the issue.

Feb <var data-var='date'>11</var>, <var data-var='time'>00:02</var> UTC Resolved - This incident has been resolved.Feb <var data-var='date'>10</var>, <var data-var='time'>10:25</var> UTC Monitoring - We have identified the potential cause and have rolled back the changes.Feb <var data-var='date'>10</var>, <var data-var='time'>09:33</var> UTC Investigating - Some delta generation requests are encountering errors and failing. We are currently investigating this issue.

Elevated Delta Errors

Feb <var data-var='date'>10</var>, <var data-var='time'>07:56</var> UTC Resolved - This incident has been resolved.Feb <var data-var='date'>10</var>, <var data-var='time'>07:17</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Feb <var data-var='date'>10</var>, <var data-var='time'>06:39</var> UTC Identified - The issue has been identified and a fix is being implemented.Feb <var data-var='date'>10</var>, <var data-var='time'>04:05</var> UTC Investigating - We're experiencing an elevated level of errors in our Cloudlink infrastructure and are currently looking into the issue.

Elevated Cloudlink Errors

Feb <var data-var='date'> 6</var>, <var data-var='time'>13:53</var> UTC Resolved - This incident has been resolved.Feb <var data-var='date'> 5</var>, <var data-var='time'>17:56</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Feb <var data-var='date'> 4</var>, <var data-var='time'>20:43</var> UTC Investigating - We're experiencing an elevated level of errors in our application builder infrastructure and are currently looking into the issue.

Jan <var data-var='date'>28</var>, <var data-var='time'>22:06</var> UTC Resolved - This incident has been resolved.Jan <var data-var='date'>28</var>, <var data-var='time'>20:08</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Jan <var data-var='date'>28</var>, <var data-var='time'>19:45</var> UTC Identified - The issue has been identified and a fix is being implemented.Jan <var data-var='date'>28</var>, <var data-var='time'>18:54</var> UTC Investigating - We're experiencing an elevated level of API errors and are currently looking into the issue.

Jan <var data-var='date'>28</var>, <var data-var='time'>01:00</var> UTC Resolved - A configuration change causing multiple component restarts resulted in the API being unavailable for a few minutes.

Partial API Outage

Dec <var data-var='date'>24</var>, <var data-var='time'>09:38</var> UTC Resolved - Insufficient AWS compute capacity overloaded the remaining nodes. This high load caused readiness probes to fail, triggering API restarts that created a feedback loop of increasing pressure.Dec <var data-var='date'>23</var>, <var data-var='time'>19:03</var> UTC Monitoring - Scaling issues during service deployment caused by unavailability of nodes from underlying scaling provider.Dec <var data-var='date'>23</var>, <var data-var='time'>16:32</var> UTC Investigating - We are currently investigating an issue affecting the availability of balenaCloud services.

Degraded Performance

Dec <var data-var='date'> 5</var>, <var data-var='time'>09:37</var> UTC Resolved - This incident has been resolved.Dec <var data-var='date'> 5</var>, <var data-var='time'>09:16</var> UTC Monitoring - Our upstream provider has implemented some fixes. balenaCloud services are back online. We are still monitoring the situation.Dec <var data-var='date'> 5</var>, <var data-var='time'>09:05</var> UTC Identified - CloudFlare, our proxy provider, is having service issues. Connectivity to balenaCloud services are currently affected.

An upstream provider outage is affecting connectivity to balenaCloud services

Nov <var data-var='date'>24</var>, <var data-var='time'>03:00</var> UTC Completed - The scheduled maintenance has been completed.Nov <var data-var='date'>24</var>, <var data-var='time'>01:55</var> UTC Update - Due to some delays in infrastructure provisioning, we are extending maintenance for another hour.Nov <var data-var='date'>24</var>, <var data-var='time'>00:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Oct <var data-var='date'>17</var>, <var data-var='time'>00:37</var> UTC Scheduled - We will be undergoing scheduled maintenance during this time.

Nov <var data-var='date'>12</var>, <var data-var='time'>19:58</var> UTC Resolved - This incident has been resolved.Nov <var data-var='date'>12</var>, <var data-var='time'>17:14</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Nov <var data-var='date'>12</var>, <var data-var='time'>15:01</var> UTC Identified - We have observed packet loss in the connections to our builder workers. We are working with our upstream provider to fix the issue. We have also provisioned a few replacement workers in an alternative location while we mitigate the problem.Nov <var data-var='date'>12</var>, <var data-var='time'>09:13</var> UTC Investigating - We are currently investigating this issue.

Builder partially degraded service

Oct <var data-var='date'>20</var>, <var data-var='time'>23:03</var> UTC Resolved - This incident has been resolved.Oct <var data-var='date'>20</var>, <var data-var='time'>12:49</var> UTC Monitoring - The underlying issue of our upstream hosting provider has been mitigated. We are continuing to monitor the situation.Oct <var data-var='date'>20</var>, <var data-var='time'>08:45</var> UTC Identified - Our upstream hosting provider is currently having an outage. We have identified that downloads from the balena registry may be slow or disrupted. Builds may also fail if the images are being pulled from Dockerhub. We are monitoring the situation.

Oct <var data-var='date'> 1</var>, <var data-var='time'>13:30</var> UTC Resolved - An update to our kube-system infrastructure resulted in a disruptive pod rollout that left some devices disconnected from Cloudlink for up to 10 minutes. We apologize for the interruption and future updates to this component will be handled within planned maintenance windows.

Elevated Device URLs/VPN Errors

Sep <var data-var='date'>30</var>, <var data-var='time'>21:56</var> UTC Resolved - This incident has been resolved.Sep <var data-var='date'>30</var>, <var data-var='time'>21:34</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Sep <var data-var='date'>30</var>, <var data-var='time'>20:35</var> UTC Identified - The issue has been identified and a fix is being implemented.Sep <var data-var='date'>30</var>, <var data-var='time'>19:47</var> UTC Investigating - We're experiencing an elevated level of errors in our Device VPN Tunnel infrastructure and are currently looking into the issue.

Elevated Device VPN Tunnel Errors

Sep <var data-var='date'>25</var>, <var data-var='time'>08:57</var> UTC Resolved - Large files in build context caused increases of memory usage on builder-services. Higher memory allocations have been implemented resolving the buffer memory issues.Sep <var data-var='date'>23</var>, <var data-var='time'>10:27</var> UTC Update - We have detected that network latency between different regions is causing issues on build context processing for build contexts containing larger files. We will deploy new servers temporarily to mitigate the impact while we work on a permanent fix.Sep <var data-var='date'>16</var>, <var data-var='time'>17:26</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Sep <var data-var='date'>16</var>, <var data-var='time'>17:01</var> UTC Identified - The issue has been identified and a fix is being implemented.Sep <var data-var='date'>16</var>, <var data-var='time'>02:33</var> UTC Investigating - We are currently investigating this issue.

Sep <var data-var='date'>17</var>, <var data-var='time'>22:01</var> UTC Resolved - This incident has been resolved.Sep <var data-var='date'>17</var>, <var data-var='time'>21:53</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Sep <var data-var='date'>17</var>, <var data-var='time'>21:32</var> UTC Identified - The issue has been identified and a fix is being implemented.Sep <var data-var='date'>17</var>, <var data-var='time'>20:54</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Sep <var data-var='date'>17</var>, <var data-var='time'>20:53</var> UTC Identified - The issue has been identified and a fix is being implemented.Sep <var data-var='date'>17</var>, <var data-var='time'>20:10</var> UTC Investigating - We're experiencing an elevated level of errors in our Device URLs and VPN infrastructure and are currently looking into the issue.

Sep <var data-var='date'>14</var>, <var data-var='time'>20:00</var> UTC Completed - The scheduled maintenance has been completed.Sep <var data-var='date'>14</var>, <var data-var='time'>18:01</var> UTC Verifying - Verification is currently underway for the maintenance items.Sep <var data-var='date'>14</var>, <var data-var='time'>16:00</var> UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Aug <var data-var='date'>15</var>, <var data-var='time'>01:00</var> UTC Scheduled - We will be undergoing scheduled maintenance during this time.

Sep <var data-var='date'>10</var>, <var data-var='time'>22:08</var> UTC Resolved - This incident has been resolved.Sep <var data-var='date'>10</var>, <var data-var='time'>21:26</var> UTC Update - All tests are passing and devices are stable. We'll keep monitoring.Sep <var data-var='date'>10</var>, <var data-var='time'>21:25</var> UTC Update - We are continuing to monitor for any further issues.Sep <var data-var='date'>10</var>, <var data-var='time'>21:19</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Sep <var data-var='date'>10</var>, <var data-var='time'>19:29</var> UTC Identified - Most devices have reconnected, but we're still experiencing an elevated level of device SSH errors and are currently looking into the issue. Connection that goes through are slow. This is also affecting the public URL feature.Sep <var data-var='date'>10</var>, <var data-var='time'>18:55</var> UTC Update - Most devices have reconnected, but we're still experiencing an elevated level of device SSH errors and are currently looking into the issue. Connection that goes through are slow. This is also affecting the public URL feature.Sep <var data-var='date'>10</var>, <var data-var='time'>15:46</var> UTC Update - It appears that the cloudlink status of many devices didn't update when disconnecting. If your device appear connected to cloudlink for more than 1h, its status is wrong and it has not yet reconnected (it will happen soon) We're still monitoring the recovery.Sep <var data-var='date'>10</var>, <var data-var='time'>15:37</var> UTC Update - During the incident a lot of devices disconnected from cloudlink. They're slowly but steadily reconnecting. We are monitoring the recovery.Sep <var data-var='date'>10</var>, <var data-var='time'>14:34</var> UTC Update - We are continuing to monitor for any further issues.Sep <var data-var='date'>10</var>, <var data-var='time'>14:25</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Sep <var data-var='date'>10</var>, <var data-var='time'>13:47</var> UTC Investigating - We're experiencing an elevated level of API errors and are currently looking into the issue.

Aug <var data-var='date'>27</var>, <var data-var='time'>20:45</var> UTC Resolved - This incident has been resolved.Aug <var data-var='date'>27</var>, <var data-var='time'>17:55</var> UTC Update - We are continuing to monitor for any further issues.Aug <var data-var='date'>27</var>, <var data-var='time'>17:55</var> UTC Monitoring - A fix has been implemented and we are monitoring the results.Aug <var data-var='date'>27</var>, <var data-var='time'>17:15</var> UTC Identified - The issue has been identified and a fix is being implemented.Aug <var data-var='date'>27</var>, <var data-var='time'>16:48</var> UTC Investigating - We're experiencing an elevated level of errors on our Dashboard when adding ESR devices and are currently looking into the issue.

Aug <var data-var='date'>19</var>, <var data-var='time'>11:29</var> UTC Resolved - This incident has been resolved.Aug <var data-var='date'>19</var>, <var data-var='time'>11:20</var> UTC Investigating - We're experiencing an elevated level of API errors and are currently looking into the issue.

Is balena.io down?

balena.io status is UP

Lastest published events

Elevated GIT/Application Builder Errors

Elevated API Errors

Builder Degraded performance

balena.io Services, Components and Regions

Recent balena.io events

balenaCloud infrastructure maintenance

Elevated GIT/Application Builder Errors

Elevated API Errors

Builder Degraded performance

Elevated Dashboard Errors

Elevated Device SSH Errors

Get your FREE account now,
5 monitors included!

Is balena.io down?

balena.io status is UP

Lastest published events

Elevated GIT/Application Builder Errors

Elevated API Errors

Builder Degraded performance

balena.io Services, Components and Regions

Recent balena.io events

balenaCloud infrastructure maintenance

Elevated GIT/Application Builder Errors

Elevated API Errors

Builder Degraded performance

Elevated Dashboard Errors

Elevated Device SSH Errors

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!