Status: Resolved Summary

Approximately two weeks ago, users began experiencing disruption when accessing analytics data within App Studio. Initial investigation did not immediately identify the root cause; a deeper analysis ultimately traced the issue to a routine infrastructure consolidation that was capping database capacity below the threshold required to handle specific event analytics queries spanning long date ranges. Full service has since been restored following a targeted increase in database resources.

Timeline

• 2026-03-11 19:32 UTC — Incident created. Initial investigation launched following a rise in user-reported loading issues on analytics, despite no prior performance degradation. The initial investigation did not surface a clear root cause at this stage.
• 2026-03-18 10:45 UTC — Automated monitoring triggered alerts for elevated database error rates, corroborated by incoming user reports of analytics failures.
• 2026-03-19 09:30 UTC — Deeper analysis by the Engineering team identified the root cause: high latency and query failures isolated to specific event analytics queries over long date ranges, linked to the earlier infrastructure consolidation.
• 2026-03-19 09:40 UTC — Remediation plan executed: database resources scaled up to meet query load demands.
• 2026-03-19 10:05 UTC — Database performance stabilized; analytics functionality verified as fully restored.

Resolution

To restore service, database resources were immediately scaled up to accommodate the throughput required by analytics queries. Once additional capacity was provisioned, query timeouts ceased and application stability was reestablished. The system remained under active monitoring until no further degradation was observed, at which point the incident was marked resolved.

Root Cause Analysis

The incident was caused by a capacity mismatch introduced during a routine infrastructure consolidation. Such consolidations are a standard part of our infrastructure lifecycle and are conducted with active monitoring in place to ensure continued performance across all services. In this instance, however, the capacity allocated to the Analytics Database was capped at a level that, while adequate for standard workloads, could not sustain the computational demands of specific event analytics queries executed over long date ranges. The initial investigation did not immediately surface this connection; it was only through deeper analysis that the consolidation was identified as the underlying cause. When these queries were executed, they exceeded available processing capacity, resulting in database timeouts and repeated failures of the analytics service; conditions that persisted until capacity was restored.

The investigation also surfaced a broader pattern: a gradual uptick in user-reported loading issues over the preceding two weeks, despite analytics having previously performed reliably. This indicated the capacity cap had been quietly eroding performance well ahead of the full outage.

Conclusion

We are straightening our processes to prevent recurrence. Monitoring coverage applied during database consolidations will be strengthened to detect query latency degradation earlier, and capacity planning will be refined to better account for the demands of specific event analytics queries over long date ranges.Looking further ahead, the team is actively working on delivering a fully revamped analytics pipeline to production later this year. This is not an incremental improvement; the new pipeline is being built to deliver deeper, richer metrics that give event organizers a comprehensive view of their event performance.

App Studio Analytics Outage

Status: Resolved Summary

On March 11, 2026, we experienced a service disruption affecting the visibility of speaker and attendee profiles across multiple events. The incident was triggered by a timing issue during a scheduled infrastructure update — a data refresh was initiated before new field definitions had fully propagated to our Search Engine, causing the indexation system to be built from an outdated configuration.

Our Search Engine relies on an indexation system to serve profile data quickly across events. When the refresh completed against the stale configuration, new fields were rejected due to strict schema enforcement, leaving profiles absent from search results and listings.

The goal of this post-mortem is to share our assessment and the steps taken to resolve the issue, while providing full transparency to our customers.

Timeline

Time \(UTC\)

Event



Mar 11 – 11:04

Infrastructure update deployed to production. Changes confirmed as applied.



Mar 11 – 13:21

Data refresh initiated as part of the planned release process. Previous indexation remained fully functional during this period.



Mar 11 – 13:42

Release step complete. Data refresh running, expected duration ~3–4 hours.



Mar 11 – 14:49

Configuration mismatch errors first logged internally.



Mar 11 – 15:33

Data refresh reported as complete. Spot checks passed. Final release step initiated.



Mar 11 – 16:52

Reports of missing profiles across multiple events surfaced. Incident response team immediately activated.



Mar 11 – 17:29

Root cause identified in under few minutes: indexation system had been built before the new fields were fully available. Internal alerts corroborated the finding.



Mar 11 – 18:08

Analysis completed confirming the refresh did not produce an indexation with the expected new fields.



Mar 11 – 18:39

Corrective data refresh triggered with the correct configuration fully in place, within 30 minutes of initial reports.



Mar 11 – 18:43

New indexation confirmed to contain the correct fields. Full refresh in progress.



Resolution

Once the root cause was identified, the team immediately initiated a new data refresh against the correct, fully propagated configuration. Profile visibility and search behavior were monitored throughout the process. Once the refresh completed, all profiles returned to normal search and listing behavior.

Root Cause Analysis

The root cause was a race condition between the deployment of new Search Engine field definitions and the triggering of a data refresh. The refresh started before the new configuration had fully propagated, causing the indexation system to be built from a stale schema. Due to strict schema enforcement, subsequent write operations referencing the new fields were rejected, leaving profiles absent from search results and listings.

Several factors contributed to this incident:

• Ambiguous deployment signal: A pipeline warning created uncertainty around whether the deployment had completed cleanly.
• Insufficient spot checks: Post-refresh verification passed at a surface level, masking the underlying schema mismatch until live traffic exposed it.

Short-term improvements:

• Add a mandatory pre-refresh validation step to confirm the Search Engine schema matches the expected configuration before any refresh is initiated.
• Update the release process to include explicit schema validation checkpoints for sub field, as a required step.

Conclusion

This incident stemmed from a timing edge case in our release process. The improvements listed above reflect our commitment to preventing similar incidents and maintaining a reliable platform for all our customers.

For any questions or concerns regarding this incident, please reach out to our support team.

Speaker & Attendee Profile Visibility Issue

Status: Resolved We are providing a detailed post-mortem report regarding the service disruption that affected Swapcard customers on February 12th, 2026, from 09:00 UTC to 09:30 UTC. This issue was caused by an unprecedented traffic spike that exceeded the capacity of one of our internal services, leading to temporary degraded performance across badge printing \(SwapAccess\) and Studio access.

The goal of this post-mortem is to share insights from our assessment and the steps taken to resolve the issue while providing transparency to our customers.

Summary

On February 12th, 2026, Swapcard experienced a 30-minute service disruption affecting badge printing via the Check-in App \(SwapAccess\) and access to the Studio interface. Customers with live events during this window experienced printing delays, degraded Studio access, and intermittent error messages.

The incident was caused by an unusual and unprecedented volume of concurrent traffic hitting one of our internal services. This traffic pattern had not been observed before and exceeded the scaling thresholds configured at the time. The sudden load caused elevated response times that cascaded to downstream services, including badge generation and Studio access. The platform self-recovered as traffic levels normalized around 09:30 UTC.

Timeline

09:00 UTC | An unprecedented spike in concurrent traffic began hitting one of our core internal services, exceeding previously observed traffic patterns.

09:00–09:15 UTC | The service could not scale fast enough to absorb the sudden load. Elevated latency cascaded to dependent services, causing badge generation timeouts and degraded Studio access.

09:15–09:30 UTC | Traffic levels began to normalize. The platform progressively recovered as request queues cleared.

09:30 UTC | Full service restoration. Badge printing and Studio access returned to normal operation.

Onsite report | Our infrastructure team conducted a thorough investigation and immediately applied improvements to prevent recurrence.

Root Cause Analysis

The root cause of this incident was an unprecedented and sudden spike in concurrent traffic that exceeded the scaling capacity of one of our internal services. This traffic pattern had not been encountered before in production, and the service's auto-scaling configuration was not tuned to react quickly enough to absorb such a rapid increase.

As response times on this service climbed, the impact cascaded to dependent features — including badge generation and Studio, which rely on it for real-time operations. This cascading effect amplified the user-facing impact beyond what the initial traffic surge alone would have caused.

Remediation

Following this incident, our infrastructure team immediately took action to strengthen the resilience of the affected services:

Scaling improvements: We have significantly increased the resource capacity and improved the scaling configuration of the affected internal service to handle traffic spikes well beyond the levels observed during this incident. The service can now absorb sudden surges much more effectively.

Resource optimization: We have optimized the resource usage of the service to ensure it operates more efficiently under load, reducing the likelihood of capacity issues even during unexpected traffic peaks.

We have deployed additional monitoring and alerting specifically targeting the failure patterns observed during this incident. This ensures that if a similar traffic surge were to occur, our team would be automatically notified within seconds and could intervene proactively before any customer-facing impact.

Conclusion

This incident was caused by an unpredictable traffic pattern that had not been previously observed on our platform. While the disruption was brief, we understand the impact it had on customers running live events during that window, and we take that seriously.

The scaling, optimization, and alerting improvements we have put in place significantly reduce the risk of a similar incident occurring in the future. Our infrastructure team continues to monitor the situation closely.

If you have any questions or concerns regarding this incident, please don't hesitate to reach out to our support team.

SwapAccess issue in badge printing

Status: Resolved We are providing a detailed post-mortem report regarding a production change that caused continuous emails to be sent again \(duplicates\) to some attendees.

The impacted scope was limited to events created before February 4, 2025 that still had continuous emails enabled at the time of the incident.

The goal of this post-mortem is to share what happened, why it happened, how it was resolved, and what we are doing to prevent a recurrence.

Summary

A background task runs every 30 minutes to process continuous email campaigns:

• Retrieve all enabled continuous email campaigns
• For each campaign, verify whether each targeted attendee already received the corresponding email
• Send the email only to attendees who have not received it yet

The incident was caused by the "already received" check relying on a database that only started being populated after February 4, 2025. For events created before that date, the delivery history existed in a different \(legacy\) datastore. As a result, the system incorrectly concluded that certain attendees had not received the email yet, and re-sent it.

Timeline

• 2026-02-04 07:56 UTC | A change was released that caused continuous emails to be incorrectly sent again \(duplicates\) to some attendees.
• 2026-02-04 11:10 UTC | The change was reverted, stopping further duplicate sends.

Resolution

We resolved the incident by reverting the change introduced at 2026-02-04 07:56 UTC, restoring the previous behavior and preventing additional duplicate emails from being sent.

Root Cause Analysis

The root cause was an incorrect assumption in the delivery-history lookup logic:

• The deduplication check queried a datastore that only contained delivery records populated starting 2025-02-04.
• For events created before 2025-02-04, delivery records were stored in a legacy datastore.
• Because the check did not account for the legacy source, it returned false negatives \("not sent yet"\), resulting in duplicate emails being sent on subsequent background-task executions.

Immediate Actions Completed

• Reverted the problematic change to stop duplicate email sends.
• Identified and confirmed the gap between legacy and current delivery-history datastores for pre-2025-02-04 events.

Short-Term Improvements

• Update the "already received" check to consult the correct source for both legacy and current events.
• Add safeguards to prevent re-sending when delivery history is missing/ambiguous \(fail-safe behavior\).
• Add targeted tests covering pre-2025-02-04 events with continuous emails enabled.

Long-Term Improvements

• Consolidate delivery-history storage to a single source of truth \(including backfill/migration strategy for legacy data\).
• Introduce idempotency guarantees at send-time \(e.g., campaign\+attendee idempotency keys\) to prevent duplicates even if checks regress.
• Add monitoring and alerting on abnormal duplicate-send rates for continuous email campaigns.

Conclusion

We recognize the impact that duplicate emails can have on attendee experience and event organizer trust. We are implementing the preventative measures above to ensure legacy data paths are consistently handled and duplicates are prevented by design.

If you have any questions or concerns regarding this incident, please reach out to our support team.

Is Swapcard down?

Swapcard status is UP

Lastest published events

App Studio Analytics Outage

Speaker & Attendee Profile Visibility Issue

SwapAccess issue in badge printing

Swapcard Services, Components and Regions

Recent Swapcard events

App Studio Analytics Outage

Speaker & Attendee Profile Visibility Issue

SwapAccess issue in badge printing

Duplicate continuous emails sent to attendees

Developer API : 504 Gateway Time-out

Get your FREE account now,
5 monitors included!

Is Swapcard down?

Swapcard status is UP

Lastest published events

App Studio Analytics Outage

Speaker & Attendee Profile Visibility Issue

SwapAccess issue in badge printing

Swapcard Services, Components and Regions

Recent Swapcard events

App Studio Analytics Outage

Speaker & Attendee Profile Visibility Issue

SwapAccess issue in badge printing

Duplicate continuous emails sent to attendees

Developer API : 504 Gateway Time-out

Get your FREE account now,5 monitors included!

Get your FREE account now,
5 monitors included!