deco - Overall sites degradation – Incident details

Third Party: Cloudflare → Cloudflare Sites and Services → Workers experiencing degraded performance

Overall sites degradation

ResolvedPartial outage1 hour 8 minutes

Affected

hosting-kubernetes

Partial outage from 12:01 AM to 1:09 AM

hosting-denodeploy

Partial outage from 12:01 AM to 1:09 AM

  • ResolvedUTC

    This incident has been resolved.

  • MonitoringUTC

    Deno deploy implemented a solution, and we are monitoring the outcomes.

    Notably, we have observed a substantial reduction in errors, plummeting from the range of 30-40% to 8-10%.

  • IdentifiedUTC

    At 8 pm, a surge in traffic prompted a swift response from Deno deploy, that was able to scale their infrastructure.

    By 9 pm, a redeployment led to the redirection of the entire traffic to new isolates; however, Deno encountered scaling challenges. We are actively liaising with Deno support to address this issue, and they have raised the overall cluster limit to effectively manage the increased data load.

  • InvestigatingUTC

    We are currently investigating this incident.