deco - Notice history

Publish has returned to normal operations. No data was lost during the incident. You are now clear to perform a publish to update the production environment.

Deno deployed implemented a fix and are currently monitoring the result.

Deno deploy was notified and is actively progressing towards a solution.

This incident has been resolved.

A deno deploy rollout increased the number of timeouts. They did a rollback and we are monitoring the number of issues on our sites.

This incident has been resolved.

Deno Deploy has implemented corrective measures as of 9:25 AM (GMT+3), and we are observing a notable reduction in the number of errors.

Deno Deploy rollouted a new infrastructure version at 7:15 am (GMT+3), which could be linked to the recent increase in errors. We have promptly initiated communication with their infrastructure team and are investigating any potential mitigating issues on our end.

No errors have been reported since the last update.

The incidence of errors has decreased, returning to standard levels. We are actively monitoring the situation.

This incident has been resolved.

Deno deploy implemented a solution, and we are monitoring the outcomes.

Notably, we have observed a substantial reduction in errors, plummeting from the range of 30-40% to 8-10%.

At 8 pm, a surge in traffic prompted a swift response from Deno deploy, that was able to scale their infrastructure.

By 9 pm, a redeployment led to the redirection of the entire traffic to new isolates; however, Deno encountered scaling challenges. We are actively liaising with Deno support to address this issue, and they have raised the overall cluster limit to effectively manage the increased data load.

This incident has been resolved.

We implemented a manual deployment on sites with this issue.

We identified an anomaly in the island builder process on Deno Deploy that requires our immediate attention.

We've identified a disruption in the island file generation process for websites updated between 4:14 PM and 5:26 PM. It's worth noting that sites with these files already cached on our Content Delivery Network (CDN) remained unaffected by this issue.