Increased US-EAST-6 error rate due to large-scale DDoS attack

Between 09:36 and 09:40 UTC, our US-EAST-6 cluster received a large-scale DDoS attack which briefly caused the cluster's load balancers to return 502 Bad Gateway responses to a subset of projects hosted in US-EAST-6. At 09:37 UTC, our automated Cloudflare-backed DDoS protections activated and began blocking malicious traffic. Meanwhile, the clusters load balancers scaled up and error rates dropped back to normal levels.

In response, we've permanently increased the baseline capacity of the clusters load balancer and are prioritising measures to better absorb the initial shock of similar DDoS attacks in the future.

Apologies for the interruption.