Increase in 502 errors

Incident Report for NamePros

Resolved

Disabling PCRE JIT resolved the 502 errors. We'll leave it disabled until we have a better solution.

Posted Feb 09, 2022 - 22:30 UTC

Update

The coredumps indicated an issue in libpcre2's JIT compiler. We've temporarily disabled the JIT compiler to ensure that's the issue.

Posted Feb 09, 2022 - 22:19 UTC

Update

Disabling Swoole didn't resolve the issue. We've obtained coredumps from the segfaults and are examining them.

Posted Feb 09, 2022 - 21:17 UTC

Update

Disabling the JIT compiler didn't resolve the issue. We've now disabled Swoole.

Posted Feb 09, 2022 - 21:09 UTC

Update

We're disable PHP 8.1's JIT compiler to rule out https://github.com/php/php-src/issues/7817 as a potential cause.

Posted Feb 09, 2022 - 20:26 UTC

Identified

The cause of the 502 errors appears to be frequent segfaults within PHP-FPM. We're continuing to investigate.

Posted Feb 09, 2022 - 20:15 UTC

Investigating

We've received reports of sporadic 502 Bad Gateway errors. The error will typically manifest as a blue page with the text, "CDN Error (5XX) Bad Gateway." Preliminary investigation indicates that this error is rare. It's safe to ignore this error if you receive it. We'll continue investigating.

Posted Feb 09, 2022 - 16:37 UTC

This incident affected: Forums, Blog and Server infrastructure (Backend servers).