没有不值得去解决的问题,也没有不值得去学习的技术!

Troubleshooting a WordPress 502: How 18,455 Sogou UA Requests Maxed Out a 1-Core ECS

Figure 2: Alibaba Cloud ECS status and alarm page, CPU reaching instance specification limit with 30 cumulative alarms.

作者:

Recently, my WordPress blog suddenly started frequently experiencing 502 Bad Gateway.

At first, my immediate reaction was whether the server configuration was no longer sufficient.

After all, the specifications of this Alibaba Cloud ECS are not high:

  • 1 vCPU
  • 2 GiB RAM
  • 2 Mbps fixed public bandwidth
  • 20 GiB SSD system disk
  • Alibaba Cloud Linux 3
  • Nginx + PHP-FPM
  • Redis running locally on the ECS
  • MySQL using Alibaba Cloud RDS, not running on this ECS

The blog itself has been running for many years, and the number of articles keeps growing. Coupled with the recent ongoing work on historical article migration, excerpt generation, and multilingual translation, I initially thought upon seeing the 502 errors that it might finally be time to upgrade the ECS from 1 core 2GB to 2 cores 4GB.

But the subsequent troubleshooting results proved:

The core issue with this 502 was not that normal traffic had exceeded the capacity of 1 core 2GB, but rather that a batch of abnormally high-frequency requests carrying the Sogou web spider User-Agent pushed PHP-FPM and the CPU to their absolute limits.

Moreover, the entire incident—from severe 502s to rapid recovery after EdgeOne interception—is a very typical process.


1. Symptom: WordPress Admin Starts Intermittent 502s

The most obvious symptom was that accessing:

Plaintext
admin.shuijingwanwq.com/wp-admin/

started intermittently returning:

Plaintext
502 Bad Gateway
nginx
Figure 1: Nginx 502 Bad Gateway when accessing the WordPress admin.
Figure 1: Nginx 502 Bad Gateway when accessing the WordPress admin.

I tried restarting the ECS.

Right after the restart, the site would temporarily recover, but after a while, the 502s would start appearing again.

This meant the problem wasn’t simply an occasional crash of a single service:

Plaintext
Restart
→ Temporarily normal
→ Load rises again
→ 502 again

Meanwhile, the Alibaba Cloud ECS console had already recorded numerous CPU performance anomalies.

30 occurrences happened in a single day:

Plaintext
实例的 CPU 性能达到规格上限
Figure 2: Alibaba Cloud ECS status and alarm page, CPU reaching instance specification limit with 30 cumulative alarms.
Figure 2: Alibaba Cloud ECS status and alarm page, CPU reaching instance specification limit with 30 cumulative alarms.

At this point, it was confirmed:

The server was indeed under sustained resource pressure, rather than experiencing occasional browser or network errors.


2. Initially Suspected Clearing Server Cache Caused It

Right before the issue occurred, I had indeed performed a server-side cache cleanup.

However, two layers of cache must be clearly distinguished here.

My website architecture is roughly:

Plaintext
访客

EdgeOne / Cloudflare

Nginx

W3 Total Cache / Redis

PHP-FPM

WordPress

阿里云 RDS

What I cleared were the W3TC Page Cache, Object Cache, and other caches on the server.

I did not clear the CDN edge caches of EdgeOne and Cloudflare.

Therefore, it cannot be simply understood as:

Plaintext
清服务器缓存
→ CDN 全部 MISS
→ 所有用户突然回源
→ ECS 被打爆

This causal relationship does not hold.

Existing CDN HITs would still be returned directly by the edge nodes.

Clearing the server cache only increased the processing cost for requests that originally needed to hit the origin, such as:

  • Natural CDN MISSes
  • First origin fetch after CDN TTL expiration
  • WordPress admin
  • REST API
  • WP-Cron
  • Logged-in user requests
  • Dynamic requests

So while the cache cleanup might have been a contributing factor at a specific point in time, it could not directly explain a full day of severe 502s.

I had to keep looking for the real resource consumer.


3. Catching the Scene: Load Reached 9, CPU Had Zero Idle

About 17 minutes after restarting the server, I started checking the system status directly.

The results were extreme:

Plaintext
load average: 9.22, 9.55, 6.79

For a server with only a 1-core CPU, this load was already very high.

Continuing to look at vmstat:

Plaintext
us  sy  id  wa
91   9   0   0
92   8   0   0
92   8   0   0
92   8   0   0
92   8   0   0

CPU idle was consistently:

Plaintext
0%

And iowait = 0.

This meant:

The CPU was not waiting on disk, nor was it simply stuck on network I/O; it was genuinely executing user-space programs continuously.

Figure 3: Server load reaches 9.x, vmstat shows CPU idle consistently at 0%.
Figure 3: Server load reaches 9.x, vmstat shows CPU idle consistently at 0%.

4. PHP-FPM Was the Real CPU Hog

Next, I checked the processes consuming the most CPU.

The results were very uniform:

Plaintext
php-fpm  13.3%
php-fpm  13.3%
php-fpm  13.3%
php-fpm  13.3%
php-fpm  13.0%
php-fpm  12.9%
php-fpm  12.9%

A total of 7 PHP-FPM workers.

While Redis was only at about:

Plaintext
1.8%

And Nginx was only at:

Plaintext
0.9%

So the main source of the ECS CPU pressure this time was already clear:

PHP-FPM.

Meanwhile, the PHP-FPM configuration on the server was:

Plaintext
pm.max_children = 7
pm.max_requests = 500
request_terminate_timeout = 600s
request_slowlog_timeout = 3s

This meant all 7 workers were already working under high load.

More critically, the PHP-FPM socket:

Plaintext
LISTEN 510 511 /dev/shm/php-cgi.sock

The waiting connection queue had reached:

Plaintext
510 / 511

Only 1 slot away from being completely full.

This was enough to explain why Nginx was returning 502:

Plaintext
Nginx

PHP-FPM 7 个 Worker 全忙

新的 PHP 请求不断进入

FastCGI 队列持续积压

510 / 511

请求超时、失败

502 / 499

5. Not a Memory Issue, Nor RDS Consuming Local CPU

There was another easily misjudged aspect this time.

The server only had 2 GiB of memory, but at the scene, there was still:

Plaintext
available ≈ 1.2 GiB

Swap was barely used, and there were no signs of:

Plaintext
OOM
Out of memory
Killed process

So this was not a typical memory exhaustion.

Additionally, my MySQL uses Alibaba Cloud RDS and does not run locally on the ECS.

Therefore, the situation of:

Plaintext
mysqld 90% CPU

did not exist.

The processes truly worth attention on the local server were:

Plaintext
PHP-FPM
Redis
Nginx

And the evidence clearly pointed to PHP-FPM.


6. Requesting the Admin Directly from 127.0.0.1 Still Yielded Numerous 502s

To rule out factors like the public network, DNS, EdgeOne, and Cloudflare, I had the ECS locally access:

Plaintext
127.0.0.1

Nginx

PHP-FPM

WordPress

I tested it 20 times consecutively.

The results were dismal.

Some requests:

Plaintext
HTTP=502
total≈0.1 秒

And a large number of requests:

Plaintext
HTTP=000
15 秒超时

Throughout the 20 tests, there was no normal recovery.

This step essentially ruled out external network factors.

The problem was within:

Plaintext
ECS
Nginx
PHP-FPM
WordPress

this internal chain.


7. The Real Breakthrough: 18,455 Out of the Last 20,000 Requests Had the Sogou UA

After continuing to analyze the Nginx access logs, I finally found the real anomaly.

The status code statistics for the last 20,000 www requests were:

Plaintext
11925  499
7551   502
296    301
148    200
43     404
29     500

This meant:

Only 148 requests successfully returned 200.

A massive number of requests had turned into 499s and 502s.

But even more astonishing was the User-Agent:

Plaintext
18455 Sogou web spider/4.0

Out of the last 20,000 requests, 18,455 were:

Plaintext
Sogou web spider/4.0

Accounting for over:

Plaintext
92%

Other Googlebots, Bingbots, Baiduspiders, and real browser visits were barely a fraction compared to it.

Figure 4: Among the latest 20,000 access logs, Sogou web spider accounts for the vast majority of requests.
Figure 4: Among the latest 20,000 access logs, Sogou web spider accounts for the vast majority of requests.

8. 500–700 Requests Per Minute, Continuously Crawling Numerous Different Pages

Next, I tallied the request volume by the minute:

Plaintext
14:01  546
14:02  623
14:04  724
14:05  678
14:09  595
14:13  638
14:14  609
14:22  592
14:25  604
14:29  572

Sustained at:

Plaintext
500~700 次/分钟

Which is approximately:

Plaintext
8~12 次/秒

And it wasn’t frantically refreshing the same URL.

They were crawling:

Plaintext
文章页
分类页
分页
?p=xxxx
带 query 参数的页面

The request volume for a single URL was typically only in the single digits or a dozen times.

This meant it was more like:

High-frequency crawling of the entire website.

For a CDN, 10 RPS is not a massive attack.

But for:

Plaintext
1 核 CPU
+
WordPress 动态 PHP
+
每次执行可能达到 3 秒以上
+
pm.max_children = 7

it was entirely sufficient to crush the origin server.


9. Source IPs Were Also Highly Distributed

Looking at the source IPs:

Plaintext
117.40.82.101
117.40.82.170
125.94.249.28
1.71.146.14
222.79.117.159
222.79.116.117
1.71.147.232
1.71.147.97
125.94.249.39
122.246.2.213
……

Many individual IPs had close to 1,000 requests.

This also meant:

Simply rate-limiting or blocking based on a single IP would probably not be effective.

What was truly highly uniform was the:

Plaintext
User-Agent: Sogou web spider/4.0

10. Are These Actually Real Sogou Spiders?

I was quite puzzled.

A mature search engine spider, facing a server continuously returning a large number of 502s, should theoretically slow down its crawl rate rather than maintain high-frequency crawling.

So I grew suspicious about whether these requests were genuinely from Sogou.

I extracted the 10 most frequent source IPs for a PTR reverse lookup:

Plaintext
117.40.82.101
117.40.82.170
125.94.249.28
1.71.146.14
222.79.117.159
222.79.116.117
1.71.147.232
1.71.147.97
125.94.249.39
122.246.2.213

The result for all of them was:

Plaintext
PTR: 无

Therefore, these IPs could not be verified as genuine Sogou Spiders via DNS reverse resolution.

However, I won’t jump to the conclusion here that:

They are definitely fake Sogou spiders.

A more accurate description would be:

This is a batch of abnormally high-frequency traffic using the Sogou web spider/4.0 User-Agent, which cannot be verified as official Sogou spiders.

The User-Agent itself can easily be forged.

So right now, the most important thing is not to track down who they really are, but to protect the production server first.


11. Blocking *Sogou* Directly in EdgeOne

Since my main Chinese site already uses EdgeOne, the most effective place to stop the bleeding was not Nginx, but:

Intercepting directly at the CDN edge nodes.

In EdgeOne, I went to:

Plaintext
安全防护
→ Web 防护
→ 自定义规则
→ 基础访问管控

And created:

Plaintext
规则名称:
屏蔽 Sogou 爬虫

匹配字段:
User-Agent

匹配方式:
通配符匹配

匹配内容:
*Sogou*

执行处置:
拦截
Figure 5: EdgeOne creating a User-Agent *Sogou* interception rule.
Figure 5: EdgeOne creating a User-Agent *Sogou* interception rule.

After the rule was published, it showed as enabled in the site-level protection policy.

Figure 6: EdgeOne's "Block Sogou Spider" rule is officially enabled.
Figure 6: EdgeOne’s “Block Sogou Spider” rule is officially enabled.

There was no need to clear the EdgeOne CDN cache here.

Security rules and CDN content caches are two different things.


12. After Blocking, Server Load Rapidly Began to Drop

A few minutes after the rule was published, the server status showed a very noticeable change.

Before blocking:

Plaintext
load average:
9.47, 9.63, 8.14

First follow-up check:

Plaintext
3.11, 7.68, 8.58

A few minutes later:

Plaintext
2.69, 5.34, 7.31

Subsequently:

Plaintext
1.81, 4.53, 6.88

The 1-minute load had rapidly dropped from nearly 10 to under 2.

Because the 5-minute and 15-minute loads carry historical data from the period of high load, their decline is slower, which is normal.


13. Sogou Dropped from 92% Directly to 0

Among the most recent 200 requests right after the rule was published:

Plaintext
Sogou = 16

It had already dropped from the previous 92% to a very low proportion.

After waiting a few more minutes, I checked the latest 100:

Plaintext
Sogou = 0

200 = 71
301 = 22
304 = 1
403 = 3
404 = 3

Most importantly:

Plaintext
502 = 0
499 = 0

Normal 200 requests had once again become the vast majority.

This was almost a perfectly clean A/B validation:

Plaintext
屏蔽前:
大量 Sogou UA
→ PHP-FPM 满载
→ 502

屏蔽后:
Sogou = 0
→ PHP-FPM 恢复
→ 502 = 0

14. PHP-FPM Queue Recovered from 510/511 to 0/511

This was one of the most compelling metrics throughout the troubleshooting process.

At the peak of the incident:

Plaintext
PHP-FPM Socket:

510 / 511

After EdgeOne interception:

Plaintext
PHP-FPM Socket:

0 / 511

The backlog of requests was completely cleared.

Accessing the WordPress admin from inside the server, I tested 5 times consecutively:

Plaintext
HTTP=200  0.110 秒
HTTP=200  0.341 秒
HTTP=200  0.321 秒
HTTP=200  0.469 秒
HTTP=200  0.247 秒

All 5 were successful.

Looking at real-time CPU again:

Plaintext
CPU idle:

8%
90%
97%

It eventually reached:

Plaintext
97% 空闲

The run queue also recovered from over a dozen waiting tasks to:

Plaintext
r = 0

At this point, this production incident could basically be confirmed as resolved.

Figure 7: After blocking Sogou, PHP-FPM Socket queue recovered from 510/511 to 0/511, and local admin access returned HTTP 200 for 5 consecutive times.
Figure 7: After blocking Sogou, PHP-FPM Socket queue recovered from 510/511 to 0/511, and local admin access returned HTTP 200 for 5 consecutive times.

15. This Also Changed My Judgment on “Whether to Upgrade to 2C4G”

When I first saw:

Plaintext
CPU 100%
Load 9+
PHP-FPM 堆积
大量 502

I was indeed prone to concluding:

1 core 2GB is no longer enough.

If I hadn’t continued checking the logs, I might have just upgraded to 2 cores 4GB directly.

But the results have now proven:

Plaintext
异常流量停止
→ CPU idle 97%

This shows:

Under normal conditions, this 1-core 2GB server actually still has considerable CPU headroom.

Therefore, I do not plan to upgrade the server just because of this 502 incident for now.

2C4G would certainly provide better peak-handling capacity.

But if the real problem is:

Plaintext
异常爬虫
→ 每分钟数百次动态请求

then upgrading the server only means:

Plaintext
从更早被打满
变成晚一点被打满

The priority for solving the problem should be:

Plaintext
先控制异常流量

观察正常业务资源占用

再决定是否真的需要升级

rather than just throwing hardware at 502s.


16. Why Temporarily Block Sogou Directly Instead of Complex Rate Limiting

There is actually a trade-off here.

The current EdgeOne rule:

Plaintext
*Sogou* → Block

means:

Genuine Sogou spiders will also be blocked.

This is clearly not the most refined approach.

The ideal would be:

Plaintext
正常、低频 Sogou
→ 放行

异常高频 Sogou
→ 限速或者拦截

But for me, Sogou currently presents two practical issues:

First, my site’s index volume in Sogou is very small to begin with.

Second, Sogou brings almost no meaningful traffic to my blog.

Yet this anomalous traffic carrying the Sogou UA has actually caused:

Plaintext
CPU 100%
PHP-FPM 队列 510/511
后台 502

Therefore, the most reasonable choice at this stage is still:

Continue keeping the *Sogou* block.

After the site stabilizes for a while, if there is indeed a necessary need to retain Sogou SEO, I can then research EdgeOne’s rate-limiting features to upgrade the “one-size-fits-all block” to “block only above a threshold.”

But there is no need to rush it now.


17. EdgeOne’s Basic Bot Management Cannot Solve This Issue for Now Either

I later also checked the Bot management capabilities of the current Personal Edition of EdgeOne.

The main features currently visible include:

Plaintext
人机校验页
AI 爬虫处置
Figure 8: Basic Bot management features currently shown in EdgeOne Personal Edition.
Figure 8: Basic Bot management features currently shown in EdgeOne Personal Edition.

But I did not see a way to directly:

Plaintext
识别真实 Sogou
→ 放行

伪造 Sogou UA
→ 拦截

with this kind of fine-grained search engine bot policy.

Therefore, at this stage, using:

Plaintext
User-Agent = *Sogou*
→ Block

is the simplest and most reliable.


18. Lessons Learned from This Incident

The biggest takeaway this time was not “learning to block a Sogou User-Agent,” but rather reaffirming a sequence for troubleshooting server incidents.

1. 502 Does Not Mean the Server Configuration Is Definitely Insufficient

Seeing:

Plaintext
1C2G
CPU 100%
502

makes it easy to immediately think:

Upgrade the server.

But the real problem this time was anomalous traffic.

If the traffic source isn’t found, upgrading will only mask the problem.

2. Load, CPU, and PHP-FPM Queue Must Be Examined Together

The three metrics that truly confirmed the problem this time were:

Plaintext
Load ≈ 9
CPU idle = 0
PHP-FPM queue = 510 / 511

This is much more meaningful than looking at a single CPU percentage.

3. Local Requests Are Perfect for Ruling Out CDN and Public Network Issues

Using:

Plaintext
127.0.0.1

to request Nginx directly can quickly determine whether the problem lies in:

Plaintext
外部网络 / CDN

or:

Plaintext
ECS / Nginx / PHP-FPM / WordPress

This time, local requests also yielded numerous 502s, directly narrowing the troubleshooting scope to the origin server’s interior.

4. CDN and Server Cache Must Be Strictly Distinguished

Clearing W3TC / Redis:

Plaintext

Clearing EdgeOne / Cloudflare.

The two layers of cache should not be conflated.

5. User-Agent Cannot Prove Spider Identity

Seeing:

Plaintext
Sogou web spider/4.0

should not automatically lead to the assumption:

This is definitely the official Sogou Spider.

UAs can be forged at will.

This time, at least the multiple high-frequency IPs sampled could not be verified as Sogou via PTR reverse lookups.

So the most accurate term for now remains:

Anomalous high-frequency traffic carrying the Sogou UA.


19. Final Outcome

This incident ultimately did not involve:

  • Upgrading the ECS
  • Modifying pm.max_children
  • Restarting PHP-FPM
  • Clearing the CDN cache
  • Adjusting Redis
  • Adjusting the RDS
  • Adding extra rules to Nginx

The core fix actually executed was just one:

Plaintext
EdgeOne
→ User-Agent 通配符 *Sogou*
→ Block

Then:

Plaintext
Sogou UA
从 92%+

0

And the server recovered from:

Plaintext
Load ≈ 9
CPU idle = 0%
PHP-FPM queue = 510 / 511
大量 499 / 502

to:

Plaintext
Load 持续下降
CPU idle 最高 97%
PHP-FPM queue = 0 / 511
后台连续 5 次 HTTP 200
最近请求 502 = 0

The entire process also reiterates:

When a server suddenly has “insufficient performance,” do not upgrade the configuration as a first resort.

First answer one question:

Exactly what traffic, what process, and what requests consumed the resources?

Sometimes what is truly needed is not more CPU, but blocking requests that shouldn’t be reaching PHP in the first place, further away from the origin server.

This time, EdgeOne happened to take on this role.

需要长期技术维护或远程问题排查?

我是拥有 15+ 年经验的 PHP / Go 后端工程师,长期关注已有系统维护、Bug 修复、性能优化、服务器排查、WordPress 网站维护和小功能迭代。

如果你的项目遇到以下情况,可以先从一次小问题排查开始合作:

  • ✅ PHP / Laravel / Yii2 老项目无人维护
  • ✅ Go / Gin 后端接口需要排查或优化
  • ✅ WordPress 网站访问慢、报错或插件冲突
  • ✅ Nginx / MySQL / Redis / Linux 服务器异常
  • ✅ CDN / Cloudflare / DNS / HTTPS 配置问题
  • ✅ 需要长期远程技术支持或兼职维护

更多介绍请查看:关于我 & 合作

微信:13980074657
邮箱:shuijingwanwq@gmail.com
Telegram:@shuijingwan
GitHub:https://github.com/shuijingwan