Recently, while continuing to add an Italian site to the A Tour of Go multilingual project, I ran into a fairly typical problem during the Production phase.
The site itself had been deployed successfully, the origin was healthy, and there were no issues with Cloudflare DNS. But once it came to public network acceptance, transient network failures caused the entire first Production process to abort.
At first, I thought simply adding more retries would fix it.
After investigating, the final adjustment turned out to be more fundamental:
I stopped routing full public network acceptance traffic through my local machine and the cross-border SOCKS link, and instead ran public network acceptance directly on zgocloud.
After the adjustment, all public network checks—including 105 sitemap URLs, CDN Cache, HTML Identity, and Socket Boundary—passed consistently.
1. The problem wasn’t a deployment failure, but a public network acceptance failure
The first Production of the A Tour of Go multilingual site isn’t just “service started, therefore success.”
The formal process also requires checking:
- release identity
- remote/source identity
- key public routes
- HTML identity
- sitemap and all URLs within it
- socket boundary
- CDN Cache
- browser automated acceptance
- HUMAN visual gate
When the problem occurred this time, the earlier release and origin checks had actually already passed.
For example:
[verify-production] remote identity: PASS
[verify-production] source routes: 7/7 PASSIn other words, there was already evidence proving:
The current release was correct, and the origin service was healthy.
But once public network requests began, the same Cloudflare URL repeatedly hit 15-second timeouts:
curl: (28) Operation timed out after 15002 milliseconds with 0 bytes received
curl: (28) Operation timed out after 15002 milliseconds with 0 bytes received
curl: (28) Operation timed out after 15002 milliseconds with 0 bytes receivedUltimately:
[首次生产] FAILED
stage: public-machine
expected: command exit 0
actual: exit 1
This log is important.
If you only saw:
first-production FAILEDIt would be easy to instinctively think you should redeploy, re-check Nginx, or even reconfigure Cloudflare DNS.
But the actual evidence doesn’t support those actions.
The failure occurred in:
stage: public-machineNot in the deploy, source health, or DNS stages.
This means the right direction was to continue checking the public network acceptance link, rather than disrupting an already working Production environment.
2. The old public network acceptance link was a bit convoluted
The Production origin for this project is located in mainland China, while the Cloudflare control plane and some public network access require cross-border network transit.
To access Cloudflare reliably, I had already been using zgocloud as an overseas egress, handling relevant traffic via SSH, SOCKS, and similar methods.
But the old public network acceptance link was essentially still similar to:
本地电脑
↓
跨境连接
↓
zgocloud
↓
Cloudflare
↓
Production OriginThere’s an obvious problem here.
What actually needs to be verified is:
zgocloud → Cloudflare → ProductionBut a large amount of public network acceptance traffic still had to go through:
本地电脑 → zgocloudThis extra cross-border leg.
For an occasional page visit, this difference might not be noticeable.
But the Production verifier doesn’t just request the homepage once.
The current A Tour of Go Production sitemap contains a total of:
105 URLsBeyond the sitemap, it also needs to check:
public routes
HTML identity
socket boundary
CDN cacheThis means public network acceptance generates a large volume of consecutive requests.
As long as any leg of the cross-border link has some probability of jitter, the chance of failure scales up significantly as the number of requests increases.
At the time, besides curl 28 timeouts, I also encountered:
curl: (97) Failed to receive SOCKS response, proxy closed connectionLooking at any single failure in isolation, it might just seem like a sporadic occurrence.
But for formal Production acceptance, it can’t simply be ignored.
3. You can’t retry indefinitely just to make Production pass
Seeing transient network errors, a very natural thought is to add retries.
For example:
失败
→ retry
→ 还失败
→ 再 retryBut formal acceptance can’t turn into “just try a few more times, eventually one will pass.”
Otherwise, the credibility of automated acceptance will actually drop.
So this time, I reinforced a few principles.
1. Transient network errors allow bounded retry
For situations clearly attributable to transient network issues, the formal verifier can perform a limited number of bounded retries.
For example, the current verify-production.sh will apply this to some:
curl exit 6
curl exit 7
curl exit 16
curl exit 28
curl exit 35
curl exit 97as well as some Cloudflare errors:
HTTP 522
HTTP 525It performs a limited number of retries.
Currently, in verify-production.sh, a single logical public HTTP request will attempt at most 3 times.
However, one important caveat here:
3 is not a universal number used for all checks across the entire project.
Different stages have their own bounded retry or readiness rules depending on what is being checked.
For example, certain Nginx readiness or Playground Origin endpoint checks use different attempt counts.
What truly needs to be standardized isn’t “how many times everything retries,” but rather:
Retries must have clear boundaries.
2. After retries are exhausted, it must fail closed
No matter how many attempts a specific stage allows, once the formal bounded retries are exhausted, it must:
FAILEDYou can’t keep assuming the network “should actually be fine.”
Nor can you skip failed checks just to let the first Production finish smoothly.
3. Stop immediately after a sitemap check failure
This is especially important for the current project.
The sitemap has:
105 URLsSuppose that by the 20th URL, all bounded retries have failed; the remaining URLs haven’t actually been verified yet.
Continuing to request the 21st, 22nd… 105th URL at that point doesn’t change the fact that earlier ones already failed.
Therefore, the current formal logic is:
If a sitemap URL still fails after bounded retries, immediately fail closed.
Only after truly and completely verifying all URLs is it allowed to output:
sitemap: 105/105 PASSThis is also something I increasingly value:
The goal of automated acceptance isn’t to increase the PASS rate, but to make the PASS itself trustworthy.
4. The real adjustment: running public network acceptance directly on zgocloud
After further analysis, I felt that rather than continuously strengthening the local SOCKS link, it was better to rethink a more fundamental question:
Where exactly should Production public network acceptance run?
Since formal public traffic ultimately needs to access Cloudflare from overseas, the more direct path is actually:
zgocloud
↓
Cloudflare
↓
Production OriginSo the Production verifier now operates as:
本地维护机
│
│ SSH
▼
zgocloud
│
│ 直接执行公网 HTTP 验收
▼
Cloudflare
▼
Production OriginThis means the logs now clearly show:
[verify-production] public network runner: zgocloud (direct)The key here isn’t “switching to a different proxy.”
It’s:
Running the actual public HTTP acceptance from a more appropriate network location.
At the same time, not all checks were moved to zgocloud.
The local machine still handles formal release identity, and the remote Production source identity is still checked via its original formal path.
In other words, what was adjusted was:
public HTTP acceptance 的执行位置rather than downgrading or bypassing:
release identity
source identity
Production authority5. Why this is simpler than continuing to optimize SOCKS
In theory, I could have continued optimizing the original link:
本地
→ SOCKS
→ zgocloud
→ CloudflareFor example, by continuing to implement:
- more connection retries
- more complex tunnel recovery
- automatic SOCKS re-establishment
- multi-egress switching
- dynamic route scoring
- automatic routing based on time of day
But for the current project, these solutions would become increasingly complex.
And the actual requirement was quite simple:
I need a relatively stable overseas network location to verify the real state Production presents to the public internet via Cloudflare.
Since zgocloud already exists, then:
直接在 zgocloud 执行公网 verifieris clearly more direct than:
先从本地跨境连接到 zgocloud
再让大量请求经过这条 tunnelThis is why I didn’t continue with complex network engineering this time.
6. Actual Production results after the change
After the adjustment, I continued using the same formal it-IT release.
I didn’t republish or redeploy.
Once I re-entered formal Production acceptance, the output became:
[verify-production] public network runner: zgocloud (direct)
[verify-production] public routes: 7/7 PASS
[verify-production] html identity: PASS
sitemap URLs: 105/105
host mismatch: 0
HTTP failure: 0
[verify-production] sitemap: 105/105 PASS
[verify-production] socket boundary: PASS
[verify-production] CDN /: HIT -> HIT -> HIT PASS
[verify-production] CDN /tour/welcome/1: HIT -> HIT -> HIT PASS
PRODUCTION MACHINE ACCEPTANCE: PASS
[首次生产] 公网验收：PASS（31.1s）
This result forms a very stark contrast with the earlier failure logs.
Old approach:
source routes: PASS
↓
公网请求 curl 28
↓
public-machine FAILEDNew approach:
zgocloud (direct)
↓
public routes: 7/7 PASS
↓
sitemap: 105/105 PASS
↓
socket boundary: PASS
↓
CDN: PASS
↓
PRODUCTION MACHINE ACCEPTANCE: PASS7. After a network failure, you shouldn’t just redeploy arbitrarily
This time there was also a very practical takeaway.
In the past, when encountering:
first-production FAILEDIt was easy to feel an urge to:
是不是重新 deploy 一次比较保险？But this actually depends on which stage the failure occurred in.
For example, this time:
source routes: PASS
remote identity: PASSThe Production service itself already had complete evidence proving it was healthy.
What actually failed was:
public-machineAnd the errors were:
curl 28
curl 97So the correct approach should be:
保留当前 release
保留 Production 现场
↓
解决或等待网络条件恢复
↓
正式 resumerather than:
重新 publish
重新 upload
重新 deploy
重新修改 DNSOtherwise, not only does it waste time, but it could also turn a simple network issue into a new deployment variable.
first-production now supports a formal resume and will re-check key identities.
So when encountering similar issues in the future, I will prioritize deciding based on:
stage
+
真实 evidenceto determine the path back, rather than starting over from scratch just because I saw FAILED.
8. Avoid peak cross-border network hours in the evening for first Production
The timing of this issue also led me to add a rule that’s more of an operational habit.
Unless there’s an urgent need:
Try to avoid peak cross-border network hours in the evening, Beijing time, for first Production, public network acceptance, and DNS/CDN activation.
I’m not trying to prove here that:
某个具体时间段一定会出现丢包nor is there a need to set up complex long-term network monitoring for it.
It’s just that from a practical maintenance cost perspective:
If:
上午执行and:
晚上执行make no business difference, then there’s no need to proactively choose a time window that is more likely to introduce cross-border network variables.
Especially since first Production itself already involves:
DNS
Cloudflare
origin
shared assets
public acceptance
browser acceptanceThere are already enough variables.
One less unknown is one less.
9. Direct reuse for adding new locales in the future
The A Tour of Go multilingual project now has several Production locales.
When adding new languages later, I don’t plan to redesign the network approach for each one.
The current formal path is reused directly:
Production deploy
↓
direct-origin acceptance
↓
Cloudflare DNS
↓
minimal public readiness
↓
zgocloud direct public machine acceptance
↓
browser acceptance
↓
HUMAN visual gate
↓
first-production finalize
↓
production_state=liveTransient network issues:
bounded retryRetries exhausted:
fail closedProduction itself hasn’t changed:
保留现场
稍后正式 resumerather than redeploying.
For subsequent locales, this part can already be directly reused as the Production baseline.
10. Summary
What I saw at the very beginning this time was actually just a few:
curl: (28)and later:
curl: (97)If I had only addressed the error codes themselves, it would have easily turned into:
再多重试几次
再把 timeout 调大一些But the problem actually worth solving was:
Why must public network acceptance go through this longer, more volatile network path?
Ultimately, I didn’t keep stacking complex proxies, automatic routing, or multi-egress monitoring. Instead, I chose a simpler approach:
需要验证 Cloudflare 公网行为
↓
就在 zgocloud 直接验证while continuing to enforce:
bounded retry
+
fail closed
+
完整 identity 验证This time also reaffirmed something for me:
Production acceptance itself is part of the Production architecture.
Where a site is deployed is one thing.
Where acceptance is executed from is another.
If the acceptance link itself is unstable, even if Production is completely healthy, you might constantly get false failures.
Rather than endlessly increasing retry counts, it’s better to first ensure:
Acceptance happens in a location that is reasonable, stable, and aligned with the actual verification target.
For the current project, zgocloud direct is already sufficient to solve this problem, and it’s simpler than continuing to maintain a complex cross-border SOCKS public network acceptance link.
需要长期技术维护或远程问题排查？
我是拥有 15+ 年经验的 PHP / Go 后端工程师，长期关注已有系统维护、Bug 修复、性能优化、服务器排查、WordPress 网站维护和小功能迭代。
如果你的项目遇到以下情况，可以先从一次小问题排查开始合作：
- ✅ PHP / Laravel / Yii2 老项目无人维护
- ✅ Go / Gin 后端接口需要排查或优化
- ✅ WordPress 网站访问慢、报错或插件冲突
- ✅ Nginx / MySQL / Redis / Linux 服务器异常
- ✅ CDN / Cloudflare / DNS / HTTPS 配置问题
- ✅ 需要长期远程技术支持或兼职维护
更多介绍请查看：关于我 & 合作
微信：13980074657
邮箱：shuijingwanwq@gmail.com
Telegram：@shuijingwan
GitHub：https://github.com/shuijingwan