In previous articles, I have written about the Unified Glossary, Semantic Compatibility in Go Learning & Documentation v2, and why a change in the glossary should not force every TranslationUnit in a locale to be retranslated and re-reviewed.
To achieve this, the system needs to do more than before.
Previously, many checks might only need to compare:
old SHA == new SHA ?Now it needs to know further:
What exactly changed in the Glossary?
↓
Which content is related to these changes?
↓
Do the existing translations already satisfy the new rules?
↓
Is it Compatible, or Affected?From the perspective of language quality and reducing redundant work, this direction is correct.
But it quickly exposed another problem:
The CLI started getting slower and slower.
And not just a little slower.
Some read-only checks that should have finished in seconds started taking minutes.
One typical measured run was:
6m54.554sAfter optimization:
10.335sThat is roughly 40 times faster.
This article will not discuss “why Semantic Compatibility is needed” again, nor will it revisit the architectural choices made previously.
This time, I am only documenting one thing:
Why a feature with a sound semantic design dragged the CLI down to the minute level, and how exactly it was later optimized back to seconds.
The performance issue was not in a particularly complex algorithm
When you first see a command run for six or seven minutes, it is easy to suspect:
Is Semantic Compatibility itself just too complex?
Are too many Pages being scanned?
Is the Markdown parser too slow?
Is the SHA calculation too heavy?
Is there an obvious O(n²) issue in some part of the Go implementation?
But after pinpointing the issue, I found that the problem was not that mysterious.
It was not that a single computation was particularly slow.
Rather:
The same expensive state was being recomputed many times within a single CLI invocation.
These two situations are very different.
If a single Compatibility preparation inherently takes 5 minutes, the optimization space might be quite limited.
But if a single preparation takes only a few seconds, yet it is redone over 100 times in a loop, then the problem is actually very clear:
It should not be repeated.
A Page loop secretly turned into over 100 full rebuilds
Take the Course SEO stale-subset preflight at the time as an example.
This command needed to check 103 Pages.
From a business logic perspective, it is natural to write something like:
for each Page:
validate current state
determine whether Page is staleThe problem was that the Page-level check would then enter Glossary Compatibility.
And Compatibility preparation was not a simple string lookup.
It needed to build a fairly complete current state, such as:
- glossary delta;
- context inventory;
- archived evidence;
- compatibility edges;
- current candidate;
- QC lineage;
- related identity.
As a result, the actual execution process gradually became:
Page 1
↓
PrepareGlossaryCompatibility()
Page 2
↓
PrepareGlossaryCompatibility()
Page 3
↓
PrepareGlossaryCompatibility()
...
Page 103
↓
PrepareGlossaryCompatibility()Finally, another path prepared it once more.
Consequently, a state that should have been shared was rebuilt about 104 times.
The situation was very similar in the Quality Check scope path.
When calculating scopes for 122 TranslationUnits one by one, it repeatedly rebuilt the same compatibility world containing about 470 contexts.
So on the surface it was just:
122 TUsBut in reality it was close to:
122 × full compatibility preparationThis is why CPU time scaled up so quickly.
A single quality-check scope run already reached about:
wall ≈ 373s
user CPU ≈ 400sThis was no longer just “the CLI is a bit slow.”
It directly affected whether the entire workflow could be used normally.
The first key decision: do not cut Semantic Compatibility
When encountering this kind of performance issue, a very easy solution to think of is:
Semantic Compatibility is too expensive, so fall back to a simple SHA gate.
Performance would certainly recover immediately.
But this would bring back the problems that had already been solved.
Modifying even a very small term in the Glossary could cause all existing results for an entire locale to be marked stale.
In other words:
Trading language quality and precise invalidation for CLI speed.
I did not want to solve it this way.
Because what the performance issue actually indicated was not:
“This Semantic Compatibility design is wrong.”
Rather:
Its execution method did not yet match its computational cost.
So the optimization goal was clear from the start:
Keep the original fail-closed semantics and compatibility results unchanged, and only reduce redundant computation.
What was really needed was invocation-local prepared state
The core idea ultimately adopted was actually very simple:
Within a single CLI invocation, the same deterministic expensive state is built only once.
The original execution method can be simplified as:
Page 1 → Prepare
Page 2 → Prepare
Page 3 → Prepare
Page 4 → Prepare
...Changed to:
Prepare once
↓
Prepared Resolver
↓
Page 1
Page 2
Page 3
Page 4
...That is, during a single command run, a shared prepared state / prepared resolver is obtained first.
Subsequent Pages, TranslationUnits, or other workflow consumers all read results from this already-built state.
It no longer re-derives everything from scratch from disk, archive, glossary, candidate, and lineage each time.
This change sounds a lot like ordinary caching.
But I deliberately did not design it as a persistent cache at the time.
Why not just use disk caching
Since the computation is expensive, a natural idea is:
Calculate the first time
↓
Save to disk
↓
Reuse directly next timeIn theory, this could be even faster.
But it would introduce a whole other set of problems:
What is the cache identity?
When does it expire?
What happens if the Glossary changes?
What happens if the Candidate changes?
What happens after a QC lineage update?
When an old cache conflicts with a new authority, which takes precedence?
Does the cache also need to enter the evidence schema?
Can Production / Reviewer rely on it?
Originally just trying to solve redundant work within a single CLI invocation, it is easy to end up creating a whole new persistent state management system.
And the actual problem at the time was not that complex.
We already knew:
Within the same invocation, many inputs do not change at all.
So there was no need to persist it across invocations.
What was ultimately adopted was:
invocation-local reuse.
After the command exits, this prepared state simply disappears.
The next execution prepares it anew based on the new current state.
This achieves most of the performance benefits without needing to add:
- persistent cache;
- cache schema;
- cache migration;
- cache invalidation;
- new receipt;
- new authority.
I later felt this point was very important.
Performance optimization can easily turn into:
Adding a whole new layer of state complexity to solve a complexity problem.
And the best part this time was not going down that path.
Prepared state does not mean “skipping validation”
There is another point here that can easily cause misunderstanding.
Sharing prepared state does not mean:
The first Page is checked, so the remaining Pages are not checked.
What is actually shared is:
That expensive foundational state which is identical for the entire invocation.
Each Page still needs to make its own determination.
For example, in the Course SEO scenario:
Shared:
glossary compatibility preparation
common inventory
common evidence state
Per Page:
seo:
seo-page:
Page-specific stale decisionSo after optimization it was still:
103 Pages
↓
103 Pages verified individuallyIt just no longer turned into:
103 Pages
↓
104 full compatibility world rebuildsThese two things must not be conflated.
If you directly reduce necessary Page-level validation for the sake of performance, you are just hiding the problem.
I also specifically added instrumentation
Just feeling that it “should only run once” is not enough.
The most common problem with this kind of optimization is:
The code looks shared, but some internal call secretly rebuilds a copy.
So during the optimization process, I also added instrumentation to the relevant paths.
The goal was very direct:
For a complete scope invocation, expensive steps like these:
PrepareCalls
FullReviewChecks
ContextInventoryBuilds
EvidenceSetReads
EvidenceVerifications
EdgePreparationsUnder normal circumstances should all be close to:
1Instead of:
122
103
470
...After optimization, the relevant instrumentation finally confirmed that these preparations had converged to a single execution.
At the same time, the original exact transition results remained unchanged.
That is:
We changed the computation process, not the state determination.
This is more important than just seeing the benchmark get faster.
The first results were already very obvious
On the quality-check scope path, the original full scope run took about:
373 secondsAfter optimization, the same scope could return to the second level.
Later, other equivalent Reviewer Bundle preparation paths also continued to reuse the same prepared resolver.
One actual result was about:
scope wall ≈ 2.10s
reviewer-bundle-equivalent wall ≈ 9.92sThe work done by these different commands varies, so I would not simply treat the two numbers as a benchmark for the exact same workload.
But it is already very clear that:
The previous pattern of rebuilding a full compatibility world for every TU is gone.
The CLI was back in a range where it could be used interactively and normally.
Course SEO exposed the same problem again
Interestingly, after optimizing Quality Check, I later encountered almost the exact same problem in Course SEO.
At the time, I needed to calculate which of the 103 Pages had stale SEO content due to the new glossary state.
The exact result was only 6 Pages:
flowcontrol/9
concurrency/1
concurrency/2
concurrency/5
concurrency/6
concurrency/9But the entire preflight took:
6m54.554sThat is:
414.554sAfter actually troubleshooting, I found the familiar pattern again:
for Page:
PrepareGlossaryCompatibility()About 103 Pages, plus the final extra path, totaling about 104 preparations.
This time the fix was very direct.
No reinventing another optimization scheme.
Instead, I continued to extend the approach already validated by Quality Check:
invocation-local shared lazy resolver.
After optimization, running it again:
10.335sOriginally:
414.554sBecame:
10.335sThat is about:
40.1×And more importantly:
stale Pages = 6Remained completely unchanged.
This is exactly the kind of performance optimization I wanted to see:
The same answer, just without repeating the same preparation work.
“Lazy” is also important
Later, this resolver further adopted a lazy approach.
The reason is that not every CLI command necessarily uses the full capabilities of Compatibility.
If the command unconditionally prepared all state at startup, although the N-times repetition problem was solved, paths that only use a few features would still bear unnecessary initialization costs.
So a more reasonable structure became:
invocation
↓
shared resolver
↓
A certain capability is truly needed for the first time
↓
prepare once
↓
all subsequent uses reuse itThis satisfies two goals simultaneously:
First, expensive state is not repeatedly prepared.
Second, if a command does not use this part of the state at all, it does not have to pay this cost upfront.
This is actually a very ordinary software engineering principle.
But in a complex workflow, without deliberate design, it is easily obscured by layers of helper functions.
The real problem often hides behind functions that “look cheap”
One thing that impressed me most during this optimization was that the calling code often did not look dangerous.
For example:
CheckPage(page)Looks like just checking a Page.
Or:
ResolveCompatibility(unit)Looks like just querying a TranslationUnit.
But if inside this function it continues to:
read archives
↓
load evidence
↓
verify identity
↓
build inventory
↓
prepare edges
↓
construct worldThen it is not a cheap getter at all.
Putting it in:
for 122 unitsAnd putting it in:
for 103 pagesWill cause performance to spiral out of control quickly.
This is also why I later felt:
Expensive preparation should become visible in the architecture.
If an operation requires building a complete compatibility world, the caller should not be misled into thinking it is just an ordinary query.
The prepared resolver effectively serves this purpose as well.
It makes the code structure closer to the real cost:
prepare expensive shared state
↓
perform many cheap queriesInstead of:
perform many apparently cheap queries
↓
each one secretly doing an expensive preparationThis is not a special optimization for a single command
Another important decision was not writing a special fast path for zh-CN or any specific command.
For example, it would have been entirely possible to add specific logic for:
quality-check scope --locale zh-CNOr make a special batch for the 103 Pages in Course SEO.
Short-term, it might have been faster.
But when adding another locale later, or when a new command enters the same Compatibility logic, the problem would return.
So what really needed optimizing was the common execution model:
Sharing the same deterministic prepared state within the same invocation.
This allowed the later:
- Quality Check;
- Reviewer Bundle;
- finalization;
- retranslation export;
- Course SEO;
- other workflows requiring compatibility;
To all gradually reuse the same approach.
That is, this was not:
“Making a slow command fast.”
Rather, it was fixing:
Why all commands that call this kind of expensive state would gradually slow down.
Performance optimization must not break fail-closed
There is another particularly tricky issue with this kind of shared state.
If the resolver, in pursuit of performance, continues to use old state when it finds an identity mismatch, then it is fast, but correctness is lost.
So the prerequisite for sharing prepared state remains:
It belongs to the same deterministic invocation and the same set of current inputs.
When there is a real inconsistency, missing data, or unprovable state, it should still:
FAILEDInstead of:
fallback to something cachedSo this optimization did not change the original fail-closed principle.
Nor did it reduce the number of validations just to make the benchmark look good.
What was actually reduced was only:
Deterministic, redundant preparation work.
From a 40x improvement, what I really care about is not the 40x
414.554s → 10.335s, about 40.1 times.
This number is certainly great for a benchmark.
But looking back, I feel what is more important is not:
“I optimized the Go CLI by 40 times.”
Rather, this optimization validated a healthier way for the system to evolve.
At the very beginning:
SHA gateFast.
Later, to improve accuracy, it introduced:
Semantic CompatibilityBetter correctness, but a clear increase in execution cost.
If at this point, due to performance issues, the new semantic capability was simply deleted, it would actually be an architectural regression.
A more reasonable path should be:
A more precise semantic model
+
An execution model that matches itUltimately retaining both:
- a more precise affected scope;
- trusted result reuse;
- fail-closed;
- deterministic evidence;
- second-level CLI.
This is what I feel is the true value of this optimization.
Performance itself is part of the workflow
When building CLIs before, I sometimes felt:
As long as the result is correct, being a bit slow is fine.
But after doing v2 this time, I increasingly feel that this idea only holds true in very small systems.
When a workflow needs to run repeatedly:
- scope;
- bundle export;
- current check;
- validation;
- QC record;
- revision;
- re-QC;
- finalization;
- Surface;
If every step takes minutes to wait, what is really lost is not just CPU time.
It is also human attention.
A command that finishes in 2 seconds can be executed continuously while judging the next step.
A 7-minute command constantly interrupts the work rhythm.
What is even more troublesome is that when a step fails, you have to wait the same number of minutes again to verify the fix.
Therefore, for this kind of engineering that heavily relies on the CLI to advance state:
Performance is not just icing on the cake.
It actually determines whether the entire workflow can iterate efficiently.
The principle formed this time
Looking back now, this performance optimization can be summarized into a very simple principle:
Expensive, deterministic state that does not change within the same invocation is prepared only once.
Then:
Prepare once
↓
Reuse many times
↓
Per-unit validation remains
↓
Exact result unchangedNo new persistent cache.
No special path written for a specific locale.
No reduction in truly necessary validation.
No change to the judgment semantics of Compatibility.
No change to fail-closed.
Just removing the redundant work originally hidden inside the loop.
Sometimes, a dozens-of-times performance improvement does not require a particularly clever new algorithm.
You just need to see clearly first:
What the system is actually repeating.
Next article
After solving the CLI performance issue, v2 has another class of problems more troublesome than speed.
That is parsers and representation.
Tour, Learn, and Docs simultaneously contain different structures like Markdown, inline code, link binding, full code, and instructional comments.
The machine must be strict enough to ensure that this content is not broken during the AI translation process.
But if the protection rules are too mechanical, they can in turn restrict the natural expression of the target language.
So next I plan to continue organizing:
Why Learn / Docs needs a new parser and structural protection, and how to find the boundary between machine safety and natural language.