In the previous article, I outlined why Go Learning & Documentation v2 ultimately chose:
One locale, maintaining only a single official glossary.
Tour, Learn, Docs, and any other content packages added in the future, all share the same set of official terminology rules.
This prevents the same Go concept from being decided repeatedly across different packages and keeps the language experience consistent across the entire locale.
But once the Unified Glossary was actually established, another question immediately arose:
Whenever the glossary changes, what should happen to the content that has already been translated and reviewed?
The simplest solution is very straightforward:
old glossary SHA
≠
new glossary SHA
↓
all TranslationUnits stale
↓
re-translate
↓
re-reviewFrom an engineering perspective, this approach is even quite attractive.
The rules are simple.
It is easy to verify.
And it is very safe.
As long as the glossary file changes, none of the previous results are trusted.
But when applying this to a long-term project that already has a large amount of high-quality translation work, I increasingly felt that:
While simple, this is not a reasonable long-term solution.
A file change does not mean all translations are wrong
Suppose a new term is added to the Simplified Chinese glossary.
This term only appears in two pages.
The concept does not appear at all in over a hundred other TranslationUnits.
If, simply because:
glossary.yaml SHA changedall pages are required to undergo Generation and QC again, then the vast majority of the work has no linguistic significance.
The English source for those pages has not changed.
The Chinese candidate has not changed.
The newly added term does not appear in them.
The previous Independent QC results still hold.
The only thing that has changed is:
The SHA of the shared glossary file they reference is now different.
If the entire system equates “file identity change” directly with “linguistic semantic invalidation,” the result is a massive amount of unnecessary rework.
This is also one of the core problems v2 later needed to solve:
What we really care about is not whether the glossary file has changed, but whether this change affects a specific TranslationUnit.
SHA is great for proving identity, but not for directly proving semantic impact
This is not to say that SHA has no value.
On the contrary, SHA is very important throughout the project.
It can accurately tell us:
Whether these two inputs are exactly the same.
For example:
old glossary SHA = A
new glossary SHA = BAs long as A != B, it clearly proves that the byte content of the glossary has changed.
But SHA cannot answer another question:
What exactly does this change mean?
It might just be:
- adjusting the order of entries;
- modifying descriptive text;
- adding a word that does not even appear in the current corpus;
- modifying a term that only affects a specific Page or two;
- modifying a core term that appears almost throughout the entire Go documentation.
From the perspective of SHA, these scenarios are all exactly the same:
changedBut from the perspective of a translation system, their scope of impact is completely different.
So in v2, glossary identity and glossary semantic impact can no longer be treated as the same thing.
The first step is not retranslation, but finding out “what exactly changed”
A more reasonable approach is to first compare:
What is the actual semantic delta between the old glossary and the new glossary?
What we care about here is not which line in the YAML changed, but what changed in the official language rules.
For example, a term might change from:
preferred: Ato:
preferred: BOr a new rule might be added:
forbidden: COr a Go concept that previously had no formal constraint enters the glossary for the first time.
What truly needs to propagate to the TranslationUnit are these language rule changes.
Not the file-level diff itself.
Thus, the entire judgment shifts from:
glossary changed?to:
which glossary semantics changed?These are two completely different questions.
Step two: In what content does this change actually appear?
Once we know which terms changed, the next question becomes:
Which TranslationUnits are related to these changes?
Suppose only one term was modified this time:
fooThe most direct thing to check then is which source contexts actually involve the concept corresponding to foo.
If a Page has nothing to do with this term from start to finish, there is no reason for it to be dragged back into the workflow due to this glossary mutation.
Therefore, the logic can be roughly understood as:
Glossary semantic delta
↓
Relevant source contexts
↓
Relevant TranslationUnitsThis step can already eliminate a large number of completely unrelated content.
But it is not over yet.
Because:
The presence of an affected term in the source does not necessarily mean the current translation has a problem.
Step three: The old translation might already comply with the new terminology rules
This is a point I consider quite important in the entire mechanism.
Suppose the glossary previously had no formal rule for translating a certain term.
But when Generation translated a certain Page, it naturally adopted the translation that was later formally established in the glossary.
For example:
Old glossary:
No formal rule
Existing candidate:
Already uses A
New glossary:
preferred / required decision = AIn terms of dependencies, this Page is indeed related to the new glossary rule.
But looking at the final linguistic result:
It already satisfies the new rule.
If we still force a re-Generation at this point, it does not make much sense.
It might even have the opposite effect.
A stable translation that has already passed Independent QC might exhibit new expression changes when sent back into the model.
Then the Reviewer has to review it again.
That is to say:
To fix a problem that does not actually exist, a new linguistic change is created instead.
Therefore, determining the affected scope cannot just ask:
Does this word appear in the source?
It must also continue to ask:
Is the current candidate already compatible with the new glossary?
Relevant and affected are not the same concept
This is also a distinction I increasingly valued later on.
A TranslationUnit can be relevant to a certain glossary change, but ultimately still be compatible.
It can be roughly understood as:
Glossary change
↓
TU relevant?
│
├── No
│ ↓
│ unaffected
│
└── Yes
↓
candidate already compatible?
│
├── Yes
│ ↓
│ compatible / carry
│
└── No
↓
affectedOnly the content that is truly affected at the end should enter subsequent processing.
This might seem like just skipping some translation tasks, but its actual significance is greater.
Because it starts to separate:
“A dependency has changed”
and:
“Existing results are now invalid”
into two different states.
This is a problem many long-running systems need to face.
The value of Compatible is not just saving one Generation
If it were just about saving a few AI calls, this mechanism would not be worth making so complex.
What is truly important is that it protects the high-quality work that has already been completed.
Suppose a certain TranslationUnit has already gone through:
Generation
↓
Machine validation
↓
Independent QC
↓
AAnd has even gone through:
B / C
↓
exact revision
↓
re-QC
↓
AAt this point, it is no longer just “text that can be regenerated at any time.”
It is effectively a verified linguistic asset.
Without an actual semantic reason, all these assets should not be discarded just because an external file’s SHA changed.
Therefore, the principle behind semantic compatibility is actually consistent with something the project increasingly emphasized later on:
Work that has been completed to a high quality and is not genuinely affected should be reused as much as possible.
History is not preserved just for the sake of preservation.
First and foremost, it should help us avoid repeating work that was already done correctly.
But “already compatible” cannot be based on guessing
On the other hand, we cannot just assume an old candidate is fine simply to reduce rework.
If the system cannot reliably prove that a TranslationUnit is compatible with the new glossary, then continuing to carry it by default is equally risky.
Therefore, the principle here cannot be:
“It looks like there should be no impact.”
Instead, we should build machine-reproducible evidence as much as possible.
At the very least, it needs to be able to answer:
- what the old glossary was;
- what the new glossary is;
- which formal rules changed;
- which source contexts are related to the changes;
- what the corresponding content is in the current candidate;
- why this context is determined to be compatible or affected.
That is to say:
Reducing rework must not come at the cost of reduced verifiability.
If a situation cannot be safely proven compatible, the more reasonable approach should still be to fail closed and send it for further inspection.
This is much safer than “trying to judge things as compatible just to boost reuse numbers.”
Preferred, mandatory, forbidden, and other rules cannot be simply equated
Another place where it is easy to make the affected scope too broad is treating all glossary rules as the same type of constraint.
But in an official glossary, different types of rules themselves can represent different linguistic meanings.
For example:
- a certain word must use a fixed expression;
- a certain word prefers a specific expression;
- a certain old translation should be forbidden;
- a certain technical term should be kept in English.
The risks of these changes are not all the same.
If an explicitly forbidden translation still exists in the candidate, the problem is usually quite clear.
But a change in a preferred term does not necessarily mean that all differing expressions must immediately be judged as errors.
How to handle this specifically still needs to follow the official glossary’s definitions for different categories.
So semantic compatibility also cannot degrade into:
whether string A appearsIt requires understanding:
What this glossary rule itself demands.
This is also why this mechanism later became much more complex than a simple string search.
Page is the governance unit that ultimately decides if it is affected
In the previous article, I just wrote that although the parser can break down many slots internally in Learn / Docs, formal language governance still uses the physical Page as the TranslationUnit.
This design is also very important here.
A Page might have multiple glossary contexts internally.
Among them:
context 1 → compatible
context 2 → compatible
context 3 → affected
context 4 → compatibleUltimately, this Page should still be treated as:
affectedBecause as long as there is one formal language issue that needs fixing, this Page needs to enter the corresponding workflow.
Conversely, if all relevant contexts can be proven compatible, then the entire Page can continue to reuse the original result.
So internally, fine-grained analysis can be done.
But ultimately, governance returns to the Page.
This aligns perfectly with the design principle from the previous article:
The machine can analyze problems very granularly, but the formal language state should still correspond to the content unit the user ultimately reads.
Affected also does not mean “retranslate the entire page”
There is another particularly easy place to misunderstand here.
A Page being judged as affected does not mean:
Regenerate the entire page.
Suppose that in a 2000-word Page, only one term needs adjustment due to a glossary change.
The most reasonable approach is certainly not to regenerate the other 1990+ words that have already passed review.
In this case, exact revision should still be used:
affected Page
↓
exact finding / affected context
↓
Generation revision
↓
machine validation
↓
Independent re-QCThat is to say:
The affected scope determines “which formal TranslationUnits need to re-enter the workflow”;
exact revision determines “exactly how much is modified within the TranslationUnit.”
These two levels cannot be confused.
Only in this way can we simultaneously achieve:
- not missing any genuinely affected Pages;
- and not amplifying a localized term change into a full page retranslation.
Why “redoing everything” might actually reduce stability
On the surface, retranslating everything is a very conservative approach.
Because all content is regenerated according to the latest glossary.
It seems the safest.
But from the perspective of a long-term project, it might actually reduce stability.
The reason is simple:
AI Generation is not a pure function.
Even if the inputs are very similar, regenerating the same content might result in different:
- sentence structures;
- word choices;
- word order;
- paragraph expressions.
Many of these changes are not fixes.
They are just another equally valid expression.
If every time a small glossary change occurs, the entire locale is regenerated, the system will produce a massive amount of unnecessary linguistic churn.
Reviewers also have to re-examine these changes that are actually unrelated to the glossary mutation.
The final result is:
The content that actually needs fixing accounts for only a small portion,
but the vast majority of time is spent verifying unrelated changes.
Therefore, “only processing affected content” is not just about saving costs.
It is also about reducing meaningless linguistic drift.
This is actually a bit like cache invalidation
Reaching this point, I realized that this problem is actually very similar to a classic problem in software engineering:
Cache invalidation.
If a certain dependency changes, the simplest strategy is of course:
clear everythingThis definitely ensures no stale cache is left behind.
But the efficiency is very poor.
A more mature system will continue to evaluate:
what actually depends on this change?The glossary is similar.
A glossary mutation is itself just a dependency change.
What truly needs to be invalidated is:
Content that semantically truly depends on this change, and whose current results no longer meet the requirements.
Not all content that has ever used the glossary.
Therefore, I later became increasingly willing to understand this mechanism as a kind of:
semantic invalidation
rather than just “terminology checking”.
But a language system is more troublesome than an ordinary cache
Of course, it also cannot be completely equated to an ordinary cache.
Dependencies in a compilation system are often more deterministic.
If file A imports file B, a relatively clear dependency graph can be established after B changes.
Natural language is not that simple.
Whether a glossary rule affects a translation might involve:
- source context;
- candidate expression;
- technical meaning;
- rule category;
- synonymous expressions;
- natural language choices within the context.
So this is not just about building a:
term → pagereverse index and being done with it.
The truly difficult part is the latter half:
Relevant does not mean invalid.
This is also the purpose of semantic compatibility.
This mechanism also exposed a new performance issue
However, changing from a simple SHA gate to semantic compatibility also brought new costs.
Previously, checking:
SHA equal?could be completed almost instantly.
Now, it requires:
- reading the glossary semantic delta;
- finding relevant contexts;
- analyzing the source;
- analyzing the candidate;
- determining compatibility;
- aggregating to the TranslationUnit;
- generating corresponding evidence.
Once these logics started being called repeatedly by a large number of commands, CLI performance quickly showed noticeable issues.
Some workflows that should have completed quickly started taking tens of seconds, or even minutes.
Later investigation revealed that the real problem was not:
the semantic compatibility direction being wrong.
But rather:
The same expensive compatibility state was being rebuilt too many times.
This is also why, in the October 8th article, I had already recorded a related performance issue once.
And later I plan to write a dedicated article on:
From minutes to seconds: how this CLI was eventually optimized.
Because it was also a fairly typical instance in this v2:
The semantic design was correct, but the execution architecture still needed further adjustment.
The final rules formed are actually not complex
Although the implementation process gradually became complex, the rules we ultimately wanted to achieve were actually quite simple.
When the glossary changes, we no longer immediately assume:
all historical results are invalid.
Instead, we sequentially determine:
1. What semantic changes actually occurred in the Glossary?
2. Which TranslationUnits are related to these changes?
3. For relevant TranslationUnits:
Does the current candidate already satisfy the new rules?
4. If already compatible:
Retain the trusted result.
5. If genuinely affected:
Only let the affected TranslationUnit enter revision / validation / re-QC.
6. When a reliable judgment cannot be made:
fail closed, do not guess compatibility.What it ultimately wants to protect are two things:
The new glossary rule must actually take effect.
While also:
Previously correctly completed work must not be entirely overturned due to unrelated changes.
Moving from “has the file changed” to “is the semantics affected”
Looking back now, this is probably also a very typical change in v2.
Early on, many state checks easily relied on:
SHA changedbecause it was simple, deterministic, and failed closed.
But as the project scale grew larger, a mere identity change was no longer sufficient to determine the workload.
The system needed to start understanding further:
What exactly changed?
Who does this change actually affect?
Which completed results still hold?
This does not mean SHA will no longer be used in the future.
On the contrary, SHA is still responsible for proving input identity.
It just no longer solely bears the burden of:
Determining whether linguistic semantics are invalid
a responsibility that never belonged to it in the first place.
And this affected scope mechanism ultimately became an increasingly important design direction in all of v2:
Only let content that is genuinely affected by changes re-enter the workflow.