Another Protected Token Validation Failure in GLM-5.2 Whole-Article Translation: From Missing Tail Tokens to Plaintext Structure Reordering

On July 30, 2026, while continuing to use the WordPress AI translation workflow I built to translate entire Chinese articles into English with SlyTranslate + GLM-5.2, I ran into a familiar problem again:

Protected token validation failed: expected_count=606, actual_count=605, missing_tokens=["SWQSTRUCT000458END"], extra_tokens=[], fixed_order=no, first_mismatch_index=269

Protected token validation failed: expected_count=606, actual_count=605, missing_tokens=["SWQSTRUCT000458END"], extra_tokens=[], fixed_order=no, first_mismatch_index=269

At first glance, it looked like yet another ordinary Protected Token loss.

But the investigation that followed turned out to be more valuable than expected.

This time, I successively tried automatic repair for trailing STRUCT Tokens, strengthened the tail-integrity requirements in the GLM Prompt, added a Tail Guard, and carried out multiple real production validations.

Only then did I discover the real issue:

The problem was not simply that “GLM occasionally drops Tokens at the end of an article.” Some unreasonable Gutenberg Plaintext structures were prompting GLM to reorganize sentences across blocks in order to produce more natural English, ultimately causing Protected Token order drift.

In the end, I did not relax validation or fall back to manual translation with ChatGPT. Instead, I reorganized the Gutenberg structure of the Chinese source article itself.

After running the whole-article GLM-5.2 translation again, it passed validation successfully.

This article documents the full investigation.

1. Why I Still Insist on Whole-Article Translation with GLM

I have already invested a considerable amount of time in building this WordPress AI translation workflow.

Chinese articles are currently translated through SlyTranslate with GLM-5.2. One important goal is to send the entire article to the model in a single request whenever possible, so the model has the full context instead of translating the title, paragraphs, lists, and other elements independently block by block.

This is especially important for technical articles.

A single article often contains references to earlier and later passages, technical terminology, relationships between commands and explanations, contextual references such as “it,” “this issue,” and “the configuration above,” as well as long-range relationships where a problem is raised earlier and a conclusion is given later.

If the content is split into many small units and the model is called separately for each one, the structure becomes easier to control, but translation quality may decline.

So I still prefer the following approach:

Let GLM see the entire article at once and translate the whole article in one pass.

The trade-off is that another problem must be solved:

How to fully preserve the WordPress Gutenberg structure while a large language model generates the entire English article.

To do this, the current workflow first replaces Gutenberg structures that the model must not modify with Protected Tokens such as SWQSTRUCT000001END. Special content such as Plaintext also has corresponding protection markers.

After GLM finishes translating, the program strictly validates the number, missing items, extra items, duplicates, and order of all Tokens.

Only after everything is correct is the Gutenberg structure restored and the English article overwritten.

Therefore, if the model drops, duplicates, or reorders any Token, the translation must fail rather than saving potentially damaged Gutenberg structure.

2. First Failure: Only the Final Token Was Missing

The article affected this time was post_id=20878.

The first failure looked like this:

expected_count=606
actual_count=605
missing_tokens=["SWQSTRUCT000458END"]
extra_tokens=[]

expected_count=606
actual_count=605
missing_tokens=["SWQSTRUCT000458END"]
extra_tokens=[]

After inspecting the diagnostic file further, I found that SWQSTRUCT000458END corresponded to the closing comment of the Gutenberg Quote block: <!-- /wp:quote -->.

GLM had already output the translated body of the quote and </blockquote>, then immediately generated </slytranslate-output>.

In other words:

The HTML itself had already been closed, but the Protected Token corresponding to the final Gutenberg closing comment was omitted.

More importantly, this happened at the very end of the article.

3. Initial Hypothesis: The Model Might Be Wrapping Up Too Early at the End of the Output

This was not the first similar incident that day.

Combined with the earlier failures, I began to suspect the following:

After GLM decides that all natural-language content has been translated, it may enter an “end the output” state too early, causing it to omit Gutenberg structure Tokens at the very end of the article where there is no visible natural-language content.

For example, the end of an article may already contain </p> and </blockquote>, while Gutenberg closing comments such as <!-- /wp:paragraph --> and <!-- /wp:quote --> still follow.

For WordPress, these are of course still part of the document structure.

But from the language model’s perspective, once the HTML looks closed, it may assume the content is finished and directly generate the closing tag of the output wrapper.

So instead of immediately modifying the Chinese article, I first tried to solve the problem in the translation pipeline itself.

4. First Improvement: Safe Automatic Repair for Trailing STRUCT Tokens

I did not want to call GLM again to translate the entire article just because a Token was occasionally omitted.

At the same time, I did not want to simply relax validation—for example, allowing the translation to pass whenever only one Token is missing.

Today the missing structure was <!-- /wp:quote -->; next time it could be a more important Gutenberg structure.

So the final approach was:

Allow deterministic repair, but still require the repaired result to pass the full strict validation again.

The new processing flow works as follows:

After GLM returns a result, the existing strict Token validation runs first. If it fails, the program then determines whether the failure is a sequence of trailing STRUCT Tokens that can be repaired safely. Only when all strict conditions are met are the missing Tokens restored automatically, after which the same full validator is run again.

The conditions include:

• No extra Tokens;
• No duplicate Tokens;
• All missing items must be STRUCT Tokens;
• The missing items must form a contiguous suffix of the expected Token sequence;
• The actual Token sequence must exactly equal the complete expected prefix after removing that trailing suffix;
• The program must be able to determine the original structure corresponding to every missing Token from the protected map.

The repaired result is not allowed to pass automatically.

Only after revalidation confirms that the Token count, order, missing items, extra items, duplicates, and other checks are all correct can restoration and saving continue.

I also added multiple test scenarios, including one missing final STRUCT Token, two consecutive missing final STRUCT Tokens, a missing Token in the middle, extra Tokens, duplicate Tokens, incorrect order, a non-STRUCT Token, and a case where revalidation still fails after repair.

After all tests passed, I deployed the changes to production.

5. Real Production Validation Failed Again—This Time Two Tokens Were Missing

After translating 20878 again, the result became:

expected_count=606
actual_count=604
missing_tokens=["SWQSTRUCT000457END","SWQSTRUCT000458END"]
extra_tokens=[]

expected_count=606
actual_count=604
missing_tokens=["SWQSTRUCT000457END","SWQSTRUCT000458END"]
extra_tokens=[]

The two missing Tokens corresponded to <!-- /wp:paragraph --> and <!-- /wp:quote -->.

GLM had again output the English content of the final paragraph, </p>, and </blockquote>, and then ended the output.

On the surface, this seemed to support the earlier hypothesis even more strongly:

After finishing the natural language and HTML, the model wrapped up immediately and omitted the final two Gutenberg Tokens.

But the newly added Tail Repair did not run.

That meant the problem was not as simple as it first appeared.

6. Second Attempt: Strengthening the Prompt and Adding a Tail Guard

Because trailing Token loss occurred repeatedly, I reviewed the Prompt actually sent to GLM-5.2.

The original Prompt already explicitly required:

• All Protected Tokens must be preserved completely;
• Each Token must remain unique;
• The count and order must not change;
• SWQSTRUCT represents Gutenberg structure;
• Check for omissions before returning the result.

However, it did not specifically emphasize the following:

Even if the natural-language content has ended, any remaining Protected Tokens must still be output.

So I added a more explicit rule for the output tail:

Pay special attention to the end of the output. Every protected placeholder at the end of the input must be reproduced exactly, one by one, even if there is no natural-language content before or after it. Outputting an HTML closing tag does not mean the task is complete; SWQSTRUCT Tokens representing Gutenberg closing comments may still follow the HTML. The body may only end after confirming that the final protected placeholder in the input has been output in its original order.

At the same time, I added a Tail Guard:

SWQTRANSLATIONTAILGUARDEND

It is placed after the real article content and the final Protected Token.

The purpose is simple:

Prevent the real final Gutenberg Token from sitting at the absolute end of the model output.

The Tail Guard itself is not part of the WordPress article. It does not enter the protected map, normalization, or Gutenberg restoration, and the program removes it after checking it.

It is simply an additional output buffer.

7. The Tail Guard Was Returned Successfully, but the Real Tokens Were Still Missing

After deployment, I translated 20878 again.

The result was still:

expected_count=606
actual_count=604
missing_tokens=["SWQSTRUCT000457END","SWQSTRUCT000458END"]

expected_count=606
actual_count=604
missing_tokens=["SWQSTRUCT000457END","SWQSTRUCT000458END"]

However, this run produced a crucial new piece of evidence.

The diagnostics showed:

translation_tail_guard_status=present, with translation_tail_guard_count=1.

The end of the raw response was actually:

SWQSTRUCT000455END</blockquote>
SWQSTRUCT000456END
SWQTRANSLATIONTAILGUARDEND
</slytranslate-output>

SWQSTRUCT000455END</blockquote>
SWQSTRUCT000456END
SWQTRANSLATIONTAILGUARDEND
</slytranslate-output>

In other words:

GLM saw the Tail Guard and successfully returned it, yet still skipped SWQSTRUCT000457END and SWQSTRUCT000458END.

This made the explanation that “the model simply reached the output boundary and did not have time to generate the last two Tokens” much less plausible.

The Guard was clearly after those two Tokens and was returned normally.

So the focus shifted from the end of the article to the Token order inside the article as a whole.

8. The Real Clue: Token Order Had Already Drifted in the Middle of the Article

Tail Repair did not run because it deliberately failed closed.

The log showed:

protected_token_tail_repair_attempted=false
protected_token_tail_repair_reason=actual_sequence_not_expected_prefix
protected_token_tail_repair_revalidation=not_run

protected_token_tail_repair_attempted=false
protected_token_tail_repair_reason=actual_sequence_not_expected_prefix
protected_token_tail_repair_revalidation=not_run

Although the final two consecutive STRUCT Tokens were indeed missing, the actual Token sequence was not the complete prefix of the expected Token sequence with those final two Tokens removed.

In other words:

The problem had already occurred in the middle of the article.

After locating the first mismatch, I found that SWQPLAINSTART000026END, which should originally have appeared after a normal Paragraph, had been moved two STRUCT Tokens earlier by GLM.

This was no longer simply a case of “the last two Tokens were omitted.”

9. Why GLM Actively Reordered These Structures

After examining the corresponding Chinese Gutenberg content, the cause became immediately clear.

The source article actually used the following structure:

Turning off “Automatic Suspend” only resolves:

Then a separate Plaintext block contained:

Prolonged inactivity

Then a normal Paragraph continued with:

the suspend caused by it.

The next sentence was similar:

But:

Then a separate Plaintext block contained:

Closing the laptop lid

followed by:

is another independent action.

Visually, this is perfectly readable on the Chinese page.

But from a natural-language perspective, these are really just two sentences:

Turning off “Automatic Suspend” only resolves suspend caused by prolonged inactivity.

And:

However, closing the laptop lid is a separate action.

In other words, Gutenberg had artificially split one complete natural-language sentence into:

Paragraph → Plaintext → Paragraph

To produce more natural English, the model proactively recombined these pieces.

As a result, the one-to-one correspondence between the natural language and the original Gutenberg blocks was broken.

At the Token level, the entire Plaintext block moved ahead of the STRUCT Tokens that originally preceded it.

10. Why This Cannot Be Fixed by Simply Reordering Tokens in Code

At first glance, it seems possible to simply move the relocated Plaintext Token back to its original position.

But the real problem is:

GLM did not only move Tokens; it also reorganized the natural language.

Text that was originally distributed across several Gutenberg blocks may already have been merged into the preceding Paragraph after translation.

If the program only forces the Tokens back to their original positions, the result could look like this:

But:

is another independent action.

Closing the laptop lid

The Token order would be restored, but the English sentence would be broken.

Therefore, this is not a local Token problem that can be repaired safely and deterministically.

Continuing to fail strict validation is the correct behavior.

11. Final Solution: Modify the Gutenberg Structure of the Chinese Article Itself

At this point in the investigation, I decided not to keep expanding the repair scope of the translation program.

What really needed adjustment was the unnecessary Plaintext structure in the Chinese source article itself.

For example, one sentence had originally been split into three blocks:

Turning off “Automatic Suspend” only resolves:

Prolonged inactivity

the suspend caused by it.

I restored it directly as a normal Paragraph:

Turning off “Automatic Suspend” only resolves suspend caused by prolonged inactivity.

The other sentence was changed to:

However, closing the laptop lid is a separate action.

This way, one complete natural-language semantic unit once again corresponds to one complete Gutenberg content unit.

GLM no longer needs to move text across blocks to produce grammatical English.

12. After Reviewing the Whole Article, I Found Far More Than Two Unnecessary Plaintext Blocks

After reviewing the entire article further, I found that I had previously used quite a few Code Block Pro / Plaintext blocks for visual presentation even though they contained only ordinary natural language.

For example, a simple plan such as “Disable automatic suspend + disable suspend on lid close + keep automatic screen blanking” is fundamentally just one sentence; there is no need to put it in a separate code block merely for presentation.

Likewise, a simple procedure such as “Close the lid → wait about 30 seconds → open it again → continue working normally” can usually be represented with a normal paragraph or list.

This does not mean Plaintext should never be used.

Real system logs, for example:

PM: suspend entry (deep)
ACPI: PM: Low-level resume complete
PM: suspend exit

PM: suspend entry (deep)
ACPI: PM: Low-level resume complete
PM: suspend exit

are obviously still appropriate for code blocks.

Shell commands, PHP, JSON, configuration files, real terminal output, and ASCII diagrams whose meaning depends on monospaced characters, indentation, or fixed alignment should also continue to preserve their original formatting.

The key question is not whether Plaintext itself is problematic, but:

Does this content genuinely require fixed formatting to express its meaning?

13. GLM Translation Succeeded After Reorganizing the Chinese Source

In the end, I reorganized the Gutenberg source of the entire article.

The principle was simple:

Use Paragraph / List for ordinary natural language, and keep Code Block Pro / Plaintext only for content that truly depends on its original formatting.

In particular, avoid structures such as:

Half a sentence → Plaintext → the rest of the sentence

where a complete sentence can only be formed across multiple Gutenberg blocks.

I then ran the whole-article GLM-5.2 translation again.

This time, the translation succeeded.

I did not disable strict Protected Token validation, nor did I fall back to manually translating the entire article with ChatGPT.

This showed that the genuinely effective direction was not to keep adding more and more special cases to Token Repair, but rather to:

Keep the Gutenberg structure of the article itself aligned as closely as possible with the semantic boundaries of the natural language.

14. The Translation Safeguards Added Earlier Are Still Kept

Although reorganizing the article structure was what ultimately solved 20878, the safeguards added earlier are still valuable.

Strict Protected Token Validation

This layer remains in place.

If missing, extra, duplicate, or order anomalies occur, the English article must not be overwritten directly.

Otherwise, damaged Gutenberg structure could be saved into WordPress.

Limited Tail STRUCT Repair

Automatic restoration is allowed only when the missing items can be confirmed to be consecutive trailing STRUCT Tokens, there are no other order anomalies, the actual sequence is the complete prefix of the expected sequence, and the repaired result can pass strict validation again.

It remains a safe fallback, not a relaxation of validation.

Strengthened Prompt Tail-Integrity Rules

The Prompt continues to tell GLM explicitly:

Closing the HTML does not mean that all Gutenberg Protected Tokens have been output.

Tail Guard

The workflow continues to use SWQTRANSLATIONTAILGUARDEND so that the real final WordPress Token does not always sit directly at the model’s output boundary.

It cannot solve every structural problem, but it is still worth keeping as an additional safeguard.

15. Code Tested, Committed, and Pushed

After the final changes were complete, I reran the relevant tests.

These included Plaintext structure, Protected Token validation, Adjacent Token repair, Token normalization, Tail STRUCT Repair, Tail Guard, PHP syntax checks, and git diff --check.

All passed.

The three implementation files remained identical.

The final commit was:

07aa11bc1b2837bb3a7a2275679608256c42b3f6

Commit message:

fix: strengthen GLM translation tail structure protection and safe repair

The code was successfully pushed to origin/main. The local main branch is synchronized with the remote, and the Git working tree is clean.

16. The Real Lesson: AI Translation Quality and Gutenberg Authoring Are Not Independent Concerns

Previously, I focused more on:

How to make the translation program protect Gutenberg.

After this incident, I realized I also need to look at the problem from the opposite direction:

How to make Gutenberg content itself more suitable for whole-article AI translation.

In particular, a single natural-language sentence should not be split across several different types of Gutenberg blocks.

For example:

Paragraph: first half of a sentence
Plaintext: a few words in the middle
Paragraph: second half of the sentence

From the WordPress structure’s perspective, these are three independent content units.

But from the language model’s perspective, they are actually one sentence.

If the model is simultaneously required to:

1. Translate the entire article into natural English;
2. Never move even a single word across Gutenberg boundaries;
3. Keep hundreds of Protected Tokens in exactly the original order;

these goals can inherently conflict under certain unusual content structures.

A more reasonable approach is:

Make each natural-language semantic unit correspond, as much as possible, to one complete Gutenberg content unit.

This benefits both translation quality and the clarity of the article’s source structure.

17. Reduce Unnecessary Plaintext and Code Blocks in Future Articles

From now on, I plan to make this rule part of my normal blog-authoring workflow.

Ordinary single-sentence explanations, a couple of sentences, simple status descriptions, short procedures, conclusions, ordinary examples, domain names, command names, parameter names, and similar content should preferably use normal paragraphs, lists, or inline code.

For example, a statement such as “allow it to pass when only one Token is missing” is simply a condition described in prose. A normal paragraph is sufficient; there is no reason to dedicate an entire code block to it.

Even a Token name such as SWQSTRUCT000458END should use inline code when it is merely mentioned in the body text.

Code blocks should be used only when multiline original formatting, indentation, line breaks, or machine-generated output genuinely needs to be preserved.

For example:

• Program code;
• Shell commands;
• Multiline configuration;
• JSON;
• System logs;
• Real terminal output;
• ASCII diagrams that require a fixed layout.

In particular, ordinary text should not be turned into large numbers of Plaintext blocks merely to make it “look like a process.”

18. Conclusion

At first, this failure appeared to be nothing more than a missing final Protected Token.

On the next validation attempt, the final two Tokens were missing.

So the investigation initially focused naturally on the output tail, the Prompt, Tail Guard, and Tail STRUCT Repair.

But the key issue ultimately turned out to be in the middle of the article:

To reorganize Chinese text split across multiple Gutenberg blocks into natural English, GLM proactively changed the correspondence between Plaintext and Paragraph blocks, causing Token order drift.

In that situation, forcing Tokens back into place programmatically cannot reliably restore the translated content.

The final solution was actually very simple:

Remove unnecessary Plaintext blocks and restore artificially fragmented natural language to normal Paragraphs.

After running the whole-article GLM-5.2 translation again, it passed validation successfully.

Therefore, the most important conclusion from this incident is not “add another Repair mechanism to the translation pipeline.”

Instead:

To make whole-article AI translation both natural and stable, the translation program needs to protect Gutenberg structure, but the Gutenberg structure of the Chinese source article should also align as closely as possible with the semantic boundaries of the natural language itself.

When preparing WordPress technical articles in the future, I will also avoid creating unnecessary Plaintext or code blocks purely for visual effect, keeping the source structure simpler, more natural, and better suited to subsequent whole-article AI translation.