WordPress Tag Cleanup (Part 1): A Failed Attempt at LLM Matching

While advancing blog search engine optimization, I encountered a severe obstacle: the multilingual synchronization mechanism generated a large number of redundant URLs like /en/tag/中文. This not only bloated the tag database but also made it difficult for search engines to properly index the English pages. To thoroughly eliminate these SEO-unfriendly URLs, I had to merge and clean up the multilingual tags…

My WordPress site uses Polylang to support both Chinese and English. Because of a PHP synchronization script, Chinese tags are automatically copied to the English language every time I publish a post. Over time, this created data redundancy: for example, both “支付宝” and “alipay” exist simultaneously under both the Chinese and English languages.

This not only bloated the tag database but also generated redundant URLs like /tag/支付宝 and /en/tag/支付宝. My ultimate goal was clear: completely eliminate URLs containing Chinese characters under the /en/tag/ path. To achieve this, I had to merge “支付宝” into “alipay” (or simply delete it) under the Chinese language, and then perform the same cleanup operation under the English language.

Initially, I thought this semantic matching task was well-suited for large language models, so I began testing major mainstream models for data cleaning. However, the final results forced me to shift my approach to writing code.

---

Phase 1: Raw Data Extraction and Integration

A prerequisite for using large language models is providing a complete dataset. Exporting the full dataset from Alibaba Cloud DMS was quite tedious.

First, I needed to query all tags marked as Chinese by Polylang whose slugs contain Chinese characters.

DMS query results only display the first 3,000 rows by default, but my number of Chinese tags exceeded this limit. When attempting a full export, the system only offered Excel format and still split the data into multiple files, which did not meet my subsequent processing needs. Therefore, I had to export the data page by page in CSV format and finally manually merge all the paginated files into a single complete dataset.

Second, I also needed to export tags marked as Chinese but whose slugs do not contain Chinese characters. This part of the data actually consists of purely English tags under the Chinese language (for example, “alipay” also exists under the Chinese language due to the synchronization mechanism). As for mixed Chinese-English tags, their slugs inevitably contain Chinese encodings, so they were already included in the previous batch of data. Exporting this purely English tag data likewise required a page-by-page export and manual merging process.

After this tedious data extraction and integration, I finally obtained the complete tag dataset. I then took this data into the chat interfaces of various large language models for testing.

---

Phase 2: Large Language Model Test Performance

The testing logic was straightforward: ask the model to compare two lists of tags, map entries with identical semantics, and cluster synonymous tags that are purely Chinese. I did not use the file upload feature during testing; instead, I pasted the data text directly into the chat dialog.

1. DeepSeek: Context Limit Exceeded

I tested the DeepSeek model first. After pasting the data, the system immediately warned that the input length exceeded the limit by about 58%. The context window for a single conversation could not accommodate data of this scale, so the task was terminated right at the initial stage.

2. Doubao: Severe Hallucination Issues

Next, I tested the Doubao model. The model accepted the complete dataset and outputted results, but upon spot-checking, I found its output suffered from severe “hallucinations.” The model failed to perform accurate semantic matching and instead fabricated numerous non-existent tag aggregation relationships. For example, it incorrectly clustered multiple Chinese tags under a “谷歌play无法打开” tag that simply did not exist in the database. Such results, which appear well-formatted but are factually erroneous, would cause severe data pollution if applied directly to the production database.

3. Qwen: Service Overloaded

Given the poor performance of the previous models, I turned to testing Qwen. However, due to a high volume of concurrent visitors, the service was overloaded, and I was unable to proceed with actual testing.

4. ChatGPT: Partially Effective but Unable to Process in Full

Finally, I tested ChatGPT (GPT-4o). Its performance was relatively superior: when I pasted a large amount of data, the model automatically recognized the data scale and switched to file upload mode. Furthermore, after analyzing the data structure, it proactively suggested splitting the mixed data into two separate files: zh_tags.csv and en_tags.csv.

During the interaction, it did output a batch of high-confidence mapping results (such as “支付宝->alipay” and “队列->queue”). However, constrained by the context length and the computational overhead of handling complex logic, the processing stopped after completing a very small fraction, rendering it unable to process the full throughput of over 3,000 data entries.

---

Reflection and Solution Reconstruction: Limitations of Large Language Models

Based on the tests above, I can conclude: for tasks that require strict comparison of massive structured data with zero error tolerance, the probabilistic generation mechanism of large language models has fundamental flaws. Context window limitations, execution interruptions, and irresolvable hallucination issues all render them unsuitable for such batch-oriented, high-precision data processing work.

Therefore, I decided to abandon the LLM-based semantic matching approach and pivot to a deterministic engineering implementation:

1. Machine translation: Call the Baidu General Translation API to reliably convert Chinese tags into English.
2. Exact matching: Use a Go script to perform string-level exact matching between the translation results and the English tag database to generate mapping relationships.
3. Automated execution: Rely on the script to stably complete 3,000+ API calls and data processing operations without interruptions.

Unable to rely on large language models to clean up these thousands of multilingual tags, I had to return to code. Only by using a program to accurately map and merge Chinese tags into English tags can I eradicate those redundant Chinese URLs from the root and complete this deep SEO cleanup. In the next article, I will document how to implement this multilingual data cleaning logic using a Go script.

系列导航