Faster Isn’t Always Better: Why I Reverted a Performance Optimization That Clearly Improved Speed

Recently, while working on Go Learning & Documentation v2, I spent a lot of time dealing with performance issues.

The reason is practical.

As the workflow became stricter, a single command might need to check all of the following simultaneously:

• current content package;
• historical evidence;
• source identity;
• candidate identity;
• glossary;
• Snapshot;
• Independent QC;
• package closure;
• Integrated Surface;
• currentness;
• dependency;
• SHA-256;
• stale detection.

These checks are all meaningful in their own right.

However, if the same deterministic data is repeatedly read, parsed, and validated within a single command, the execution time keeps growing.

Some low-frequency commands initially took several minutes.

This naturally leads to a goal:

Make it faster.

The problem is, as I got deeper into performance optimization, I gradually encountered a harder question to answer:

Exactly how fast is fast enough?

One round of optimization was particularly typical.

The baseline measurement for the same read-only planning path was about:

Time before optimization: 164.03 seconds

Time before optimization: 164.03 seconds

After a relatively conservative round of changes, it dropped to:

Time after optimization: 125.65 seconds
Time reduction: about 23.4%

Time after optimization: 125.65 seconds
Time reduction: about 23.4%

This was already a good result.

But later, when pushing a more aggressive approach, I managed to compress a similar path down to about:

Fastest measured: 35.93 seconds

Fastest measured: 35.93 seconds

Numerically, this looks much better.

The problem is:

In the end, I still reverted this optimization.

Because while the performance numbers were better, the proof of correctness did not hold up alongside them.

This post organizes why I ultimately chose:

To keep the less extreme but simpler optimization that is easier to prove correct.

Rather than treating “35 seconds” as a result I had to keep.

Initially, performance had indeed become a real problem

Early in this project, many commands had a relatively small workload.

For example, when only processing Tour, the official content scope was fairly stable.

Later, it expanded to:

• Site shell;
• Learn;
• Docs;
• structured content;
• Integrated Surface;
• Preview;
• future content packages;

After that, a command that looks like it merely:

“recalculates the current scope”

might now need to pass through many official authorities behind the scenes.

If every layer repeats the process of:

• reading files;
• parsing JSON;
• building a catalog;
• calculating SHA;
• validating historical reference;
• checking current state;

the time naturally adds up.

So performance optimization wasn’t for:

making benchmarks look good.

It was because maintenance work had already begun to suffer a real impact.

A low-frequency command that often requires manually waiting several minutes noticeably slows down the entire workflow.

But performance optimization shouldn’t start with “adding a cache”

Upon seeing repeated calculations, the most common approach that comes to mind is usually:

Add a cache.

For example:

• memoization;
• LRU;
• persistent cache;
• global cache;
• decoded object cache;
• historical validation cache.

From a pure performance perspective, these approaches are all tempting.

Because if:

the first validation is expensive

the first validation is expensive

then the most direct thought is:

do not validate a second time
→ directly reuse the first result

do not validate a second time
→ directly reuse the first result

The problem is that one of the core capabilities of this project is precisely:

Detecting that “now” and “just now” are no longer the same state.

That is:

• currentness;
• stale detection;
• dependency mutation detection;
• identity validation.

If the cache design isn’t strict enough, the very things it optimizes away most easily are these checks.

“Passed last time” cannot automatically become “still passes this time”

Assume a file just passed validation:

10:00: File SHA correct
10:00: validation PASS

10:00: File SHA correct
10:00: validation PASS

At 10:03, the command runs again.

If the cache logic is simply:

This path already PASSed before, reuse it directly.

Then if the file changed in between, it might go completely undetected.

For an ordinary tool, this might just mean the cache isn’t fresh enough.

But for the current workflow, this could mean:

stale content is treated as current.

Or:

a changed dependency continues to reuse old authorizations.

So formal currentness checks cannot be simplified to:

“I’ve seen this object before.”

What actually needs to be proven is:

The official authority I am reading again now still corresponds to the exact bytes that were proven.

So I later divided caching into two categories

The first category is relatively safe:

Within a single invocation, reusing deterministic transformations on exact bytes that have already been obtained.

For example, the same command has already read an identical set of bytes.

Next, it needs to:

• decode;
• parse;
• construct a purely deterministic object.

This kind of result can usually be reused within the current invocation.

Because it does not claim:

the disk hasn’t changed.

It only says:

For the same exact bytes that have already been read, there is no need to perform the same pure computation twice.

This is a crucial distinction.

The other type of cache carries much higher risk

The other type is:

Caching “what state the disk is in right now”.

For example, caching:

• a certain path is already current;
• a certain closure has already PASSed;
• a certain dependency has already been validated;
• a certain authority didn’t change last time;
• a certain Snapshot was still valid last time.

Once this kind of cache crosses invocations, things get much more complicated.

Because it must answer:

After caching, which dependencies might change?

After a change, who is responsible for invalidating the cache?

Are there indirect dependencies not included in the key?

What if the file content hasn’t changed, but the parser authority has?

What if the source hasn’t changed, but the semantic inputs have?

The earlier books.yaml 10→12 case actually already proved:

Dependencies are not just file bytes.

So cross-invocation cache invalidation is much harder than it initially seems.

The first thing to do should be profiling, not guessing

In this round of performance optimization, one thing I later appreciated was:

Measure first, then modify.

Don’t just assume a function is slow because its name looks complicated.

Instead, first obtain:

• wall-clock;
• CPU profile;
• actual invocation count;
• repeated call paths;
• real fixture.

Then look at:

where the time is actually being spent.

Because many steps that look “heavy” actually only execute once.

Conversely, some small-looking helpers might be called dozens of times repeatedly within the same graph.

If you don’t profile, it’s easy to optimize in the wrong direction.

First high-value target: eliminate obvious duplicate work

What’s truly worth prioritizing is usually not “all repeated calculations”.

Rather:

Obvious, high-frequency, duplicate work that accounts for a large proportion of total time.

For example, within the same command invocation:

build a complete validation graph once
→ subsequent stages rebuild the same graph again

build a complete validation graph once
→ subsequent stages rebuild the same graph again

In this case, if the input is still the same set of validated current bytes, it’s well suited for:

invocation-local reuse.

Because the payoff is large.

And the correctness contract is relatively easy to explain.

This type of optimization typically features:

Simple implementation, obvious payoff, clear invalidation boundaries.

This is the type of optimization I am most willing to keep later on.

Second high-value target: don’t repeatedly parse historical representations that definitely won’t change

The workflow stores a lot of historical evidence.

The reason historical evidence exists is precisely because:

they have become immutable history.

But “history is immutable” does not mean:

it has to be fully parsed all over again via different paths every time it passes through a helper.

If, within the same formal validation process:

• exact historical bytes have already been read;
• SHA has already been verified;
• reference identity has already been confirmed;

then subsequent pure transformations can reuse the already obtained results as much as possible.

The key point remains:

Reuse parsing results, rather than skipping authority validation.

This boundary is critical.

Third high-value target: do cheap rejections first

There is another very simple but effective optimization:

Adjusting predicate order.

Assume an object ultimately needs to check:

first check cheap exact mismatch
→ then check expensive historical graph
→ finally do full rebuild

first check cheap exact mismatch
→ then check expensive historical graph
→ finally do full rebuild

If the first step can already determine:

identity mismatch.

there is no need to continue with the expensive validations that follow.

Compared to building a complex cache, this type of adjustment usually has several advantages:

• simple code;
• correctness is easy to prove;
• introduces no new long-term state;
• adds no invalidation issues.

So I increasingly lean towards:

Looking for these simple gains first.

Why 35.93 seconds was ultimately reverted

Later, more aggressive optimizations pushed the time very low.

From:

164.03 seconds

164.03 seconds

to:

35.93 seconds

35.93 seconds

Looking purely at the numbers, it’s close to:

nearly an order of magnitude faster.

If this were an ordinary benchmark, it would be easy to think:

This should definitely be merged.

But a formal workflow cannot look only at benchmarks.

It must also answer:

Is the output exactly the same?

Has currentness been weakened?

Can dependency mutations still be detected?

Are stale inputs still rejected?

Do targeted negative tests still fail?

Is historical integrity still intact?

If any one of these key questions cannot be answered:

no matter how fast it is, it cannot be called a PASS.

A targeted test FAILED might be more important than being a minute faster

The most dangerous thing about performance optimization is:

the happy path is usually the easiest to speed up.

Because normal inputs are typically:

• current;
• complete;
• free of mutation;
• free of stale;
• free of concurrent changes.

Caches can easily perform beautifully on these paths.

But what’s really important is the negative path.

For example:

dependency has changed.

source identity mismatch.

prior evidence stale.

historical bytes modified.

current file changes between two checks.

In these cases, the system must reject.

If, after optimization:

normal fixture: PASS, and exceptionally fast
abnormal fixture: originally should FAIL, but now PASSes

normal fixture: PASS, and exceptionally fast
abnormal fixture: originally should FAIL, but now PASSes

Then this is not:

a high-performance implementation with a minor bug.

Rather, it is:

the correctness contract has been broken.

So this type of optimization must be reverted.

Faster, but the cost of proof might also be much higher

There is another rather subtle situation.

The optimization itself isn’t necessarily proven wrong.

But it introduces a very heavy proof burden.

For instance, a multi-level cache might need to maintain:

cache key
→ source identity
→ parser identity
→ glossary identity
→ authorization identity
→ dependency graph identity
→ historical reference identity
→ mutation-sensitive invalidation

cache key
→ source identity
→ parser identity
→ glossary identity
→ authorization identity
→ dependency graph identity
→ historical reference identity
→ mutation-sensitive invalidation

At this point, what really needs to be evaluated is not:

how many more seconds can it save?

Rather:

How much code and how many tests need to be added to prove that this optimization of a few dozen seconds will never return a stale state?

If the answer is:

introducing a whole new cache architecture.

Then for a low-frequency deterministic workflow, it’s likely no longer worth the cost.

Performance gains also have diminishing marginal returns

Assume a command originally took:

about 10 minutes

about 10 minutes

Optimized to:

about 2 minutes

about 2 minutes

This is usually very worthwhile.

Maintainers can clearly feel the difference every time they run it.

But if you then go from:

2 minutes

2 minutes

continuing to optimize down to:

1 minute 30 seconds

1 minute 30 seconds

the value needs to be re-evaluated.

If this 30 seconds requires:

• lots of caching code;
• complex invalidation protocols;
• new concurrency considerations;
• more tests;
• higher maintenance costs;

then:

not optimizing might actually be the better engineering decision.

“Can still be optimized” does not equal “worth optimizing”

This is an important principle I later added to the project.

Performance tasks should not use:

REMAINING_DUPLICATE_WORK=none

REMAINING_DUPLICATE_WORK=none

as the stopping condition.

Because in reality, you can almost always find duplicate work.

A much more reasonable stopping condition is:

REMAINING_HIGH_VALUE_OPTIMIZATION=none

REMAINING_HIGH_VALUE_OPTIMIZATION=none

That is to say:

not “there is no duplicate computation left at all”.

Rather:

there are no high-value bottlenecks left that are worth spending engineering costs to address.

These two criteria are completely different.

Some low-yield duplicate computations should be allowed to exist

This point is a bit counterintuitive at first.

Since we already know:

a certain step will be executed repeatedly.

Why not eliminate it?

Because eliminating duplicate computation itself also carries a cost.

Assume a repeated check only takes:

100 milliseconds

100 milliseconds

but to avoid it, you need to:

• add a new cache;
• manage the lifecycle;
• add invalidation;
• add more identity;
• write a dozen tests.

Then keeping this 100 milliseconds might actually be the cheapest and safest solution.

So:

Duplicate computation is not inherently a bug.

It is only worth optimizing when it causes actual high costs and can be safely eliminated in a simple way.

Complex caches also increase future modification costs

There is another easily underestimated issue with performance optimization:

future maintenance.

Adding an LRU today might look like just a few dozen lines of code.

But later, whenever a new formal dependency is added, you must continue to answer:

Does it go into the cache key?

Does the old cache invalidate automatically?

Is the historical reader affected?

Will the current checker still re-read the disk?

If a dependency is missed, the system might not immediately throw an error.

Instead:

it silently returns stale results.

These types of errors are often harder to spot than “running a bit slow”.

So I increasingly lean towards:

Preferring to compute once more rather than maintaining an additional set of long-term state that is hard to prove complete.

Why invocation-local reuse is more suitable for this type of project

Compared to persistent caches, I prefer reuse within the scope of a single invocation.

The reason is simple.

Its lifecycle is very clear:

command starts
→ reads current exact bytes
→ completes formal currentness check
→ reuses deterministic results within this invocation
→ command ends
→ reuse state disappears

command starts
→ reads current exact bytes
→ completes formal currentness check
→ reuses deterministic results within this invocation
→ command ends
→ reuse state disappears

The next command:

starts over from disk authority.

This way, you won’t encounter:

a cache built yesterday still thinking it is current today.

This aligns perfectly with the semantics of a formal workflow.

export and current-check cannot be merged into a single proof just for speed

There is another very important example.

The Generation Bundle’s:

export

and:

current-check

Although both might perform many of the same calculations, they are two independent invocations.

One of the points of current-check is:

re-running the canonical exporter to confirm that inputs haven’t changed since the export.

If, for performance, this is changed to:

it was already calculated during export, so the check just reuses it.

Then you’ve essentially optimized away:

freshness proof

along with it.

So the formal contract explicitly requires:

the memo scope of the two invocations must be separate.

Performance tests also cannot secretly share the same memo scope and then conclude:

current-check is very fast.

Because this kind of benchmark does not correspond to the real formal process.

The benchmark itself might also be “cheating”

There is also a very practical issue with performance testing.

If the benchmark uses:

• warm cache;
• old parsed objects;
• the current result from the last invocation;
• an already constructed graph;

then the numbers might look fantastic.

But the formal execution does not have these preconditions.

So performance tests must try to answer:

Does this measurement really represent the command the user will execute?

Rather than:

How fast is some internal function under ideal cache conditions?

Real fixtures are important.

Fixed targeted tests are also important.

Why I didn’t keep pursuing 35 seconds

When a workflow that was originally very slow has already been reduced to:

an acceptable range of about a minute or two.

The nature of the gains from further optimization has already changed.

The previous gain was:

eliminating obvious waiting.

Subsequent gains look more and more like:

continuing to squeeze the benchmark for better numbers.

If this requires:

• typed memo;
• LRU;
• multi-level cache;
• persistent state;
• architecture refactor;

then it has begun to cross another boundary:

performance complexity might exceed performance gains.

Stopping at this point is not:

an optimization failure.

Instead, it might indicate:

the engineering goal has been achieved.

Codex quota is itself also part of performance cost

There is another issue that might not have been considered much before:

implementing optimizations also consumes resources.

Especially when heavily using Codex for repository engineering, a complex cache scheme might require:

• design;
• modification;
• targeted tests;
• failure diagnosis;
• rollback;
• re-implementation;
• re-validation.

All of these consume the 5h quota.

If the remaining project still has:

• correctness blocker;
• Activation;
• Preview;
• Production;

then continuing to spend a large amount of quota just to:

save a few dozen more seconds.

carries a high opportunity cost.

So the cost-effectiveness of performance optimization cannot look only at:

runtime saved

runtime saved

It should also consider:

runtime saved
versus
implementation cost
versus
verification cost
versus
regression risk
versus
Codex quota

runtime saved
versus
implementation cost
versus
verification cost
versus
regression risk
versus
Codex quota

I later set a more explicit stopping condition for performance optimization

My current line of thinking is closer to:

First, ask:

Is this a high-frequency or low-frequency command?

Then ask:

Does the current execution time already affect actual work?

Then ask:

Does the profile still show an obvious bottleneck?

Then ask:

Is the next optimization simple, local, and easy to prove?

If the answers gradually become:

low-frequency;

already acceptable;

remaining bottlenecks are scattered;

requires a complex cache to drop significantly further;

Then you should stop.

That is:

Performance tasks do not need to push execution time to the theoretical minimum.

Reverting an optimization does not mean it was wasted effort

One thing that stood out to me in this round is:

An optimization ultimately being reverted does not mean it has no value at all.

Because it at least answered:

Can this direction be safely pushed further?

For instance, the 35.93 second result proved:

there is indeed a lot of theoretical headroom for performance.

But the corresponding tests and proofs told me:

this particular implementation approach is not worth keeping.

This is actually quite valuable information.

Compared to forcibly keeping a risky optimization in the repository, reverting it still allows you to retain:

• profile;
• benchmark;
• failed test evidence;
• an understanding of cache boundaries;
• a clearer performance policy.

This is more valuable than an implementation that is “very fast but unclear if safe”.

Every optimization should be individually revertible

I also increasingly prefer transactional progression later on.

Do not do all at once:

Optimization A
→ Optimization B
→ Optimization C
→ Final unified testing

Optimization A
→ Optimization B
→ Optimization C
→ Final unified testing

Because if it ultimately fails, it’s hard to know:

exactly which change broke the contract?

A better approach is:

Complete Optimization A
→ Run targeted tests
→ Keep if PASS

Complete Optimization B
→ Run targeted tests
→ FAILED
→ Precisely revert B's own changes

Complete Optimization A
→ Run targeted tests
→ Keep if PASS

Complete Optimization B
→ Run targeted tests
→ FAILED
→ Precisely revert B's own changes

This way, you won’t end up with a worktree piled with unproven optimizations just because you were chasing speed.

“Identical output after optimization” is still not necessarily enough

There is another easily overlooked issue in performance optimization.

Even if the happy-path outputs JSON where:

before == after

before == after

it might not be enough.

Because you must also test:

If a dependency changes in the middle, will the new implementation still detect it?

That is, you must prove both:

Normal input results have not changed.

And also prove:

Inputs that should fail still fail.

This is why negative fixtures are particularly important for performance optimization.

A correctness baseline must be established earlier than a benchmark

The ideal optimization order should be:

First fix the correctness baseline
→ Then record before timing
→ Profile to find bottlenecks
→ Make one modification
→ Run fixed targeted tests
→ Re-measure timing after PASS

First fix the correctness baseline
→ Then record before timing
→ Profile to find bottlenecks
→ Make one modification
→ Run fixed targeted tests
→ Re-measure timing after PASS

Rather than:

First change a lot of code
→ See that it's very fast
→ Then try to find a way to prove it's fine

First change a lot of code
→ See that it's very fast
→ Then try to find a way to prove it's fine

The latter easily introduces psychological bias.

Because after seeing:

35 seconds

people will naturally want to:

Can we tweak the tests to keep this?

This is exactly the most dangerous direction.

The correct order should be the reverse:

Test contracts do not make way for optimizations.

Faster, but tests FAILED, cannot be called a performance PASS

So the criteria I now use for performance tasks are actually quite simple.

For example:

Execution time: significantly decreased
targeted correctness tests: FAILED

Execution time: significantly decreased
targeted correctness tests: FAILED

The final conclusion can still only be:

RESULT=FAILED

RESULT=FAILED

It cannot be written as:

Performance optimization succeeded, but there are a few tests that need to be handled.

Because if the tests cover formal correctness invariants, then this optimization has not yet been established.

This is actually the same mindset as the entire translation workflow

I later realized that the principles of performance optimization are very similar to the translation lifecycle.

In translation, you cannot skip machine validation and Independent QC just because:

the translation looks good.

In performance, you also cannot skip correctness proof just because:

the benchmark looks fantastic.

The shared principle for both is actually:

You cannot substitute surface results for formal proof.

Shifting from “fastest” to “high-value”

If you only pursue the fastest, the goal might be:

find all duplicate work
→ eliminate it all
→ push benchmark as close to the theoretical minimum as possible

find all duplicate work
→ eliminate it all
→ push benchmark as close to the theoretical minimum as possible

But I now prefer to write the goal as:

identify high-cost bottlenecks
→ use simple solutions to reduce obvious duplication
→ maintain correctness and freshness
→ reach an actually acceptable execution time
→ stop

identify high-cost bottlenecks
→ use simple solutions to reduce obvious duplication
→ maintain correctness and freshness
→ reach an actually acceptable execution time
→ stop

The final judgment is not:

can it still be faster.

Rather:

Is the next second of performance gain still worth the corresponding engineering cost and risk.

A few principles I now agree with more

After these rounds of optimization, I think the performance strategy can be condensed into a few rules.

First:

Profile first, then optimize; don’t guess bottlenecks based on feeling.

Second:

Prioritize obvious, high-yield duplicate computations; do not try to eliminate all duplicate work.

Third:

Prefer invocation-local reuse; cautiously introduce persistent cache, LRU, and complex memoization.

Fourth:

No optimization may degrade correctness, freshness, stale detection, identity, hash validation, dependency mutation detection, and fail-closed.

Fifth:

Independent invocations like export and current-check, which are inherently meant to re-prove freshness, cannot share old PASSes due to performance optimization.

Sixth:

Performance tests must not only test the happy path, but also retain negative fixtures like stale, mutation, and invalid identity.

Seventh:

Promote each optimization as an independent commit; if targeted tests FAIL, precisely revert that specific optimization.

Eighth:

After a low-frequency workflow has been reduced from several minutes to a minute or two, saving a few dozen more seconds is usually not worth introducing a complex cache architecture.

Ninth:

Codex quota, implementation time, verification costs, and future maintenance burdens are also part of performance optimization costs.

Tenth:

The true completion criteria for a performance task is not having zero duplicate computation, but having no remaining high-value optimizations.

That is:

REMAINING_HIGH_VALUE_OPTIMIZATION=none

REMAINING_HIGH_VALUE_OPTIMIZATION=none

Ultimately, the one sentence I am now most willing to keep is:

The fastest implementation is not necessarily the cheapest implementation; and the cheapest implementation is not just about runtime.

For workflows that emphasize currentness, historical evidence, and recoverability:

Stably reducing the time from 10 minutes to 2 minutes is usually more valuable than risking failure detection to chase 30 seconds.