fiatcode/weft
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions

perf(graph): single-pass index parse + unpinned backlink context & release v2.2.2 #25

Merged
fiatcode merged 2 commits from perf/index-build-cleanup into main 2026-06-18 11:10:51 +07:00
Conversation 0 Commits 2 Files changed 4 +106 -54
fiatcode commented 2026-06-18 11:09:29 +07:00
Owner
Copy link

Two behavior-preserving index-build optimizations (cut v2.2.2).

Single-pass parse. BuildIndex's second pass split each page into lines 3× — once for context lookup, once in ExtractWikiLinks, once in ExtractTodos — and re-walked the fence state in each extractor. Folded into one parseBody that walks the lines once and returns (lines, links, todos); BuildIndex reuses the returned lines for context. Added a strings.Contains(line, "[[") fast-path so link-free lines skip the backtick split + regex. ExtractWikiLinks/ExtractTodos kept as thin wrappers — public API and tests unchanged.

Unpinned context. strings.Split lines are substrings sharing the whole body's backing array, so a retained Ref.Context pinned the entire file body in memory for the life of the index. lineContext now strings.Clones the line so it owns its bytes. Pinned by a new TestLineContextDoesNotPinFileBody (watched fail → pass).

Full suite + vet + gofmt green.

Two behavior-preserving index-build optimizations (cut v2.2.2). **Single-pass parse.** `BuildIndex`'s second pass split each page into lines 3× — once for context lookup, once in `ExtractWikiLinks`, once in `ExtractTodos` — and re-walked the fence state in each extractor. Folded into one `parseBody` that walks the lines once and returns `(lines, links, todos)`; `BuildIndex` reuses the returned lines for context. Added a `strings.Contains(line, "[[")` fast-path so link-free lines skip the backtick split + regex. `ExtractWikiLinks`/`ExtractTodos` kept as thin wrappers — public API and tests unchanged. **Unpinned context.** `strings.Split` lines are substrings sharing the whole body's backing array, so a retained `Ref.Context` pinned the entire file body in memory for the life of the index. `lineContext` now `strings.Clone`s the line so it owns its bytes. Pinned by a new `TestLineContextDoesNotPinFileBody` (watched fail → pass). Full suite + vet + gofmt green.
BuildIndex's second pass split each page into lines three times — once for
context lookup, once inside ExtractWikiLinks, once inside ExtractTodos — and
re-walked the fence state in each extractor. Fold that into one parseBody that
walks the lines once and returns (lines, links, todos); BuildIndex reuses the
returned lines for context. Add a strings.Contains(line, "[[") fast-path so
link-free lines skip the backtick split + regex.

Also clone the line in lineContext: strings.Split lines are substrings sharing
the whole body's backing array, so a retained Ref.Context pinned the entire
file body in memory for the life of the index. Cloning detaches it.

ExtractWikiLinks/ExtractTodos kept as thin wrappers over parseBody — public API
and tests unchanged. Behavior-preserving; existing suite stays green.
docs(changelog): v2.2.2
All checks were successful
CI / vet-and-test (pull_request) Successful in 25s
Details
e5ad1b7b19
fiatcode changed title from perf(graph): single-pass index parse + unpinned backlink context to perf(graph): single-pass index parse + unpinned backlink context & release v2.2.2 2026-06-18 13:44:26 +07:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
fiatcode
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
fiatcode/weft!25
No description provided.