Edit Precision
v0.12 result: Calor wins (1.36x across 217 paired programs)
What the calculator measures: Heuristic target identifiability and estimated collateral-edit risk. It does not run an AI agent or apply real edits.
Why Targeting Matters
An editing tool must find the intended declaration, locate its scope, and avoid changing similar code nearby. Calor provides two useful signals:
- explicit structural markers such as
§F,§L, and§IF; - optional source IDs that can distinguish otherwise similar structural targets.
§L{for1:i:1:100:1}
§IF{if1} (> i 50)
§P i
§L{for2:j:1:50:1}
§IF{if2} (< j 25)
§P j“Change for1 to start at 0” has a direct syntactic target. In C#, an
instruction such as “change the first loop” depends on location and surrounding
context.
How the Published Metric Is Calculated
The headline score combines two source-based heuristics:
| Component | Weight | What it does |
|---|---|---|
| Structural targeting | 40% | Scores explicit module/function IDs and an indented body; compares those signals with C# namespaces, classes, methods, and brace depth |
| Simulated task scoring | 60% | Synthesizes edit descriptions and assigns fixed scores from target presence, uniqueness, and source-pattern checks |
The task set covers adding a precondition, renaming a function, adding a parameter, changing a return type, and—when the detector finds one—changing a loop bound. These are calculator estimates, not observed task completions.
The structural Calor score starts at 0.50 and adds:
| Detected factor | Points |
|---|---|
| At least one explicit module ID | 0.15 |
| At least one explicit function ID | 0.20 |
| Indented body present | 0.10 |
Legacy §V pattern present | 0.10 |
That final detector is a known limitation: current variable bindings use §B,
not §V, so current source does not earn the legacy-variable bonus. Optional
name-first declarations also do not earn an explicit-ID bonus until IDs are
materialized in source.
The C# structural score starts at 0.50, adds namespace, class, and method signals, subtracts for deeper brace structure, and is capped at 0.85.
What 1.36x Does—and Does Not—Mean
The result means this deterministic calculator assigned higher targeting scores to the Calor representation in the published corpus. It is evidence about the calculator's explicit-structure hypothesis. It is not a measured 36% improvement in production edit success, an agent accuracy experiment, or proof that an ID survives a refactor automatically.
An explicit ID survives a rename or move only when the editing tool preserves it. Copies and extracted declarations need new IDs, and indentation—not closer tags—defines structural scope in v0.12.
For actual agent behavior, use a blinded task suite that performs edits, runs the compiler and tests, and measures unintended changes. Keep that evidence separate from this static metric.
Example Targeting Operations
Delete function f002 -> remove its §F opener and indented body
Rename function f001 -> change the name while preserving the explicit ID
Move loop for1 -> move its opener and indented body together
Add contract to f003 -> insert §Q in the function's contract sectionC# has strong IDE and Roslyn refactoring support, which this source-pattern calculator does not model. That is another reason not to interpret the ratio as a universal tooling comparison.
Next
- Benchmark Methodology - Corpus, calculators, and validity limits
- Stable Identifiers - Optional source IDs and preservation rules
- Refactoring Stability - The separate refactoring heuristic