Information Density

Category: Information Density v0.12 result: C# narrowly wins (0.98x Calor/C# across 217 programs) What it measures: Semantic elements per token

Overview

The Information Density metric measures how much semantic content is packed into each token. Higher density means more meaning per token.

Why It Matters

Not all tokens carry equal semantic weight:

• public carries visibility information
• { carries only structural information
• §E{cw} carries effect information

Languages that pack more meaning into fewer tokens use context windows more efficiently.

How It's Measured

Semantic Elements Counted

Calor elements:

C# elements (via Roslyn):

Density Calculation

Density = TotalSemanticElements / TokenCount

Higher density = more semantic content per token.

Why C# Narrowly Leads

1. Implicit Information

C# encodes information implicitly:

public static int Add(int a, int b) => a + b;

This single line conveys:

• Visibility (public)
• Static binding
• Return type (int)
• Two parameters with types
• Expression body

Calor equivalent:

§F{f001:Add:pub}
  §I{i32:a}
  §I{i32:b}
  §O{i32}
  §R (+ a b)

Same semantics, but spread across more tokens.

2. Indent Form Eliminates Most Closing Tags

In v0.12 indent form, every structural block (§M{…}, §F{…}, §CL{…}, …) is terminated by dedent, so no structural closer is written. Current authored source uses only §/C for call argument lists and §/LAM for block lambdas.

C# closing braces (}) carry minimal semantic weight but are still counted as tokens.

3. Effect Information is "Extra"

Calor's effect declarations add semantic content that C# doesn't have:

§E{cw,fs:r,net:rw}    // 3+ semantic elements

But C# has no equivalent, so it doesn't get penalized for missing this information.

Detailed Breakdown

Example: FizzBuzz

Calor:

§M{m001:FizzBuzz}
  §F{f001:Main:pub}
    §O{void}
    §E{cw}
    §L{for1:i:1:100:1}
      §IF{if1} (== (% i 15) 0) → §P "FizzBuzz"
      §EI (== (% i 3) 0) → §P "Fizz"
      §EI (== (% i 5) 0) → §P "Buzz"
      §EL → §P i

Tokens: ~80 Density: 13/80 = 0.16

C#:

for (int i = 1; i <= 100; i++)
{
    if (i % 15 == 0) Console.WriteLine("FizzBuzz");
    else if (i % 3 == 0) Console.WriteLine("Fizz");
    else if (i % 5 == 0) Console.WriteLine("Buzz");
    else Console.WriteLine(i);
}

Tokens: ~50 Density: 17/50 = 0.34

Ratio: 0.16/0.34 = 0.47x

Sub-Metrics

The calculator provides detailed breakdowns:

Overall Density

Total semantic elements / total tokens

Type Density

Type annotations / total tokens

Contract Density

Contract elements / total tokens

Effect Density

Effect declarations / total tokens

Interpretation Nuance

The v0.12 ratio is 0.98x, so the aggregate difference is small. Its meaning still depends on what the calculator counts:

Calor's "Extra" Semantics

Calor includes semantic elements C# doesn't have:

• Explicit effects
• First-class contracts
• Optional explicit structural IDs

These are counted as semantic elements but have no C# equivalent.

Quality vs Quantity

Information density measures quantity of semantic content per token, not quality or usefulness.

Calor's explicit effects might be more valuable for agent reasoning than multiple C# type annotations, even if effects contribute fewer semantic elements.

When Density Matters Less

Low density is acceptable when:

1. Explicitness has value - Contracts, effects worth the tokens
2. Comprehension matters - Clear structure aids understanding
3. Context is available - Large context windows reduce pressure
4. Precision is needed - IDs enable precise targeting

The Tradeoff

This metric captures Calor's fundamental tradeoff:

Calor deliberately trades density for explicitness.

Improving Calor's Density

Potential improvements (with tradeoffs):

1. Shorter tags - §F could become @F (less distinctive)
2. Remove IDs - Skip unique identifiers (lose precision)

Note: Phase 4 already eliminated explicit closing tags via indent form (block boundaries are signalled by dedent), so the previous "implicit closing" suggestion has been realized.

Summary

The 0.98x result is the only C# win in the current eight-metric dashboard. It is a near-parity result for this specific semantic-elements-per-token definition, not a general measure of code quality.

Next

• Results - Summary of all metrics
• Tradeoffs - Understanding the design decisions