From 92b20c54820dd6955d3127e3e4136f0b1b3fd4fc Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Sun, 8 Nov 2026 05:59:09 +0000
Subject: [PATCH 1/4] Write the base-2 entropy, cross-entropy and divergence
 with their macros

The fragments wrote them as H_2 and D_2, which reads as the Renyi entropy of
order 2 (collision entropy) used elsewhere on the site. Use \entropyf,
\crossentropyf and \kldivf, and say base-2 logarithms in the prose.
Closes #106.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9puSJHfSFxeRBZUfnt
---
 _subfiles/_exm-cross-entropy-decomposition.qmd |  6 +++---
 _subfiles/_rem-cross-entropy-loss.qmd          |  4 ++--
 _subfiles/_rem-cross-entropy-wrong-code.qmd    |  6 +++---
 _subfiles/_thm-cross-entropy-decomposition.qmd | 10 +++++-----
 _subfiles/_thm-kl-divergence-nonnegative.qmd   | 10 +++++-----
 _subfiles/_thm-source-coding.qmd               |  8 ++++----
 _subfiles/_thm-wrong-code.qmd                  |  6 +++---
 7 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/_subfiles/_exm-cross-entropy-decomposition.qmd b/_subfiles/_exm-cross-entropy-decomposition.qmd
index 0fc04ee9..b45013ef 100644
--- a/_subfiles/_exm-cross-entropy-decomposition.qmd
+++ b/_subfiles/_exm-cross-entropy-decomposition.qmd
@@ -21,9 +21,9 @@ c(
 )
 ```
 
-The cross-entropy is $H_2(p, q) = `r round(cross_entropy, 3)`$ bits.
-The entropy is $H_2(p) = `r shannon_entropy`$ bits,
-and the divergence is $D_2(p \,\|\, q) = `r round(divergence, 3)`$ bits.
+The cross-entropy is $\crossentropyf{p}{q} = `r round(cross_entropy, 3)`$ bits.
+The entropy is $\entropyf{p} = `r shannon_entropy`$ bits,
+and the divergence is $\kldivf{p}{q} = `r round(divergence, 3)`$ bits.
 They satisfy @thm-cross-entropy-decomposition:
 `r shannon_entropy` + `r round(divergence, 3)` = `r round(cross_entropy, 3)`.
 
diff --git a/_subfiles/_rem-cross-entropy-loss.qmd b/_subfiles/_rem-cross-entropy-loss.qmd
index c4ed0511..a10863f7 100644
--- a/_subfiles/_rem-cross-entropy-loss.qmd
+++ b/_subfiles/_rem-cross-entropy-loss.qmd
@@ -10,9 +10,9 @@ toward the distribution the labels came from.
 
 For a single labeled example,
 let $e$ be the distribution that puts all its mass on the observed label.
-Then $H_2(e) = 0$
+Then $\entropyf{e} = 0$
 ([as for a coin that always lands heads](https://morrison-lab.github.io/pds/expectation.html#exm-entropy-coin)),
-so the example's cross-entropy loss $H_2(e, q)$ equals $D_2(e \,\|\, q)$:
+so the example's cross-entropy loss $\crossentropyf{e}{q}$ equals $\kldivf{e}{q}$:
 every bit the loss reports is a wasted one.
 
 :::
diff --git a/_subfiles/_rem-cross-entropy-wrong-code.qmd b/_subfiles/_rem-cross-entropy-wrong-code.qmd
index dad7f9e4..70e355c1 100644
--- a/_subfiles/_rem-cross-entropy-wrong-code.qmd
+++ b/_subfiles/_rem-cross-entropy-wrong-code.qmd
@@ -2,13 +2,13 @@
 #### Cross-entropy is the cost of the wrong code
 
 @thm-cross-entropy-decomposition splits the [cross-entropy](https://morrison-lab.github.io/pds/expectation.html#def-cross-entropy) into the [entropy](https://morrison-lab.github.io/pds/expectation.html#def-entropy) and the divergence.
-By @thm-source-coding, no uniquely decodable code averages fewer than $H_2(p)$ bits.
+By @thm-source-coding, no uniquely decodable code averages fewer than $\entropyf{p}$ bits.
 By @thm-wrong-code, a code built for $q$
-costs about $D_2(p \,\|\, q)$ bits per value more than that.
+costs about $\kldivf{p}{q}$ bits per value more than that.
 By the [divergence theorem](entropy.qmd#thm-kl-divergence-nonnegative), the excess vanishes
 only when the code was built for $p$
 [@cover2006elements, section 2.6; @yeung2008information, section 2.5].
 @cover2006elements [section 5.4] state @thm-wrong-code
-in this form, with $H_2(p) + D_2(p \,\|\, q)$ in place of $H_2(p, q)$.
+in this form, with $\entropyf{p} + \kldivf{p}{q}$ in place of $\crossentropyf{p}{q}$.
 
 :::
diff --git a/_subfiles/_thm-cross-entropy-decomposition.qmd b/_subfiles/_thm-cross-entropy-decomposition.qmd
index 3dc46eb9..13d50bc0 100644
--- a/_subfiles/_thm-cross-entropy-decomposition.qmd
+++ b/_subfiles/_thm-cross-entropy-decomposition.qmd
@@ -3,13 +3,13 @@
 
 Let $p$ and $q$ be [probability mass functions](https://morrison-lab.github.io/pds/random-variables.html#def-pmf) on the values $x_1, \ldots, x_n$,
 with $q(x) > 0$ wherever $p(x) > 0$.
-Write $H_2(p)$ for the [entropy](https://morrison-lab.github.io/pds/expectation.html#def-entropy) of $p$,
-$H_2(p, q)$ for the [cross-entropy](https://morrison-lab.github.io/pds/expectation.html#def-cross-entropy) of $q$ relative to $p$,
-and $D_2(p \,\|\, q)$ for the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence) of $q$ from $p$,
+Write $\entropyf{p}$ for the [entropy](https://morrison-lab.github.io/pds/expectation.html#def-entropy) of $p$,
+$\crossentropyf{p}{q}$ for the [cross-entropy](https://morrison-lab.github.io/pds/expectation.html#def-cross-entropy) of $q$ relative to $p$,
+and $\kldivf{p}{q}$ for the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence) of $q$ from $p$,
 all with base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm).
 Then
 
-$$H_2(p, q) = H_2(p) + D_2(p \,\|\, q).$$
+$$\crossentropyf{p}{q} = \entropyf{p} + \kldivf{p}{q}.$$
 
 :::
 
@@ -30,6 +30,6 @@ $$
 = -\sum_x p(x) \log_2 p(x) + \sum_x p(x) \log_2 \frac{p(x)}{q(x)}.
 $$
 
-Reading left to right, these are $H_2(p, q)$, $H_2(p)$ and $D_2(p \,\|\, q)$.
+Reading left to right, these are $\crossentropyf{p}{q}$, $\entropyf{p}$ and $\kldivf{p}{q}$.
 
 :::
diff --git a/_subfiles/_thm-kl-divergence-nonnegative.qmd b/_subfiles/_thm-kl-divergence-nonnegative.qmd
index 88c9e213..065f1fee 100644
--- a/_subfiles/_thm-kl-divergence-nonnegative.qmd
+++ b/_subfiles/_thm-kl-divergence-nonnegative.qmd
@@ -7,11 +7,11 @@ and suppose that
 
 $$\sum_x p(x) \, \bigl\lvert \log_2 \bigl( p(x) / q(x) \bigr) \bigr\rvert < \infty.$$
 
-Then the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence) satisfies
+Then the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence), with base-2 logarithms, satisfies
 
-$$D_2(p \,\|\, q) \ge 0,$$
+$$\kldivf{p}{q} \ge 0,$$
 
-with $D_2(p \,\|\, q) = 0$ exactly when $q = p$.
+with $\kldivf{p}{q} = 0$ exactly when $q = p$.
 
 :::
 
@@ -38,9 +38,9 @@ $$
 \le 0.
 $$
 
-The left side is $-\ln 2 \cdot D_2(p \,\|\, q)$, so $D_2(p \,\|\, q) \ge 0$.
+The left side is $-\ln 2 \cdot \kldivf{p}{q}$, because the divergence uses base-2 logarithms, so $\kldivf{p}{q} \ge 0$.
 
-If $D_2(p \,\|\, q) = 0$, both relations hold with equality.
+If $\kldivf{p}{q} = 0$, both relations hold with equality.
 The first forces $q(x) = p(x)$ at every $x$ with $p(x) > 0$.
 The second then gives $\sum_x q(x) = 1$,
 so $q$ has no mass where $p$ has none, and $q = p$.
diff --git a/_subfiles/_thm-source-coding.qmd b/_subfiles/_thm-source-coding.qmd
index 15925cf4..9b6bcf84 100644
--- a/_subfiles/_thm-source-coding.qmd
+++ b/_subfiles/_thm-source-coding.qmd
@@ -3,18 +3,18 @@
 
 Let $X$ be a discrete random variable
 that takes finitely many values, at least two of them with positive probability,
-and let $H_2(X)$ be its
+and let $\entropyf{X}$ be its
 [entropy](https://morrison-lab.github.io/pds/expectation.html#def-entropy)
 with base-2 logarithms.
 
 1. Every [uniquely decodable](#def-uniquely-decodable) binary code for $X$
-   has [expected length](#def-expected-length) $L \ge H_2(X)$.
+   has [expected length](#def-expected-length) $L \ge \entropyf{X}$.
 
 2. Some uniquely decodable binary code for $X$ has expected length
-   $L < H_2(X) + 1$.
+   $L < \entropyf{X} + 1$.
 
 3. Coding blocks of $n$ independent copies of $X$ at a time,
    some uniquely decodable binary code spends, on average,
-   fewer than $H_2(X) + 1/n$ binary digits per copy.
+   fewer than $\entropyf{X} + 1/n$ binary digits per copy.
 
 :::
diff --git a/_subfiles/_thm-wrong-code.qmd b/_subfiles/_thm-wrong-code.qmd
index ed52e77a..0f3c6d54 100644
--- a/_subfiles/_thm-wrong-code.qmd
+++ b/_subfiles/_thm-wrong-code.qmd
@@ -8,9 +8,9 @@ Suppose a code gives each value $x$ with $p(x) > 0$
 the length $\ell(x) = \lceil -\log_2 q(x) \rceil$.
 Then its [expected length](#def-expected-length) $L$ under $p$ satisfies
 
-$$H_2(p, q) \le L < H_2(p, q) + 1,$$
+$$\crossentropyf{p}{q} \le L < \crossentropyf{p}{q} + 1,$$
 
-where $H_2(p, q)$ is the
+where $\crossentropyf{p}{q}$ is the
 [cross-entropy](https://morrison-lab.github.io/pds/expectation.html#def-cross-entropy)
 of $q$ relative to $p$, with base-2 logarithms.
 
@@ -32,6 +32,6 @@ $$
 < -\sum_{x} p(x) \log_2 q(x) + 1.
 $$
 
-The middle sum is $L$, and the outer sums are $H_2(p, q)$.
+The middle sum is $L$, and the outer sums are $\crossentropyf{p}{q}$.
 
 :::

From c157ae1010c0559282162e3163619af70d397228 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Sun, 8 Nov 2026 05:59:42 +0000
Subject: [PATCH 2/4] Link the terms the term-link check finds on the edited
 lines

Link bits, binary digits, uniquely decodable code, loss and logarithms
on the lines of the base-2 notation change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9puSJHfSFxeRBZUfnt
---
 _subfiles/_rem-cross-entropy-loss.qmd        | 2 +-
 _subfiles/_rem-cross-entropy-wrong-code.qmd  | 2 +-
 _subfiles/_thm-kl-divergence-nonnegative.qmd | 4 ++--
 _subfiles/_thm-source-coding.qmd             | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/_subfiles/_rem-cross-entropy-loss.qmd b/_subfiles/_rem-cross-entropy-loss.qmd
index a10863f7..6957fd53 100644
--- a/_subfiles/_rem-cross-entropy-loss.qmd
+++ b/_subfiles/_rem-cross-entropy-loss.qmd
@@ -12,7 +12,7 @@ For a single labeled example,
 let $e$ be the distribution that puts all its mass on the observed label.
 Then $\entropyf{e} = 0$
 ([as for a coin that always lands heads](https://morrison-lab.github.io/pds/expectation.html#exm-entropy-coin)),
-so the example's cross-entropy loss $\crossentropyf{e}{q}$ equals $\kldivf{e}{q}$:
+so the example's cross-entropy [loss](https://morrison-lab.github.io/pds/expectation.html#def-loss-function) $\crossentropyf{e}{q}$ equals $\kldivf{e}{q}$:
 every bit the loss reports is a wasted one.
 
 :::
diff --git a/_subfiles/_rem-cross-entropy-wrong-code.qmd b/_subfiles/_rem-cross-entropy-wrong-code.qmd
index 70e355c1..b93aa3c3 100644
--- a/_subfiles/_rem-cross-entropy-wrong-code.qmd
+++ b/_subfiles/_rem-cross-entropy-wrong-code.qmd
@@ -2,7 +2,7 @@
 #### Cross-entropy is the cost of the wrong code
 
 @thm-cross-entropy-decomposition splits the [cross-entropy](https://morrison-lab.github.io/pds/expectation.html#def-cross-entropy) into the [entropy](https://morrison-lab.github.io/pds/expectation.html#def-entropy) and the divergence.
-By @thm-source-coding, no uniquely decodable code averages fewer than $\entropyf{p}$ bits.
+By @thm-source-coding, no [uniquely decodable code](#def-uniquely-decodable) averages fewer than $\entropyf{p}$ [bits](entropy.qmd#def-bit).
 By @thm-wrong-code, a code built for $q$
 costs about $\kldivf{p}{q}$ bits per value more than that.
 By the [divergence theorem](entropy.qmd#thm-kl-divergence-nonnegative), the excess vanishes
diff --git a/_subfiles/_thm-kl-divergence-nonnegative.qmd b/_subfiles/_thm-kl-divergence-nonnegative.qmd
index 065f1fee..102e13cb 100644
--- a/_subfiles/_thm-kl-divergence-nonnegative.qmd
+++ b/_subfiles/_thm-kl-divergence-nonnegative.qmd
@@ -7,7 +7,7 @@ and suppose that
 
 $$\sum_x p(x) \, \bigl\lvert \log_2 \bigl( p(x) / q(x) \bigr) \bigr\rvert < \infty.$$
 
-Then the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence), with base-2 logarithms, satisfies
+Then the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence), with base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm), satisfies
 
 $$\kldivf{p}{q} \ge 0,$$
 
@@ -38,7 +38,7 @@ $$
 \le 0.
 $$
 
-The left side is $-\ln 2 \cdot \kldivf{p}{q}$, because the divergence uses base-2 logarithms, so $\kldivf{p}{q} \ge 0$.
+The left side is $-\ln 2 \cdot \kldivf{p}{q}$, because the divergence uses base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm), so $\kldivf{p}{q} \ge 0$.
 
 If $\kldivf{p}{q} = 0$, both relations hold with equality.
 The first forces $q(x) = p(x)$ at every $x$ with $p(x) > 0$.
diff --git a/_subfiles/_thm-source-coding.qmd b/_subfiles/_thm-source-coding.qmd
index 9b6bcf84..2072b2fc 100644
--- a/_subfiles/_thm-source-coding.qmd
+++ b/_subfiles/_thm-source-coding.qmd
@@ -15,6 +15,6 @@ with base-2 logarithms.
 
 3. Coding blocks of $n$ independent copies of $X$ at a time,
    some uniquely decodable binary code spends, on average,
-   fewer than $\entropyf{X} + 1/n$ binary digits per copy.
+   fewer than $\entropyf{X} + 1/n$ [binary digits](entropy.qmd#def-bit) per copy.
 
 :::

From eeb5ceca144e71773ac2041707598914dc97a60f Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Sun, 8 Nov 2026 07:01:55 +0000
Subject: [PATCH 3/4] Break the changed lines at clause boundaries

The Claude review found the base-2 proof line holding three clauses.
Break it, the statement line beside it, and the loss line at clauses.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9puSJHfSFxeRBZUfnt
---
 _subfiles/_rem-cross-entropy-loss.qmd        | 4 +++-
 _subfiles/_thm-kl-divergence-nonnegative.qmd | 8 ++++++--
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/_subfiles/_rem-cross-entropy-loss.qmd b/_subfiles/_rem-cross-entropy-loss.qmd
index 6957fd53..838dc57d 100644
--- a/_subfiles/_rem-cross-entropy-loss.qmd
+++ b/_subfiles/_rem-cross-entropy-loss.qmd
@@ -12,7 +12,9 @@ For a single labeled example,
 let $e$ be the distribution that puts all its mass on the observed label.
 Then $\entropyf{e} = 0$
 ([as for a coin that always lands heads](https://morrison-lab.github.io/pds/expectation.html#exm-entropy-coin)),
-so the example's cross-entropy [loss](https://morrison-lab.github.io/pds/expectation.html#def-loss-function) $\crossentropyf{e}{q}$ equals $\kldivf{e}{q}$:
+so the example's cross-entropy
+[loss](https://morrison-lab.github.io/pds/expectation.html#def-loss-function) $\crossentropyf{e}{q}$
+equals $\kldivf{e}{q}$:
 every bit the loss reports is a wasted one.
 
 :::
diff --git a/_subfiles/_thm-kl-divergence-nonnegative.qmd b/_subfiles/_thm-kl-divergence-nonnegative.qmd
index 102e13cb..19e5b7a7 100644
--- a/_subfiles/_thm-kl-divergence-nonnegative.qmd
+++ b/_subfiles/_thm-kl-divergence-nonnegative.qmd
@@ -7,7 +7,9 @@ and suppose that
 
 $$\sum_x p(x) \, \bigl\lvert \log_2 \bigl( p(x) / q(x) \bigr) \bigr\rvert < \infty.$$
 
-Then the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence), with base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm), satisfies
+Then the [Kullback--Leibler divergence](https://morrison-lab.github.io/pds/expectation.html#def-kl-divergence),
+with base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm),
+satisfies
 
 $$\kldivf{p}{q} \ge 0,$$
 
@@ -38,7 +40,9 @@ $$
 \le 0.
 $$
 
-The left side is $-\ln 2 \cdot \kldivf{p}{q}$, because the divergence uses base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm), so $\kldivf{p}{q} \ge 0$.
+The left side is $-\ln 2 \cdot \kldivf{p}{q}$,
+because the divergence uses base-2 [logarithms](https://morrison-lab.github.io/mds/algebra-exponentials.html#def-logarithm),
+so $\kldivf{p}{q} \ge 0$.
 
 If $\kldivf{p}{q} = 0$, both relations hold with equality.
 The first forces $q(x) = p(x)$ at every $x$ with $p(x) > 0$.

From 8ed259844ac335f57d8bdb312dbadeca43b0a134 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Sun, 8 Nov 2026 08:50:55 +0000
Subject: [PATCH 4/4] Break the merged line in the wrong-code proof at its
 clause boundary

The merge of main joined the two clauses on one line, which the
semantic line break rule does not allow.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9puSJHfSFxeRBZUfnt
---
 _subfiles/_thm-wrong-code.qmd | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/_subfiles/_thm-wrong-code.qmd b/_subfiles/_thm-wrong-code.qmd
index 6a9e6695..5b0815f8 100644
--- a/_subfiles/_thm-wrong-code.qmd
+++ b/_subfiles/_thm-wrong-code.qmd
@@ -32,6 +32,7 @@ $$
 < -\sum_{x} p(x) \log_2 q(x) + 1.
 $$
 
-The middle sum is $L$ by @thm-expected-length-sum, and the outer sums are $\crossentropyf{p}{q}$.
+The middle sum is $L$ by @thm-expected-length-sum,
+and the outer sums are $\crossentropyf{p}{q}$.
 
 :::