Skip to content

fix(converter): keep <br> as a hard break in html → markdown lists and tables - #271

Merged
pchuri merged 2 commits into
mainfrom
fix/html-br-hard-break
Oct 4, 2026
Merged

pchuri merged 2 commits into
mainfrom
fix/html-br-hard-break

Conversation

@pchuri

@pchuri pchuri commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Problem

After #248, storage → markdown keeps <br/> inside list items as a backslash hard break and inside table cells as an inline <br>. html → markdown (lib/html-to-markdown.js, used by convert --input-format html) still turned both into a space, so the two converters disagreed on the same markup (the kind of drift #149 was about).

htmlToMarkdown('<ul><li>line1<br/>line2</li></ul>')
// before: "- line1 line2"
// after:  "- line1\\\n  line2"

htmlToMarkdown('<table><tbody><tr><td>line one<br/>line two</td></tr></tbody></table>')
// before: "| line one line two |\n| --- |"
// after:  "| line one<br>line two |\n| --- |"

Changes

  • The walker's hard-break helpers moved into lib/markdown-cleanup.js so both converters share them: collapseInline, escapeContinuationLine, and resolveHardBreaks (joins lines with a backslash break and doubles an odd run of trailing backslashes). StorageWalker now calls them; its behavior is unchanged and all of its existing tests pass untouched.
  • html → markdown list items: each <br> is marked with the existing HARD_BREAK sentinel (outside code spans, where a break cannot live and stays a space), whitespace is collapsed around it, and it is resolved per item. Continuation lines are indented to the item's content column by the existing list rendering, and lines that would start a block (- , 1. , #, >, ---, fences, table delimiter rows) are escaped.
  • html → markdown table cells: a break is written as an inline <br>. The final tag-stripping pass would remove a literal <br>, so the cell text carries &lt;br&gt;, which the entity pass turns back into <br>.
  • Character references: html → markdown decodes entities only after list items are rendered, so the break handling must not look at undecoded text. The converter's entity pass is now one decodeEntities function, and resolveHardBreaks takes decode/encode hooks: the block-opener and trailing-backslash checks look at the decoded line (so -&#32;b, ---&#45;, &amp;#35; b or a backslash written as &#92; are handled), a line that needs escaping is written back with & and < re-encoded (the passes before the final decode would otherwise treat them as the start of a reference or a tag), and every other line is kept exactly as written. &nbsp; lines next to a break are dropped before the collapse, and only when a break is present, so text without breaks is untouched.
  • <BR> counts as a break in lists and cells, since HTML tag names are case-insensitive (the walker's XML parser is case-sensitive and does not).
  • Top-level <br/> is still a soft break in both converters (out of scope, as noted in the issue).

Behavior notes

  • Only html that has a <br> inside a list item or table cell produces different output; everything else is unchanged.
  • Paragraphs in a list item are still merged into one line, as before; a break inside them is kept.

Testing

  • Unit tests for the shared helpers.
  • html → markdown tests for the two repros, marker widths (10.), nesting, whitespace and consecutive breaks, edge breaks, trailing backslashes (literal and as character references), the full set of block-start lines (including ones written as character references), a literal fence line, code spans, table cells, and a literal U+E003.
  • Parity tests run 43 markups from the walker's <br/> tests, plus character-reference and &nbsp; variants, through both htmlToMarkdown and storageToMarkdown and expect identical output.
  • A differential fuzz of 6000 generated documents without a break in a list or cell shows byte-identical output to main.
  • npm test (40 suites, 1730 tests) and eslint pass.

Not covered here: inline <code> containing backticks, which html → markdown wraps in single backticks regardless of content (an existing limitation unrelated to breaks).

Fixes #253

pchuri added 2 commits October 4, 2026 21:35
…d tables

html → markdown turned <br/> in list items and table cells into a space,
while storage → markdown has kept them since #248 (a backslash hard
break in list items, an inline <br> in cells), so the converters
disagreed on the same markup.

Move the walker's hard-break helpers (collapseInline,
escapeContinuationLine and the trailing-backslash handling, now
resolveHardBreaks) into markdown-cleanup so both converters share them.
In html → markdown, mark each <br> with HARD_BREAK outside code spans,
resolve it per list item, and write it as an inline <br> in table cells.
Backslash and block-opener character references are decoded where they
decide how a break is escaped, since entities are otherwise decoded after
list items are rendered. Top-level <br> stays a soft break.

Fixes #253
html → markdown decodes entities only after list items are rendered, so
the break handling looked at undecoded text. A continuation line that
only decodes to a block opener (`-&#32;b`, `---&#45;`, `&amp;#35; b`)
was not escaped, a reference to a sentinel codepoint could become a live
sentinel, and `&nbsp;` next to a break left a stray backslash.

Extract the converter's entity pass into one decodeEntities function and
let resolveHardBreaks take decode/encode hooks: the opener and
trailing-backslash checks look at the decoded line, a line that needs
escaping is written back with & and < re-encoded, and other lines are
kept as written. Drop &nbsp; lines next to a break before the collapse,
only when a break is present. Treat <BR> as a break, as HTML tag names
are case-insensitive.
@pchuri
pchuri merged commit d853acf into main Oct 4, 2026
6 checks passed
@pchuri
pchuri deleted the fix/html-br-hard-break branch October 4, 2026 14:22
github-actions Bot pushed a commit that referenced this pull request Oct 4, 2026
## [2.27.3](v2.27.2...v2.27.3) (2026-10-04)

### Bug Fixes

* **converter:** keep <br> as a hard break in html → markdown lists and tables ([#271](#271)) ([d853acf](d853acf)), closes [#248](#248) [#253](#253)
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 2.27.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

html → markdown turns <br/> in list items and table cells into a space (diverges from storage → markdown)

1 participant