Repository navigation
fix(mdxish): terminate html flow block when opener has trailing text - #1495
eaglethrost merged 12 commits into
Conversation
993085e to
13742c1
Compare
13742c1 to
993085e
Compare
|
|
||
| // Tag at start of line + trailing text, no closer (e.g. `<li> test`). CommonMark | ||
| // still opens an HTML block here, so we terminate it. | ||
| const HTML_LINE_WITH_TRAILING_CONTENT_REGEX = /^(<[a-z][^<>]*>|<\/[a-z][^<>]*>)\s*\S/; |
There was a problem hiding this comment.
Should this be gated on the type 6 tag list? li is on it so <li> test is right, but the regex here matches any lowercase tag with trailing text, and type 6 only opens for listed names. So <b> bold text, <your-api-key> replace this and <https://example.com> is our docs now get a blank line and split into two paragraphs. I think we already have the list, magic-block-transformer.ts:155 builds BLOCK_LEVEL_TAGS from htmlBlockNames for the same decision. Could we just capture the name and check it?
const m = line.match(/^<\/?([a-z][a-z0-9-]*)(?=[\s/>])[^<>]*>\s*\S/);
if (m && htmlBlockNames.includes(m[1])) return true;Would you mind adding a negative test for <b> bold text followed by a plain line? The existing fixtures all still pass with the new regex, so nothing pins this.
There was a problem hiding this comment.
Done. The check now only applies to htmlBlockNames, and it also skips tags whose closer appears later in the same block. Added the tests as well!
| @@ -6,6 +6,10 @@ const STANDALONE_HTML_LINE_REGEX = /^(<[a-z][^<>]*>|<\/[a-z][^<>]*>)+\s*$/; | |||
|
|
|||
| const HTML_LINE_WITH_CONTENT_REGEX = /^<[a-z][^<>]*>.*<\/[a-z][^<>]*>(?:[^<]*)$/; | |||
There was a problem hiding this comment.
Small one, but I think HTML_LINE_WITH_CONTENT_REGEX is dead now, anything it matches also matches the new regex below.
There was a problem hiding this comment.
We should keep it because the regex still catches lines like <span>a</span> b
…blocks-right-under-an-opening-html-tag-that-has-text # Conflicts: # __tests__/transformers/terminate-html-flow-blocks.test.ts # processor/transform/mdxish/terminate-html-flow-blocks.ts
jamestclark
left a comment
There was a problem hiding this comment.
Thanks for picking this up! The ticket's case renders now and the fixture is really helpful. Left two comments, I think both are worth sorting out before merge.
| * Per line, the last line index of each closing tag name within that line's | ||
| * blank-line-delimited run (one shared map per run). | ||
| */ | ||
| function mapClosingTagsByRun(lines: string[]): Map<string, number>[] { |
There was a problem hiding this comment.
Does this cover sibling <li>s where only the last one is closed? With something like
<li> First
[block:callout]
{"type":"info","body":"x"}
[/block]
<li> Second</li>
mapClosingTagsByRun sees the </li> on the last line, so no blank line goes in. I don't think anything re-parses that block though, the tokenizer ends at depth 1 (syntax.ts:478,511) and findBalancedClosingTagIndex returns -1, so the callout still renders as text. Same for nested <div>s or a </div> inside an attribute value. Legacy lists drop </li> pretty often, so I think this one's worth catching.
Could we just reuse the balance check the re-parse already does? Then this map and CLOSING_TAG_NAME_REGEX can go:
const closed = findBalancedClosingTagIndex(restOfRun, tagName) >= 0;Would you mind adding the sibling <li> case to the fixture too?
There was a problem hiding this comment.
Quite an edge edge but yeah not covered, will do so. Added test in terminate-html-flow-blocks.test.ts:302
| const rawContentFacts = { | ||
| const boundaryFacts = { | ||
| insideRawContent: rawContentDepths.some(depth => depth > 0), | ||
| leavesBlockOpen: leavesBlockTagOpen(line, i, closersByRun[i]), |
There was a problem hiding this comment.
I think this reaches into ~~~ fences now. protectCodeBlocks only masks backtick fences, so
~~~html
<li> test
next line
~~~
gets a blank line inside the code sample, where next leaves it alone. Should tilde fences be protected in protectCodeBlocks itself? tables/tag-walker.ts:8 already masks them, and lone tags like ~~~\n<div>\nnext\n~~~ hit this on next too, so that'd cover both. Would you mind adding a test that the example above comes back unchanged?
There was a problem hiding this comment.
Yep thanks for the catch, fixed! Added it in protect-code-blocks.ts:12
jamestclark
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround! The sibling <li> fix looks great, and I like that findBalancedOpenerOffsets gets checked against findBalancedClosingTagIndex in a test. The ~~~ change has a problem though: code blocks that used to be protected aren't anymore, so I've left a comment there. I think that one needs fixing before merge.
| const FENCE_MARKERS = ['```', '~~~'] as const; | ||
|
|
||
| /** Earliest backtick or tilde fence marker in `text`, or null if there is none. */ | ||
| function findNextFence(text: string) { |
There was a problem hiding this comment.
I think findNextFence is a bit too eager, it matches ~~~ anywhere in the text, not just at the start of a line. So with
Use `~~~` to open a tilde fence.
```html
<div>
next
```
there's no second ~~~, so the loop breaks at :54 and the ```html block never gets protected. It did before this change, and now terminateHtmlFlowBlocks puts a blank line inside the sample. If prose mentions ~~~ twice, everything between the two mentions gets masked instead. This also runs before removeJSXComments in mdxish() (lib/mdxish.ts:291), so I think it affects the whole pipeline, not just this transform.
Could we match fences line by line, where the closer uses the same character and is at least as long as the opener? Something like
const FENCE_OPEN_REGEX = /^ {0,3}(`{3,}|~{3,})/;
// closer: same char, length >= opener, alone on its lineI think that also fixes ~~~~~~ fences closing on their own opener. indexOf(marker, 3) at :53 finds a match inside the six tildes, so the body stays unprotected. Backtick fences have had the same bug for a while. It would also drop the indexOf('~~~') that scans the whole rest of the doc after every block. Would you mind adding tests for an inline `~~~` before a backtick fence, and a 6-tilde fence?
There was a problem hiding this comment.
Ah yeah that's right, sorry about that 🙏 protectCodeBlocks now matches fences line by line
| ['<your-api-key> replace this', 'plain line'], | ||
| ['<https://example.com> is our docs', 'plain line'], | ||
| ['<lit> test', 'plain line'], | ||
| [' <li> test', '[block:parameters]'], |
There was a problem hiding this comment.
Not blocking, just curious. The QA snippet in the description starts with <li> test (one space), and this test checks that exact shape doesn't get a blank line. Is that still the expected result for the ticket's doc, or was the space just from pasting?
…blocks-right-under-an-opening-html-tag-that-has-text
…pening-html-tag-that-has-text
…pening-html-tag-that-has-text
…pening-html-tag-that-has-text
## Version 16.2.4 ### 🛠 Fixes & Updates * **mdxish:** keep html comments syntax as written when serializing ([#1692](#1692)) ([71c24b4](71c24b4)), closes [#1686](#1686) * **mdxish:** render footnote refs in table ([#1494](#1494)) ([c41296f](c41296f)) * **mdxish:** terminate html flow block when opener has trailing text ([#1495](#1495)) ([a273c3d](a273c3d)) <!--SKIP CI-->
This PR was released!🚀 Changes included in v16.2.4 |

🎯 What does this PR do?
Magic blocks placed directly under an opening HTML tag with trailing text were rendering as literal text because CommonMark consumed them into the HTML flow block. The
terminateHtmlFlowBlockspreprocessor only recognized lines that were purely tags or had a matched open+close pair, so<li>test slipped pastisLineHtmland no blank-line terminator was inserted.Adds a third regex (
HTML_LINE_WITH_TRAILING_CONTENT_REGEX) for lines that start with a lowercase HTML tag followed by trailing text🧪 QA tips
verify that the table renders as expected
📸 Screenshot or Loom
monorepo showcase
