Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 0 additions & 22 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,17 +117,6 @@ This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
`<kbd>`, and `[x]{kbd onclick="…"}` is a bare `<kbd>`. A span with no semantic
name is unchanged. A DERIVED attribute yields to an AUTHORED one of the same
name, so `[x]{abbr="gen" title="authored"}` carries `title` once.
- **A caption on a block quote is now that quote's attribution** (PART 9 §4a,
markup-carve/carve#1159). `> To be` followed by `^ Hamlet` no longer parses as
a `figure` wrapping a `block_quote`; it is a `block_quote` carrying an
`attribution`, and HTML renders `<footer>Hamlet</footer>` inside the
`<blockquote>` rather than a `<figure>` / `<figcaption>` pair. A quote is not
a figure, takes no number, and no longer turns up in a walk for figures. The
Markdown, plain-text, ANSI and Carve writers all carry the attribution, and
the HTML importer reads a trailing `<footer>` in a `<blockquote>` back as the
attribution so the renderer's own output round-trips. A `<figure>` wrapping a
quote and a `<figcaption>` imports as a quote with an attribution too, since
a quote is no longer a figure target.
- **`code` and `mark` leave the built-in semantic registry.** Both spellings
follow the spec's seven-name list - `abbr`, `time`, `samp`, `var`, `kbd`,
`cite`, `dfn` - so `:code[x]` and `:mark[x]` take the generic
Expand Down Expand Up @@ -350,17 +339,6 @@ This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
closing tag or, for a complete tag alone on a line, at that line. Cut early, a
`<div>x</div>` followed by prose fenced the element alone and migrated the
prose as a paragraph outside the block the source had put it in.
- **A quote attribution stays attached to its quote on every target**
(markup-carve/carve#1179, PART 11 §10c). It used to follow the quote as a
sibling separated by a blank line, which kept the words but not what they
mean - read back, the attribution was attached to nothing, and a round trip
produced a blockquote with no attribution at all. Markdown now emits a
`<footer>` element inside the quote (that target already writes `<u>`,
`<mark>` and `<ins>` for constructs with no Markdown spelling, and through a
CommonMark reader `<footer>` opens an HTML block rather than being wrapped in
a paragraph, so the rendered HTML matches the HTML target's); the terminal
carries its quote bar onto the attribution line; plain text attaches by
adjacency, dropping the blank line. A quote with no attribution is unchanged.
- **Presentation targets no longer discard authored text**
(markup-carve/carve#1179). `docs/graceful-degradation.md` states the floor as
a MUST - "losing the click is fine; losing the words is not" - and three kinds
Expand Down
2 changes: 1 addition & 1 deletion spec
Submodule spec updated 42 files
+23 −0 RELEASING.md
+1 −1 docs/case-study/parsing-ast.md
+1 −1 docs/cheatsheet.md
+1 −1 docs/extensions.md
+1 −1 docs/index.md
+2 −3 docs/native-features-analysis.md
+0 −51 docs/versioning.md
+3 −3 package-lock.json
+1 −1 package.json
+4 −8 resources/ast-schema.json
+3 −0 resources/ast-value-divergence.txt
+0 −13 resources/engine-fmt-drift.txt
+12 −81 resources/engine-pin-drift.txt
+8 −8 resources/examples/core.md
+57 −47 resources/examples/edge-cases.md
+47 −133 resources/grammar.ebnf
+0 −2 resources/normative-clauses.txt
+31 −1 scripts/spec/ast-values.mjs
+32 −28 scripts/spec/html.mjs
+1 −1 tests/ast-schema.test.mjs
+4 −4 tests/corpus/05-lists-20.html
+2 −2 tests/corpus/07-blockquote-with-attribution.ansi
+4 −4 tests/corpus/07-blockquote-with-attribution.html
+2 −2 tests/corpus/07-blockquote-with-attribution.md
+1 −0 tests/corpus/07-blockquote-with-attribution.txt
+4 −4 tests/corpus/282-two-blank-lines-detach-a-caption-5.html
+9 −7 tests/corpus/306-a-captioned-quote-holds-more-than-one-block-2.html
+6 −4 tests/corpus/306-a-captioned-quote-holds-more-than-one-block-3.html
+6 −4 tests/corpus/306-a-captioned-quote-holds-more-than-one-block-4.html
+7 −5 tests/corpus/306-a-captioned-quote-holds-more-than-one-block-5.html
+7 −5 tests/corpus/306-a-captioned-quote-holds-more-than-one-block.html
+4 −4 tests/corpus/55-blockquote-caption-after-a-blank-line.html
+6 −6 tests/examples/demo.html
+0 −6 tests/extension-catalog-claims.test.mjs
+5 −1 tests/lint-rule-table-claims.test.mjs
+37 −6 tests/optional-corpus.test.mjs
+5 −4 tests/schema-fields-are-produced.test.mjs
+4 −4 tests/spec/05-lists.test
+4 −4 tests/spec/07-blockquote-with-attribution.test
+4 −4 tests/spec/282-two-blank-lines-detach-a-caption.test
+35 −25 tests/spec/306-a-captioned-quote-holds-more-than-one-block.test
+4 −4 tests/spec/55-blockquote-caption-after-a-blank-line.test
1 change: 0 additions & 1 deletion src/ast.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,6 @@ export interface Paragraph extends BaseNode {
export interface BlockQuote extends BaseNode {
type: 'block_quote'
children: BlockNode[]
attribution?: InlineNode[]
}

export interface List extends BaseNode {
Expand Down
5 changes: 0 additions & 5 deletions src/default-attributes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,11 +131,6 @@ function visit(node: AnyNode, byType: Map<string, Record<string, string>>): void
case 'figure':
visit((block as unknown as { target: AnyNode }).target, byType)
break
case 'block_quote': {
const attribution = (block as unknown as { attribution?: InlineNode[] }).attribution
if (attribution) for (const c of attribution) visit(c, byType)
break
}
default:
break
}
Expand Down
1 change: 0 additions & 1 deletion src/footnote-numbering.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,6 @@ function walkBlockInlines(
visit(node.children)
break
case 'block_quote':
if (node.attribution) visit(node.attribution)
node.children.forEach((c) => walkBlockInlines(c, visit, depth + 1))
break
case 'list':
Expand Down
1 change: 0 additions & 1 deletion src/heading-ids.ts
Original file line number Diff line number Diff line change
Expand Up @@ -963,7 +963,6 @@ export function resolveHeadingIds(
fn(b.children)
break
case 'block_quote':
if (b.attribution) fn(b.attribution)
b.children.forEach((c) => walkBlock(c, fn))
break
case 'list':
Expand Down
45 changes: 2 additions & 43 deletions src/html-import.ts
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ class Importer {
const attrs = this.attrs(node, path)
if (/^h[1-6]$/.test(tag)) return [{ type: 'heading', level: Number(tag[1]) as 1 | 2 | 3 | 4 | 5 | 6, children: this.inlines(node.childNodes ?? [], path, depth + 1), ...(attrs ? { attrs } : {}) }]
if (tag === 'p') return [{ type: 'paragraph', children: this.inlines(node.childNodes ?? [], path, depth + 1), ...(attrs ? { attrs } : {}) }]
if (tag === 'blockquote') return [this.blockQuote(node, path, depth, attrs)]
if (tag === 'blockquote') return [{ type: 'block_quote', children: this.blocks(node.childNodes ?? [], path, depth + 1), ...(attrs ? { attrs } : {}) }]
if (tag === 'ul' || tag === 'ol') return [this.list(node, path, depth, tag === 'ol', attrs)]
if (tag === 'pre') {
const code = node.childNodes?.find((n) => n.tagName === 'code')
Expand Down Expand Up @@ -347,53 +347,12 @@ class Importer {
return { type: 'table', rows, ...(attrs ? { attrs } : {}) }
}

/**
* A `<blockquote>`, with a trailing `<footer>` read as its ATTRIBUTION.
*
* PART 9 §4a puts the source of a quotation on the quote itself, and this
* renderer emits it as a `<footer>` inside the `<blockquote>` - so without
* this, the engine's own HTML did not survive a round trip through the
* importer: the footer came back as an ordinary second paragraph and the
* `^ ` line was gone from the Carve it wrote (carve#1159).
*
* The LAST footer child, because that is the one the renderer emits and the
* one an author writing HTML by hand puts after the quoted text. An earlier
* footer stays an ordinary block.
*/
private blockQuote(node: P5Node, path: string, depth: number, attrs?: Attrs): BlockNode {
const children = node.childNodes ?? []
const candidates = children.map((n, index) => ({ n, index })).filter(({ n }) => n.tagName === 'footer')
// The slot holds INLINE content, so a footer carrying blocks does not fit
// it. Flattening one would run its paragraphs together with no separator;
// leaving it an ordinary block inside the quote keeps every word, which is
// the better answer when the shape cannot be represented.
const last = candidates.filter(({ n }) => !(n.childNodes ?? []).some((c) => c.tagName && BLOCK.has(c.tagName))).at(-1)
const footerIndex = last?.index ?? -1
const footer = footerIndex === -1 ? undefined : children[footerIndex]
const body = footer ? children.filter((n) => n !== footer) : children
return {
type: 'block_quote',
children: this.blocks(body, path, depth + 1),
...(footer ? { attribution: this.inlines(footer.childNodes ?? [], `${path}/footer[1]`, depth + 1) } : {}),
...(attrs ? { attrs } : {}),
}
}

private figure(node: P5Node, path: string, depth: number, attrs?: Attrs): BlockNode[] {
const captionNode = node.childNodes?.find((n) => n.tagName === 'figcaption')
const body = (node.childNodes ?? []).filter((n) => n !== captionNode)
const targets = this.blocks(body, path, depth + 1)
const target = targets[0]
// A quote is no longer a figure target (PART 9 §4a): `<figure>` wrapping a
// `<blockquote>` and a `<figcaption>` is the shape this renderer used to
// emit, and it comes back as the quote carrying the caption as its
// attribution rather than as a figure the schema would refuse.
if (target?.type === 'block_quote' && captionNode) {
target.attribution = this.inlines(captionNode.childNodes ?? [], `${path}/figcaption[1]`, depth + 1)
if (attrs && !target.attrs) target.attrs = attrs
return targets
}
if (target && ['image', 'table', 'code_block', 'paragraph'].includes(target.type)) {
if (target && ['image', 'block_quote', 'table', 'code_block', 'paragraph'].includes(target.type)) {
return [{ type: 'figure', target: target as never, caption: this.inlines(captionNode?.childNodes ?? [], `${path}/figcaption[1]`, depth + 1), ...(attrs ? { attrs } : {}) }, ...targets.slice(1)]
}
this.add('element-unwrapped', 'Unwrapped figure without a representable target', 'warning', path)
Expand Down
36 changes: 10 additions & 26 deletions src/parse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4694,6 +4694,7 @@ function parseBlockQuote(lexer: Lexer): BlockQuote | Figure {
const subLexer = nestedSubLexer(lexer, inner, firstLineIndex, innerLineNumbers)
const children = parseBlocks(subLexer, 0)
const bq: BlockQuote = { type: 'block_quote', children }
const quoteEndIndex = lexer.pos
// Optional caption with ^
// Allow one blank line between
let lookahead = 0
Expand All @@ -4705,17 +4706,12 @@ function parseBlockQuote(lexer: Lexer): BlockQuote | Figure {
// or is separated by at most ONE blank line.
if (cap && lookahead <= 1) {
for (let i = 0; i <= lookahead; i++) lexer.consume()
// PART 9 §4a: the caption on a quote is its ATTRIBUTION. The quote is
// returned as itself rather than wrapped in a figure, so it takes no
// figure number and nothing walking the tree for figures finds it
// (carve#1159). The block loop attaches the span, which now covers the
// quote plus its caption - there is no inner node left without one.
// PARSED AS ORDINARY INLINE, not as a caption. A caption's `#` is the
// number placeholder, and an attribution has no number to place - PART 9
// §4a says the placeholder stays LITERAL there, which it cannot do if the
// parser turns it into a `caption_number` that renders as nothing.
bq.attribution = parseCaptionInline(lexer, cap[1]!, false)
return bq
attachBlockPos(lexer, bq, firstLineIndex, quoteEndIndex)
return {
type: 'figure',
target: bq,
caption: parseCaptionInline(lexer, cap[1]!),
} as Figure
}
}
return bq
Expand Down Expand Up @@ -6904,19 +6900,7 @@ function startsInterruptingBlock(lexer: Lexer, content?: string): boolean {
* is sitting on does not end with the caption text, the mapping is not exact
* and the positions are dropped, as before.
*/
/**
* @param captionContext - false for a quote's ATTRIBUTION (PART 9 §4a). The
* only difference is the bare `#`: a caption's is the number placeholder, and
* an attribution has no number to place, so §4a keeps it literal. Everything
* else - the continuation lines, the document anchors that give each inline
* node a real position - is shared, and was the reason a separate parser here
* produced text nodes whose `pos` sliced back to the wrong bytes.
*/
function parseCaptionInline(
lexer: Lexer,
firstLine: string,
captionContext = true,
): InlineNode[] {
function parseCaptionInline(lexer: Lexer, firstLine: string): InlineNode[] {
const capIndex = lexer.pos - 1
const capLine = lexer.lines[capIndex]
const anchors: Array<{ offset: number; column: number }> = []
Expand All @@ -6932,7 +6916,7 @@ function parseCaptionInline(
const text = readCaptionText(lexer, firstLine, anchorable ? anchors : undefined)
if (!anchorable) {
return stripPositions(
parseInline(text, lexer.abbrDefs, lexer.linkDefs, undefined, captionContext),
parseInline(text, lexer.abbrDefs, lexer.linkDefs, undefined, true),
)
}
return parseInline(
Expand All @@ -6946,7 +6930,7 @@ function parseCaptionInline(
startColumn: anchors[0]!.column,
lineAnchors: anchors,
}),
captionContext,
true,
)
}

Expand Down
5 changes: 2 additions & 3 deletions src/profile-filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
*
* carve-js' AST is heterogeneous: children live under different fields
* depending on the node (`children`, `items`, `rows`, `cells`, `terms`,
* `definitions`, `inline`, `content`, `target`, `caption`, `title`,
* `attribution`). We expose a uniform child-list view over those fields so
* `definitions`, `inline`, `content`, `target`, `caption`, and `title`). We
* expose a uniform child-list view over those fields so
* the walk mirrors carve-php's `getChildren()` / `removeChild()` /
* `replaceChildNode()` semantics.
*/
Expand Down Expand Up @@ -145,7 +145,6 @@ function childArrays(node: NodeLike): ChildArray[] {
break
case 'block_quote':
push(node['children'], true)
if (node['attribution']) push(node['attribution'], false)
break
case 'heading':
case 'paragraph':
Expand Down
18 changes: 1 addition & 17 deletions src/render-ansi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,23 +159,7 @@ function renderBlock(node: BlockNode, ctx: AnsiContext): string {
{
const out = renderBlocks(node.children, ctx)
ctx.blockQuoteDepth--
if (node.attribution === undefined) return out
// The attribution keeps the caption's styling it had while a quote was
// a figure - italic and dim - so a terminal reader sees the same thing
// it saw before, only inside the quote rather than under a figure.
//
// PART 11 §10c T2: it also carries the QUOTE BAR. The bar is already
// this target's marker for "inside the quote", and the attribution was
// the one line in the quote that did not get it - so the source read as
// a separate block that merely happened to follow. Nothing new is
// invented; the prefix the body lines already use is applied one line
// further.
const bar = `${style('│', FG_CYAN + DIM)} `
// Trim the caption's own block separator BEFORE prefixing: prefixing it
// would draw a bar on the trailing blank lines, so the quote appeared to
// continue past its own end.
const attribution = prefixLines(trimEndNonNbsp(renderCaption(node.attribution, ctx)), bar)
return `${trimEndNonNbsp(out)}\n${bar.trimEnd()}\n${attribution}\n\n`
return out
}
case 'list':
return renderList(node, ctx)
Expand Down
7 changes: 1 addition & 6 deletions src/render-carve.ts
Original file line number Diff line number Diff line change
Expand Up @@ -636,12 +636,7 @@ function renderBlock(node: BlockNode, ctx: CarveContext): string {
.split('\n')
.map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n')
// PART 9 §4a: an attribution is written back as the `^` line it was read
// from. Without this the writer dropped it - `fmt` is not allowed to lose
// content (PART 11 §1), and the loss was silent because the field is new.
const attribution =
node.attribution === undefined ? '' : `\n^ ${renderInlines(node.attribution, ctx)}`
return withAttrs(body) + attribution
return withAttrs(body)
}
case 'list':
return withAttrs(renderList(node, ctx))
Expand Down
16 changes: 2 additions & 14 deletions src/render-html.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1184,28 +1184,16 @@ function renderBlockQuote(node: BlockQuote, opts: RenderOptions, level: number):
child.type === 'paragraph' ? null : renderBlock(child, opts, level + 1),
)
const visible = node.children.filter((_, i) => rendered[i] !== '')
// PART 9 §4a: a `^` caption on a quote is its ATTRIBUTION, and it renders
// INSIDE the quote - where a quotation's source belongs - rather than turning
// the quote into a figure with a figcaption (carve#1159).
const attribution =
node.attribution === undefined
? ''
: `\n${pad} <footer>${renderInlines(node.attribution, opts)}</footer>`
if (visible.length === 1 && visible[0]!.type === 'paragraph' && attribution === '') {
if (visible.length === 1 && visible[0]!.type === 'paragraph') {
const para = visible[0] as Paragraph
const inner = renderInlines(para.children, opts)
return `${pad}<blockquote${attrs}><p${renderAttrs(para.attrs)}${sourceLineAttr(opts, para.pos?.startLine, para.attrs)}>${inner}</p></blockquote>`
}
if (attribution !== '' && visible.length === 1 && visible[0]!.type === 'paragraph') {
const para = visible[0] as Paragraph
const inner = renderInlines(para.children, opts)
return `${pad}<blockquote${attrs}>\n${pad} <p${renderAttrs(para.attrs)}${sourceLineAttr(opts, para.pos?.startLine, para.attrs)}>${inner}</p>${attribution}\n${pad}</blockquote>`
}
const inner = node.children
.map((child, i) => rendered[i] ?? renderBlock(child, opts, level + 1))
.filter((piece) => piece !== '')
.join('\n')
return `${pad}<blockquote${attrs}>\n${inner}${attribution}\n${pad}</blockquote>`
return `${pad}<blockquote${attrs}>\n${inner}\n${pad}</blockquote>`
}

function renderList(node: List, opts: RenderOptions, level: number): string {
Expand Down
22 changes: 2 additions & 20 deletions src/render-markdown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -154,26 +154,8 @@ function renderBlock(node: BlockNode, ctx: MarkdownContext): string {
return `${fence}${info}\n${content}\n${fence}\n\n`
}
case 'block_quote': {
let body = trimNonNbsp(renderBlocks(node.children, ctx))
// PART 11 §10c T1. The attribution is the quotation's SOURCE, so it stays
// inside the quote. It used to follow as a sibling paragraph, which kept
// the words but not what they mean - read back, it was attached to
// nothing, and round-tripping produced a blockquote with no attribution.
//
// Markdown has no attribution syntax but does admit HTML, and this target
// already writes `<u>`, `<mark>`, `<sub>`, `<ins>` and `<del>` for
// constructs with no Markdown spelling. Through a CommonMark reader
// `<footer>` opens an HTML BLOCK inside the quote (it is not wrapped in a
// paragraph), so the rendered HTML is what the HTML target produces from
// the same source.
if (node.attribution !== undefined) {
body += `\n\n<footer>${trimNonNbsp(renderInlines(node.attribution, ctx))}</footer>`
}
const quoted = body
.split('\n')
.map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n')
return `${quoted}\n\n`
const lines = trimNonNbsp(renderBlocks(node.children, ctx)).split('\n')
return `${lines.map((line) => `> ${line}`).join('\n')}\n\n`
}
case 'list':
return renderList(node, ctx)
Expand Down
14 changes: 2 additions & 12 deletions src/render-plain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,18 +136,8 @@ function renderBlock(node: BlockNode, ctx: PlainContext): string {
const label = node.label ? `${stripControls(node.label)}\n\n` : ''
return `${header}${label}${stripControls(node.content)}\n\n`
}
case 'block_quote': {
const quoted = `"${trimNonNbsp(renderBlocks(node.children, ctx))}"`
// PART 11 §10c T3. ADJACENCY, not a blank line. A blank line is what
// separates blocks on this target, so putting one here said the
// attribution was a block of its own rather than the quotation's source -
// the words survived, the attachment did not. No punctuation is invented:
// a dash prefix would put a character in the output the author never
// wrote.
const attribution =
node.attribution === undefined ? '' : `\n${renderInlines(node.attribution, ctx)}`
return `${quoted}${attribution}\n\n`
}
case 'block_quote':
return `"${trimNonNbsp(renderBlocks(node.children, ctx))}"\n\n`
case 'list':
return renderList(node, ctx)
case 'thematic_break':
Expand Down
Loading
Loading