the big flat tree refactor

The motivation is to have one array represent the entire layout tree
rather than parent boxes storing children arrays. I started down this
rabbit hole in December after I realized text-align: justify needs to
bsearch inline content to reapply spacing after wrapping and reshaping.

It is a much more memory-efficient layout: up to 3% less memory is used
in CellEngine. I learned this technique from Chrome:

> The flat list is created for each inline formatting context in the
> order of a depth-first search of its inline layout subtree. Each entry
> in the list is a tuple of (object, number of descendants)

https://developer.chrome.com/docs/chromium/renderingng-data-structures#inline_fragment_items

For dropflow, it makes more sense to store the last descendant's index
instead of the number of descendants, since we maintain a parent stack
during iteration. The index of the box itself doesn't necessarily need
to be stored - usually you have the index during iteration - but I found
passing it around to be cumbersome, and it makes for a much better .id!

This refactor also includes a much more efficient strategy for handling
mixed block and inline content, documented in the comment. Generating
boxes was deeply affected due to the new way of storing trees.

Whitespace collapsing was also deeply affected since it can remove tree
nodes. It now happens in the generate phase, which is now called
flow.layout instead of flow.generate, and the old flow.layout is now
called flow.reflow. I want to use "layout" as a noun from now on, and
"reflow" for the verb, which is about what Firefox does. The new Layout
class will eventually store flat arrays of items and fragments, not just
the tree nodes.

Reflow is faster, mainly due to the removal of whitespace collapsing.
The new memory layout had slightly disappointing results, although it
did lead to fewer CPU cache misses, but the speedup was nothing to
remark about, except in perf-2 where it's a solid 2-3% faster. Painting
also got faster.

Author: chearon

README.md

@@ -115,7 +115,7 @@ const spanStyle = flow.style({
 });
 
 // Create a DOM
-const rootElement = flow.dom(
+const el = flow.dom(
   flow.h('div', {style: divStyle}, [
     'Hello, ',
     flow.h('span', {style: spanStyle}, ['World!'])
@@ -124,7 +124,7 @@ const rootElement = flow.dom(
 
 // Layout and paint into the entire canvas (see also renderToCanvasContext)
 const canvas = createCanvas(250, 50);
-await flow.renderToCanvas(rootElement, canvas);
+await flow.renderToCanvas(el, canvas);
 
 // Save your image
 fs.writeFileSync(new URL('file:///hello.png'), canvas.toBuffer());
@@ -152,14 +152,14 @@ const roboto1 = new flow.FontFace('Roboto', new URL('file:///Roboto-Regular.ttf'
 const roboto2 = new flow.FontFace('Roboto', new URL('file:///Roboto-Bold.ttf'), {weight: 700});
 flow.fonts.add(roboto1).add(roboto2);
 
-const rootElement = parse(`
+const el = parse(`
   <div style="background-color: #1c0a00; color: #b3c890; text-align: center;">
     Hello, <span style="color: #73a9ad; font-weight: bold;">World!</span>
   </div>
 `);
 
 const canvas = createCanvas(250, 50);
-flow.renderToCanvas(rootElement, canvas);
+flow.renderToCanvas(el, canvas);
 
 canvas.createPNGStream().pipe(fs.createWriteStream(new URL('hello.png', import.meta.url)));
 ```
@@ -185,12 +185,12 @@ Then, you can either render the DOM into a canvas using its size as the viewport
 
 1. [Render DOM to canvas](#render-dom-to-canvas)
 
-Or, you can use the lower-level functions to retain the layout, in case you want to re-layout at a different size, choose not to paint (for example if the layout isn't visible) or get intrinsics:
+Or, you can use the lower-level functions to retain the layout, in case you want to reflow at a different size, choose not to paint (for example if the layout isn't visible) or get intrinsics:
 
 1. [Load dependent resources](#load)
-2. [Generate a tree of layout boxes from the DOM](#generate)
-3. [Layout the box tree](#layout)
-4. [Paint the box tree to a target like canvas](#paint)
+2. [Create a layout for the DOM](#layout)
+3. [Reflow the layout](#layout)
+4. [Paint the layout to a target like HTML5 canvas](#paint)
 
 ## Fonts
 
@@ -378,7 +378,7 @@ Parses HTML. If you don't specify a root `<html>` element, content will be wrapp
 This is only for simple use cases. For more advanced usage continue on to the next section.
 
 ```ts
-function renderToCanvas(rootElement: HTMLElement, canvas: Canvas): Promise<void>;
+function renderToCanvas(el: HTMLElement, canvas: Canvas): Promise<void>;
 ```
 
 Renders the whole layout to the canvas, using its width and height as the viewport size.
@@ -393,7 +393,7 @@ class Image {
   reason: unknown;
 }
 
-function load(rootElement: HTMLElement): Promise<LoadableResource[]>;
+function load(el: HTMLElement): Promise<LoadableResource[]>;
 ```
 
 Ensures that all of the fonts and images required by the document are loaded.
@@ -405,7 +405,7 @@ For images, it means fetching the image data so it's ready for layout. In additi
 Because the whole fallback list for every unique set of font properties is appended to the returned loaded list, it may contain duplicate fonts. And since there is only one `Image` instance per unique URL, there may be duplicate images. Deduplication is left up to the caller since it impacts performance. The list is in document order.
 
 ```ts
-function loadSync(rootElement: HTMLElement): LoadableResource[];
+function loadSync(el: HTMLElement): LoadableResource[];
 ```
 
 If your URLs are all file:/// URLs in Node/Bun, `loadSync` can be used to load dependencies.
@@ -421,38 +421,42 @@ function revokeObjectURL(url: string): void;
 
 These functions can be used to send image buffers to `<img src>`. Since there is no associated document (unlike the browser), memory is retained between `createObjectURL` and `revokeObjectURL`.
 
-## Generate
+## Create a layout
 
-### `generate`
+### `layout`
 
 ```ts
-function generate(rootElement: HTMLElement): BlockContainer
+class Layout {
+  root(): BlockContainer;
+}
+
+function layout(el: HTMLElement): Layout;
 ```
 
-Generates a box tree for the element tree. Box trees roughly correspond to DOM trees, but usually have more boxes (like for anonymous text content between block-level elements (`div`s)) and sometimes fewer (like for `display: none`).
+Creates a layout, which consists of a box tree (the input to layout) as well as the fragmentation tree and glyphs (the output of layout). You need to call [`flow.reflow`](#reflow) at least once before painting or getting intrinsics.
 
-`BlockContainer` has a `repr()` method for logging the tree.
+Box trees roughly correspond to DOM trees, but usually have more boxes (like for anonymous text content between block-level elements (`div`s)) and sometimes fewer (like for `display: none`).
 
 Hold on to the return value so you can lay it out many times in different sizes, paint it or don't paint it if it's off-screen, or get intrinsics to build a higher-level logical layout (for example, spreadsheet column or row size even if the content is off screen).
 
-## Layout
+## Reflowing a layout
 
-### `layout`
+### `reflow`
 
 ```ts
-function layout(root: BlockContainer, width = 640, height = 480);
+function reflow(layout: Layout, width = 640, height = 480);
 ```
 
 Position boxes and split text into lines so the layout tree is ready to paint. Can be called over and over with a different viewport size.
 
-In more detail, layout involves:
+In more detail, reflowing involves:
 
 * Margin collapsing for block boxes
 * Passing text to HarfBuzz, iterating font fallbacks, wrapping, reshaping depending on break points
 * Float placement and `clear`ing
 * Positioning shaped text spans and backgrounds according to `direction` and text direction
-* Second and third pass layouts for intrinsics of `float`, `inline-block`, and `absolute`s
-* Post-layout positioning (`position`)
+* Calculating intrinsics for the content of `float`s, `inline-block`s, and `absolute`s
+* Post normal flow positioning (`position`)
 
 ## Paint
 
@@ -465,31 +469,31 @@ There is also a toy HTML target that was used early on in development, and kept
 ### `paintToCanvas`
 
 ```ts
-function paintToCanvas(root: BlockContainer, ctx: CanvasRenderingContext2D): void;
+function paintToCanvas(root: Layout, ctx: CanvasRenderingContext2D): void;
 ```
 
 Paints the layout to a browser canvas, node-canvas, or similar standards-compliant context.
 
 ### `paintToSvg`
 
 ```ts
-function paintToSvg(root: BlockContainer): string;
+function paintToSvg(root: Layout): string;
 ```
 
 Paints the layout to an SVG string, with `@font-face` rules referencing the URL you passed to `flow.FontFace`.
 
 ### `paintToSvgElements`
 
 ```ts
-function paintToSvgElements(root: BlockContainer): string;
+function paintToSvgElements(root: Layout): string;
 ```
 
 Similar to `paintToSvg`, but doesn't add `<svg>` or `@font-face` rules. Useful if you're painting inside of an already-existing SVG element.
 
 ### `paintToHtml`
 
 ```ts
-function paintToHtml(root: BlockContainer): string;
+function paintToHtml(root: Layout): string;
 ```
 
 Paint to HTML! Yes, this API can actually be used to go from HTML to HTML. It generates a flat list of a bunch of absolutely positioned elements. Probably don't use this, but it can be useful in development and is amusing.
@@ -529,8 +533,8 @@ Typically when you use `query`, you'll be getting a `BlockContainer`:
 
 ```ts
 const dom = parse('<div id="d" style="width: 100px; height: 100px;"></div>');
-const root = flow.generate(dom);
-flow.layout(root, 200, 200);
+const layout = flow.layout(dom);
+flow.reflow(layout, 200, 200);
 const [box] = dom.query('#d')!.boxes as flow.BlockContainer[];
 box.getContentArea().width; // 100
 box.getContentArea().height; // 100

examples/bidi-1.ts

@@ -19,13 +19,13 @@ const rootElement = parse(`
   </html>
 `);
 
-const blockContainer = flow.generate(rootElement);
+const layout = flow.layout(rootElement);
 
-blockContainer.log();
+flow.log(layout);
 flow.loadSync(rootElement);
 
 const canvas = createCanvas(200, 250);
-flow.layout(blockContainer, canvas.width, canvas.height);
+flow.reflow(layout, canvas.width, canvas.height);
 const ctx = canvas.getContext('2d');
-flow.paintToCanvas(blockContainer, ctx);
+flow.paintToCanvas(layout, ctx);
 fs.writeFileSync(new URL('bidi-1.png', import.meta.url), canvas.toBuffer());

examples/fallbacks-1.ts

@@ -17,14 +17,14 @@ const rootElement = parse(`
   </div>
 `);
 
-const blockContainer = flow.generate(rootElement);
+const layout = flow.layout(rootElement);
 
 flow.loadSync(rootElement);
-blockContainer.log();
+flow.log(layout);
 
 const canvas = createCanvas(400, 150);
-flow.layout(blockContainer, canvas.width, canvas.height);
+flow.reflow(layout, canvas.width, canvas.height);
 
 const ctx = canvas.getContext('2d');
-flow.paintToCanvas(blockContainer, ctx);
+flow.paintToCanvas(layout, ctx);
 fs.writeFileSync(new URL('fallbacks-1.png', import.meta.url), canvas.toBuffer());

examples/images-1.ts

@@ -74,11 +74,11 @@ const rootElement = flow.dom(
 
 // Normal layout, logging
 await flow.load(rootElement);
-const blockContainer = flow.generate(rootElement);
-flow.layout(blockContainer, 1200, 1800);
-blockContainer.log({containingBlocks: true});
+const layout = flow.layout(rootElement);
+flow.reflow(layout, 1200, 1800);
+flow.log(layout, undefined, {containingBlocks: true});
 const canvas = createCanvas(1200, 1800);
 const ctx = canvas.getContext('2d');
-flow.paintToCanvas(blockContainer, ctx);
+flow.paintToCanvas(layout, ctx);
 
 fs.writeFileSync(new URL('images-1.png', import.meta.url), canvas.toBuffer());

examples/inlines-1.ts

@@ -27,12 +27,12 @@ const rootElement = parse(`
 
 flow.loadSync(rootElement);
 
-const blockContainer = flow.generate(rootElement);
-blockContainer.log({css: 'zoom'});
+const layout = flow.layout(rootElement);
+flow.log(layout, undefined, {css: 'zoom'});
 
 const canvas = createCanvas(600, 400);
-flow.layout(blockContainer, canvas.width, canvas.height);
+flow.reflow(layout, canvas.width, canvas.height);
 
 const ctx = canvas.getContext('2d');
-flow.paintToCanvas(blockContainer, ctx);
+flow.paintToCanvas(layout, ctx);
 fs.writeFileSync(new URL('inlines-1.png', import.meta.url), canvas.toBuffer());

examples/perf-1.ts

@@ -48,32 +48,32 @@ flow.loadSync(rootElement);
 const canvas = createCanvas(1600, 1600);
 const ctx = canvas.getContext('2d');
 
-const blockContainer = flow.generate(rootElement);
-flow.layout(blockContainer, canvas.width, canvas.height);
+const layout = flow.layout(rootElement);
+flow.reflow(layout, canvas.width, canvas.height);
 ctx.clearRect(0, 0, canvas.width, canvas.height);
-flow.paintToCanvas(blockContainer, ctx);
+flow.paintToCanvas(layout, ctx);
 fs.writeFileSync(new URL('perf-1.png', import.meta.url), canvas.toBuffer());
 
 bench('altogether', () => {
-  const blockContainer = flow.generate(rootElement);
+  const layout = flow.layout(rootElement);
   flow.clearWordCache();
-  flow.layout(blockContainer, canvas.width, canvas.height);
+  flow.reflow(layout, canvas.width, canvas.height);
   ctx.clearRect(0, 0, canvas.width, canvas.height);
-  flow.paintToCanvas(blockContainer, ctx);
+  flow.paintToCanvas(layout, ctx);
 }).gc('inner');
 
-bench('generate', () => {
-  do_not_optimize(flow.generate(rootElement));
+bench('flow.layout', () => {
+  do_not_optimize(flow.layout(rootElement));
 }).gc('inner');
 
-bench('layout', () => {
+bench('flow.reflow', () => {
   flow.clearWordCache();
-  flow.layout(blockContainer, canvas.width, canvas.height);
+  flow.reflow(layout, canvas.width, canvas.height);
 }).gc('inner');
 
-bench('paint', () => {
+bench('flow.paint', () => {
   ctx.clearRect(0, 0, canvas.width, canvas.height);
-  flow.paintToCanvas(blockContainer, ctx);
+  flow.paintToCanvas(layout, ctx);
 }).gc('inner');
 
 await run();

examples/perf-2.ts

@@ -1910,33 +1910,33 @@ flow.loadSync(rootElement);
 const canvas = createCanvas(800 , 28696 );
 const ctx = canvas.getContext('2d');
 
-const blockContainer = flow.generate(rootElement);
-flow.layout(blockContainer, 800, 28696);
+const layout = flow.layout(rootElement);
+flow.reflow(layout, 800, 28696);
 ctx.clearRect(0, 0, 800 , 28696 );
-flow.paintToCanvas(blockContainer, ctx);
+flow.paintToCanvas(layout, ctx);
 
 fs.writeFileSync(new URL('perf-2.png', import.meta.url), canvas.toBuffer());
 
 bench('altogether', () => {
-  const blockContainer = flow.generate(rootElement);
+  const layout = flow.layout(rootElement);
   flow.clearWordCache();
-  flow.layout(blockContainer, 800, 28696);
+  flow.reflow(layout, 800, 28696);
   ctx.clearRect(0, 0, 800 , 28696 );
-  flow.paintToCanvas(blockContainer, ctx);
+  flow.paintToCanvas(layout, ctx);
 }).gc('inner');
 
-bench('generate', () => {
-  do_not_optimize(flow.generate(rootElement));
+bench('flow.layout', () => {
+  do_not_optimize(flow.layout(rootElement));
 }).gc('inner');
 
-bench('layout', () => {
+bench('flow.reflow', () => {
   flow.clearWordCache();
-  flow.layout(blockContainer, 800, 28696);
+  flow.reflow(layout, 800, 28696);
 }).gc('inner');
 
-bench('paint', () => {
+bench('flow.paint', () => {
   ctx.clearRect(0, 0, 800 , 28696 );
-  flow.paintToCanvas(blockContainer, ctx);
+  flow.paintToCanvas(layout, ctx);
 }).gc('inner');
 
 await run();

examples/perf-3.ts

@@ -26,18 +26,18 @@ const html = flow.dom(
   flow.h('html', {style}, words[Math.floor(Math.random() * words.length)])
 );
 flow.loadSync(html);
-const blockContainer = flow.generate(html);
-flow.layout(blockContainer, 100, 20);
-flow.paintToCanvas(blockContainer, ctx);
+const layout = flow.layout(html);
+flow.reflow(layout, 100, 20);
+flow.paintToCanvas(layout, ctx);
 fs.writeFileSync(new URL('perf-3.png', import.meta.url), canvas.toBuffer());
 
 bench('altogether', () => {
   const html = flow.dom(
     flow.h('html', {style}, words[Math.floor(Math.random() * words.length)])
   );
-  const blockContainer = flow.generate(html);
+  const layout = flow.layout(html);
   flow.clearWordCache();
-  flow.layout(blockContainer, 100, 20);
+  flow.reflow(layout, 100, 20);
 }).gc('inner');
 
 bench('dom', () => {
@@ -47,14 +47,14 @@ bench('dom', () => {
   do_not_optimize(html);
 }).gc('inner');
 
-bench('generate', () => {
-  const blockContainer = flow.generate(html);
-  do_not_optimize(blockContainer);
+bench('flow.layout', () => {
+  const layout = flow.layout(html);
+  do_not_optimize(layout);
 }).gc('inner');
 
-bench('layout', () => {
+bench('flow.reflow', () => {
   flow.clearWordCache();
-  flow.layout(blockContainer, 100, 20);
+  flow.reflow(layout, 100, 20);
 }).gc('inner');
 
 await run();