Skip to the text

Docs Guides

Known limits

What the engine deliberately leaves alone, what it knowingly does not do, and what has not been looked at. Nothing here is a surprise to the author; some of it is waiting for a page that needs it.

What the rules leave alone

Rule behaviour to know about

Money and grouped numbers

Fonts and avoid

Opt-in rules

Eleven rules are off until named in enable (number.group is described in the rules page) (rules.md). Their known limits:

HTML mode

Markdown mode

Rehype

Hanging punctuation

Speed

Since 0.2.0 the package is faster than typograf on English text in one piece, and about 2.5 times slower than tipograph and smartypants, which do much less (smartypants only quotes, dashes and ellipses; neither has abbreviations or ranges). The benchmark (apps/bench/RESULTS.md, generated on the author’s machine) reports, for English text in one piece, text mode, best of three (one run at 1 MB):

Tool 10 KB 100 KB 1000 KB
this package 0 ms 3 ms 30 ms
typograf 1 ms 6 ms 58 ms
typopo 20 ms 1053 ms 99 s (run in full this time)
richtypo 2493 ms not run (the file estimates about 248 s, extrapolating a quadratic) not run
tipograph 0 ms 1 ms 13 ms
smartypants 0 ms 1 ms 10 ms
retext-smartypants 4 ms 28 ms 545 ms

Every figure here is from one run of the benchmark (the 0.12.0 run), so only the ratios between the tools mean anything: the 0.10.0 run on a busier machine put this package at 63 ms and typograf at 118 at 1 MB, with every other tool about twice what it shows above. That is 0.03 s against 0.06 s for typograf at 1 MB; 0.2.0 took it from 0.32 s (the output is identical: a differential run against the engine as it was before the speed work, over 2.5 million combinations, finds no difference). The cost is linear in the text. What made the old engine slow was a regular expression that opens with a lookbehind, which the engine tries at every position; the rules now find a literal first and read the character before it (src/core/scan.ts), and a rule is skipped when a cheap test (Rule.when) shows its trigger is absent. The quote rule is about 10 ms per MB of ordinary prose (the 0.10.0 run, see backlog.md), 60 to 100 ms on text full of unpaired marks.

Since 0.2.1 no known input makes a rule superlinear: a run of one opening mark (“ repeated 40,000 times) took a minute and now takes about 7 ms, and text full of dialogue or unpaired marks costs about 45 to 70 ms per MB (it was 180 to 200). Every rule is linear on a long run of one character (spaces, digits, x, -, --, ., +-, 1x, !, )), and the opt-in space.before-punctuation, punctuation.double and symbol.degree no longer open with a lookbehind (about 1, 2 and 5 ms per MB when switched on, from 40, 40 and 200). bun scripts/stress.ts times each rule at 20,000, 40,000 and 80,000 characters of each such run and flags a doubling that costs more than 2.5 times; src/core/linear.test.ts holds the loose version of that as a test. The zone scan that sets code spans, addresses and emails aside before the rules run is outside the rules, and until 0.12.0 it cost 300 ms per MB on text with lone backticks and no line break (a regex flag it did not need); no benchmark text has a backtick, which is why the table above never showed it. The scan is timed as a row of its own by scripts/stress.ts and the profile since then.

On a JavaScriptCore engine (Bun) a lookbehind with \p{L} or \p{N} does not see letters outside the Basic Multilingual Plane. The scan helpers reproduce whatever the running engine does, so the output equals the old engine’s on the same runtime. One residual difference exists, only for ill-formed UTF-16 (a lone high surrogate directly after a fraction, followed by a character at U+E000 or above) and only with symbol.fraction on: the ½ is skipped. Well-formed text cannot reach it. A second JavaScriptCore quirk is in the original zone regex (src/core/zones.ts): when its email alternative scans a run of word characters, dots and hyphens that holds a www. and an astral character follows, the old engine reported the address one character late (.www.😀)😀 zoned ww.😀)😀). The rewrite finds the address correctly, so on such text, and only there, the output differs from the old engine’s. Speed numbers depend on the machine; re-run the benchmark (see contributing.md) to see your own. src/speed.test.ts is a loose budget (linear scaling, a 1 MB ceiling), not a measurement.

Benchmark caveats

These are repeated at the top of apps/bench/RESULTS.md.

Where the time goes in this package, and what to do about it, is in backlog.md.

Distribution

Something wrong or unclear here? Report an issue.