Dev Blog #5: Stress testing the parser.

Today it was time to see whether the work optimising the tag dictionary would pay off, by hooking it into the note parser.

The note parser's job is to take the raw text you enter, scan it for known tags and format it accordingly. That can involve bolding or italicising words and phrases, colourising the note's background, outline and text, or pulling in other information related to the content you've supplied. It can also entail recursion, as #tagged notes can in turn refer to other user and system tags that also apply formatting.

Because of these connections, modifying a note can have wide-ranging impact. A single edit could in principle affect the appearance of every other note in the document.

There are some (awkward and complicated) ways to minimise the amount of parsing work needed when a note is edited. For instance, if the edit does not destroy or create a new #tag, all the existing links between notes are guaranteed to still be valid. I duly set off along the path of caching those links before realising that I really didn't care if the best case was fast. I want Davenport to be fast, period.

So I grabbed a bigger set of random words (3,500) and wrote a testbed to create a spiderweb of 3,500 interlinked notes, each sporting a #tag definition, mentioning one or two other tagged words and some system formatting, as well as up to 40 words of junk text that would bog down the dictionary search with failures.

The result: a full-document recursive re-parse and format with no caching takes around 0.002 seconds, roughly 1/8th of the time available for smooth interactivity, which is a decent start. Caching of recursive results should bring that down even further - especially in real-world use where some tags will be referenced much more often than others (eg the main characters in a novel).