During the runtime the data of Tiddlywiki is stored in javascript objects. These objects are synchronized with the DOM-Representation of Tiddlywiki. This means every change of the original data of a Tiddler, fires an event which changes all DOM-Representations of the Tiddler and the javascript object. The bare-bones Wiki store is created during the boot process and is kept in a object called \$tw.Wiki. This object contains amongst others a hashmap of the different Tiddlers of Tiddlywiki. The Hashmap is used to store the javascript object representation of the different Tiddlers. Furthermore this object is used to manage the tiddlers during runtime, it provides methods for adding tiddlers, search tiddlers by name and delete tiddlers. As shown in the picture below, every change at the DOM triggers an event which changes the corresponding widget which again changes the store of the tiddlers. The whole image shows how WikiText is parsed by a set of rules into the parse tree and this parse tree is rendered as a tree of widgets. This Rendertree is synchronised to the DOM. Every modification on the Rendertree provokes a start of the rendering-pipeline. As well as every change on the wikiText triggers an event at the RenderTree. This Process uses a selective updating so that only the changed parts are updated. This means only widgets which have to change the DOM in consequence of a changed tiddler are refreshed.

The next important part of the application are deserializers. Deserializers are responsible to load tiddlers from various sources. One of the deserializers provided by the microkernel for example can load new tiddlers from the current DOM tree. The counterpart of deserializers are saver. A saver is used to save the complete store or the complete TiddlyWiki for that matter at once. After a user made changes to some tiddlers, this method can be used to download the current state of the store as a completely new TiddlyWiki application.

Another way of persisting data are syncadaptors. Like deserializer and saver, a syncadaptor can load tiddlers into the store and can persist changed tiddlers. But in contrast to deserializer and saver it can load and persist single tiddlers. A syncadaptor can provide a list of loadable tiddlers registers at the store for changes. Now when a tiddler is changed and these changes are written to the store, the syncadaptor is triggered and persists the new changes, for each tiddler individually.

Tiddlers can be persisted from/to harddisk or synced with a remote server.

TiddlyWiki has two approaches to saving the user data (tiddlers) depending on whether you are using the single file version, or are using the node.js server.

TiddlyWiki on node.js saves each tiddler in a separate text file. For details on the file formats supported for tiddlers on node.js, see Tiddler Files.

The single file version of TiddlyWiki saves the tiddlers within the HTML file. See Data Storage in Single File TiddlyWiki

TiddlyWiki has two approaches to saving the user data (tiddlers) depending on whether you are using the single file version, or are using the node.js server.

TiddlyWiki on node.js saves each tiddler in a separate text file. For details on the file formats supported for tiddlers on node.js, see Tiddler Files.

The single file version of TiddlyWiki saves the tiddlers within the HTML file. See Data Storage in Single File TiddlyWiki

Modules with module-type: tiddlerdeserializer can provide functions to create tiddlers out of any textual representation. Each function must be associated with a type like application/json or text/html. They get the textual representation of the tiddlers, some default field values and the type of the text block as arguments. They return an array of JavaScript objects representing the resulting tiddlers.

Deserializers are not managed by the syncer module. Instead the concept of deserializers is in fact part of the microkernel. This is necessary because the microkernel needs a way of loading tiddlers before it can load the core plug-in and execute it's startup modules. (Due to the fact that the core plug-in and it's modules are represented as tiddlers.)

The load-modules startup module loads additional deserializers and pushes them into the store. The core plug-in for example contains a few deserializers which can read a whole TiddlyWiki 5 or classic HTML file and load the individual tiddlers into the store.

The microkernel only contains a bare store and some deserializers to load tiddlers from JSON files or from the DOM of the current HTML file. The core plug-in adds some more deserializers and a new mechanism for persisting and synchronising tiddlers.

This mechanism is provided as a global module in $:/core/modules/syncer.js. The saver module has three responsibilities:

1. Save the whole wiki.
2. Provide the ability to download single tiddlers as files.
3. Synchronise the local wiki store with a remote wiki store, i.e. running in Node.js

The syncer module is connected mainly to two other modules. For one it registers to changes at the wiki store (Event Mechanism) and if any changes occur they are synced to the remote store. Then it provides a function saveWiki(options). This function can be used by other modules. For example the RootWidget uses this function to save the whole wiki or start downloading single tiddlers.

The syncer itself does not provide a concrete implementation of saving, downloading or syncing the tiddlers. Instead it loads modules of type saver and syncadaptor and manages the saving/syncing process.

Deserializer

Modules with module-type: tiddlerdeserializer can provide functions to create tiddlers out of any textual representation. Each function must be associated with a type like application/json or text/html. They get the textual representation of the tiddlers, some default field values and the type of the text block as arguments. They return an array of JavaScript objects representing the resulting tiddlers.

Deserializers are not managed by the syncer module. Instead the concept of deserializers is in fact part of the microkernel. This is necessary because the microkernel needs a way of loading tiddlers before it can load the core plug-in and execute it's startup modules. (Due to the fact that the core plug-in and it's modules are represented as tiddlers.)

The load-modules startup module loads additional deserializers and pushes them into the store. The core plug-in for example contains a few deserializers which can read a whole TiddlyWiki 5 or classic HTML file and load the individual tiddlers into the store.

Saver

Modules with module-type: saver provide functionality to save the whole wiki. There are three methods a saver can support:

save

This method is used, when the user requests a save, for example by clicking the save button in the sidebar.

autosave

This method is used automatically by TW when tiddlers are changed, created or deleted by the user.

download

This message is used when the wiki or a single tiddler should explicitly be downloaded. The control panel for example uses this method to provide a button which saves the wiki as a static HTML file.

A saver module has to export two functions. canSave(wiki) returning true if this module is capable of working and create(wiki) returning an instance of a saver object. This saver object has to provide an info property containing a name, a priority, an array of methods it supports and a method save(text,method,callback). This method is called from TW with the actual text which should be saved, the method which is used and a callback function to report errors: callback("Error while saving") or to notify that saving went well: callback(null, "Saving went well :)"). If the saver method successfully saved the file it has to return true, or false otherwise. Saves are triggered by messages from the UI. The syncer module uses the saver with the highest priority capable of the requested method to save the file.

The core plug-in contains a saver capable of saving the current state of the wiki to the local hard drive by using a special Firefox extension called Tiddlyfox. If this extension is not available, the savers canSave method would return false. A saver with a lower priority would then ask the user to save the current state as a new HTML file.

Syncadaptor

A module with module-type: syncadaptor provides functionality to get a list of tiddlers (this list is provided as SkinnyTiddlers, which are normal tiddlers without the text field) and to load, save and delete single tiddlers. A syncadaptor can also provide functions to login and logout so that syncadaptor modules can be used to synchronise tiddlers with a remote server.

The syncer module only uses one syncadaptor and honours a special system tiddler $:/config/SyncFilter containing a filter string. Tiddlers matching this filter string are saved to the server with a syncadapter. It uses the WebServer API to load modified tiddlers from the server, which returns only non-system tiddlers.

Event Mechanism

Most of the following mechanisms need a way to get notified, when anything in the wiki store changes. The core -plug-in adds an event system to the bare wiki store. The event system provides the ability to listen to events. The most important is the "change" event which notifies the listeners when tiddlers have changed, with a list of the changed tiddlers. The event mechanism is one of the few mechanisms which needs a hook at the microkernel: The microkernel contains an empty function "enqueueTiddlerEvent(event)" and calls this function when a tiddler is added or deleted. The event mechanism from the core plug-in overwrites this function with it's own implementation. The functions providing the event system are added via the wikimethod module type.

Caching

The core plug-in adds two caches to the wiki store. A global cache and a cache for each tiddler. While the global cache is cleared whenever any tiddler changes, the tiddler cache is only cleared when the associated tiddler changes. The idea of the tiddler cache is, that sometimes we want to calculate information from a tiddler but don't want to recalculate every time we need this information. Imagine a tiddler which contains links to other tiddlers and at one point we want to have a list of the linked tiddlers (LinksFilter). With the tiddler cache we can parse the WikiText, extract the linked tiddlers and put this list in the cache. Until the tiddler is changed, the cache can be used to access this information in the next requests.

The same idea holds for the global cache. For example when a filter string is parsed and a list of matching tiddlers is constructed. This list can be put in the global cache for later reuse until something changes in the store.

Like the Event Mechanism, the cache needs hook functions in the microkernel. The microkernel calls clearGlobalCache() and clearCache(tiddlertitle) when a tiddler changes. The core's cache mechanism overwrites this functions. The functions providing the cache system are added via the wikimethod module type.

System Tiddlers

The core plug-in introduces a segregation of tiddlers. The tiddler model is central in the whole TiddlyWiki application and can be used in various roles. Because of the fact that a TiddlyWiki user works with tiddlers (taking the role of a wiki page) and tiddlers are the building blocks of the whole application (including modules, UI elements, etc.) we need to identify the internal tiddlers.

The core plug-in introduces the concept of system tiddlers. It builds on the convention that application internal tiddler names start with $:/. Then the core plug-in introduces a set of new functions to the wiki store which are used to retrieve tiddlers like getTiddlers(options) and forEachTiddler(callback, options). These functions work with all tiddlers in the store but the options parameter provides the ability to sort tiddlers by a field-name and exclude tiddlers with a specific tag. By default it doesn't return system tiddlers. To get a list of all tiddlers including system tiddlers, this must be requested explicitly via the options. If a function wants to present a list of tiddlers to the user it can use this new functions so that internal application tiddlers wouldn't clutter the resulting list. These functions are added via the wikimethod module type.

Tags and Filter Mechanism

Tags

The core plug-in extends the store by a simple mechanism to tag a tiddler. This provides the functionality to

• retrieve tiddlers with a specific tag
• retreive a hashmap of { tag: [tiddler1, tiddler3, tiddler3] }
• list or iterate tiddlers not tagged with a specific tag

The tags are stored directly in the tiddler. Each tiddler can have a field named "tag", which contains an array of its tags. The above functions use this field to calculate their results and cache them in a global cache.

Organising information with tags is easy, intuitive and is common throughout the web. In most cases the functions mentioned above are not enough and a more sophisticated way of querying is needed. But instead of focusing only on tags TiddlyWiki introduces a querying system that isn't bound to tags or single tiddlers.

Filter mechanism

This filter mechanism is build on the idea of a pipeline of single filter operators. Filters are noted as strings and can look like

[tag[task]!tag[done]interesting[very]]

This example would (implicitly) put all available tiddlers into the pipe. The first operator tag[task] would only pass tiddlers which are tagged with "task". The !tag[done] operator is negated and only passes tiddlers which are not tagged with "done". The last filter operator passes only tiddlers with a field "interesting" set to "very". So as a result this filter would be used to obtain all tiddlers which are marked as task, aren't already done and are very interesting.

There are many filter operators already build into the core plug-in including the mentioned tag- and field operators, filter operators to sort the tiddlerlist by a specified field, etc. But more sophisticated operators are possible, too. An example would be the search-operator. This filter operator looks for the searched string in the text and in the tags of the tiddlers. If the provided filter operators are not enough, a developer can add new filters by adding a module with the filteroperator type.

System Tags

Tags and the filter mechanism are used throughout the core plug-in for internal puproses. Tags which start with $:/ are normally hidden from the casual user, similar to System Tiddlers

The filter mechanism is added to the wiki store with the wikimethod module type.

TiddlyWiki is a personal notebook application based on a wiki application. In addition to a static Web-Site, TiddlyWiki is implemented as a single page application. This is a approach to build rich internet applications, it includes the possibility to put application logic into web-pages to make them dynamically. Furthermore this means the whole application is delivered in one HTML file, consisting of source code to dynamically change the view and behaviour of the application as well as the data of the application. During the runtime nothing must be loaded from a server to the TiddlyWiki application. The HTML file contains everything needed to start the application. TiddlyWiki is highly customisable because of a very sophisticated module concept. Except of a micro-kernel written in JavaScript the whole application consist of a own data-structure called tiddlers and a own markup language called wikiText. Even the modules are realised as tiddlers.

The aim of this documentation is to overview the idea behind the TiddlyWiki application as well as give a overview of the architecture to the reader. This means after reading the documentation the reader is has the knowledge how the overall application works and where the points are where the reader can extend the functionality of the application.

Section Overview:

TiddlyWiki - A quick Overview

This section describes the idea of the TiddlyWiki application. This should give the reader a overview over what TiddlyWiki consists of give a brief introduction to the topics of the main documentation.

Microkernel Architecture

The heart of TiddlyWiki is it's microkernel. This section will describe what bare mechanisms are included in the microkernel and how additional modules are loaded.

TiddlyWiki Core Application

After the microkernel has been explained this section focuses on the core plug-in. It describes the central modules of the core and explains how they work together to build a single file application.

Conclusion

A briefly summary about this documentation bringing out the advantages, disadvantages and experiences with TiddlyWiki.

Messages are events that are triggered by the user. They are generated by widgets for example when the user clicks on a ButtonWidget. Each message has a type property like "tm-delete-tiddler" and a parameter.

{type: "tm-delete-tiddler", param: "MyOldTiddler"}

When such a message is created by a widget it sends the message to it's parent widget which sends it to it's own parent widget and so on. On this way each widget can try to dispatch the message. This concept is realised in the base widget object. It provides a function dispatchEvent(message) which is called by the children to bubble the message up the widget tree. Another function addEventListener(type,listener) can be used to bind a function to a specific message type. If the listener returns false, the message is send to the parent widget.

The TiddlyWiki core plug-in handles a lot of messages in the NavigatorWidget.

In the universe of TiddlyWiki everything is a tiddler. Even the application logic is stored in tiddlers that are marked as "application/javascript". These tiddlers, which contain application logic, are called modules and a CommonJS compatible module system is responsible for assembling the individual modules into the TiddlyWiki application. The result is a tree representing the whole TiddlyWiki application containing module tiddlers, data tiddlers and some JavaScript functions and objects.

Only a small part of the TiddlyWiki is not managed as tiddlers, the microkernel. The microkernel is the first thing to run, when the application is started and it puts some initial objects and functions into the application tree, which are needed to load and manage tiddlers. After the microkernel built this initial application tree, the remaining parts of the application can be loaded as module tiddlers. Beside some utility functions the most important object that is contributed by the boot kernel is "$tw.wiki", consisting of JavaScript structures and functions that are used to store and manage the loaded tiddlers. Among other things this store can be used to add new tiddlers, remove tiddlers and retrieve tiddlers by name.

The microkernel constructs a initial $tw object containing the needed structures and functions.

The whole application is basically built from three parts. At first, the microkernel provides the basic functionality to handle tiddlers. The second part are tiddlers representing core functionality. These are for example modules which extend the store by more sophisticated functions, UI tiddlers and widget modules, a WikiText parser, sophisticated deserializers, savers, syncadapters, etc. These core modules are provided as plug-in to the microkernel. Consequently, a plug-in is a single tiddler which itself contains multiple tiddlers, forming the plug-in. Each of this tiddler might be a module providing new functionality (i.e. a module tiddler marked with "module-type: saver" can extend the application with new methods of saving the current wiki state.). Tiddlers provided in plug-ins are called shadow tiddlers. They are immutable and can not be edited or deleted but we can create a new tiddler with the same name to override a shadow tiddler.

The first stage of WikiText processing is the parser. A Parser is provided by a module with module-type: parser and is responsible to transform block of text to a parse-tree. The parse-tree consists of nested nodes like

{type: "element", tag: <string>, attributes: {}, children: []} - an HTML element
{type: "text", text: <string>} - a text node
{type: "entity", entity: <string>} - an HTML entity like &copy; for a copyright symbol
{type: "raw", html: <string>} - raw HTML

The core plug-in provides a recursive descent WikiText parser which loads it's individual rules from individual modules. Thus a developer can provide additional rules by using module-type: wikirule. Each rule can produce a list of parse-tree nodes. A simple example for a wikirule producing a <hr> from --- can be found in horizrule.js

HTML tags can be embedded into WikiText because of the html rule. This rule matches HTML tag syntax and creates type: "element" nodes. But the html-rule has another special purpose. By parsing the HTML tag syntax it implicitly parses WikiText widgets. It the recognises them by the $ character at the beginning of the tag name and instead of producing "element" nodes it uses the tag name for the type:

{type: "list", tag: "$list", attributes: {}, children: []} - a list element

The Widgets part will reveal why this makes sense and how each node is transformed into a widget. Another special characteristic of the html-rule or the parse nodes in general is the attributes property. Attributes in the parse-tree are not stored as simple strings but they are nodes of its own to make indirect text references available as attributes as described in Widgets:

{type: "string", value: <string>} - literal string
{type: "indirect", textReference: <textReference>} - indirect through a text reference

TiddlyWiki supports a wide range of methods to persist your data. One of this methods is the HTML5 fallback saver. This methods works on almost every browser. With this method a copy of the entire wiki will be downloaded by the browser. This means you get a new file everytime you hit the save button. To avoid this and because every Browser has a different API to allow writing direct to the file system there a some plugins for the different browsers. These plug-ins allow the user to save direct to the current open TiddlyWiki-File. The Listing below shows the HTML5-compliant to save the changes via the HTML5 fallback saver by downloading the TW as a complete HTML-file.

DownloadSaver.prototype.save = function(text,method,callback) {
	...
	var link = document.createElement("a");
	link.setAttribute("target","_blank");
	...
	link.setAttribute("href","data:text/html," + encodeURIComponent(text));
	...
	link.setAttribute("download",filename);
	document.body.appendChild(link);
	link.click();
	document.body.removeChild(link);
	return true;
};

The previous parts of this chapter showed how WikiText is transformed to DOM nodes which dynamically react to tiddler changes and a way to compose tiddlers from other tiddlers. This last part describes how the TiddlyWiki core plug-in starts up a UI build from tiddlers and WikiText.

After the microkernel has loaded it starts executing Startup Modules. The core plug-in contains two startup modules which are responsible to kick off the UI:

rootwidget.js is a startup module and creates an instance of the base widget. This widget is globally accessible $tw.rootWidget. The DOM node associated to this widget is the current browser window's DOM (document).

At first, the root widget has no children but provides some basic event handlers (Messages) like:

• tm-notify: Displays the message given in param as a notification.
• tm-save-wiki: Triggered by a save button, the user can click. This handler uses the syncer module described in Extended Persistence to save the current wiki.
• tm-auto-save-wiki: Similar to tm-save-wiki but not triggered directly by the user but automatically triggered when a wiki page is edited and saved. A Saver implementation which starts a download of the updated wiki file would not support the auto-save method and would only be used when the tm-save-wiki message is used.
• tm-download-file: This message also uses the syncer module described in Extended Persistence but explicitly demands to choose a saver with the download-method to start downloading a single tiddler.

After the root widget is loaded another startup module $:/core/modules/startup/render.js creates a transclude widget which contains the contents of $:/core/ui/PageTemplate which is now bound to the browsers DOM document. The render function of the transclude widget is initially executed and a listener is registered at the store which executes the refresh function of the transclude widget to trigger the Selective Update process.

Techniques for including other tiddlers and Templates are finally used in $:/core/ui/PageTemplate to build the TiddlyWiki UI only from tiddlers written in WikiText (with widgets implemented in javascript):

For example to implement the list of open wiki pages the $:/core/ui/PageTemplate contains a navigator widget which maintains a list of open tiddlers in a field of $:/StoryList and handles events like tm-navigate by adding a tiddler specified as parameter to the top of the list in $:/StoryList. The story tiddler transcluded in $:/core/ui/PageTemplate then uses a ListWidget to transclude all tiddlers in $:/StoryList through a special template $:/core/ui/ViewTemplate. A event of the type tm-close-tiddler would remove a specified tiddler from $:/StoryList. The Event Mechanism would trigger a changed event which triggers a call of the ListWidget's refresh function which would remove the tiddler from the list, closing the tiddler.

Modules with module-type: saver provide functionality to save the whole wiki. There are three methods a saver can support:

save

This method is used, when the user requests a save, for example by clicking the save button in the sidebar.

autosave

This method is used automatically by TW when tiddlers are changed, created or deleted by the user.

download

This message is used when the wiki or a single tiddler should explicitly be downloaded. The control panel for example uses this method to provide a button which saves the wiki as a static HTML file.

A saver module has to export two functions. canSave(wiki) returning true if this module is capable of working and create(wiki) returning an instance of a saver object. This saver object has to provide an info property containing a name, a priority, an array of methods it supports and a method save(text,method,callback). This method is called from TW with the actual text which should be saved, the method which is used and a callback function to report errors: callback("Error while saving") or to notify that saving went well: callback(null, "Saving went well :)"). If the saver method successfully saved the file it has to return true, or false otherwise. Saves are triggered by messages from the UI. The syncer module uses the saver with the highest priority capable of the requested method to save the file.

The core plug-in contains a saver capable of saving the current state of the wiki to the local hard drive by using a special Firefox extension called Tiddlyfox. If this extension is not available, the savers canSave method would return false. A saver with a lower priority would then ask the user to save the current state as a new HTML file.

TiddlyWiki - A quick Overview

This section describes the idea of the TiddlyWiki application. This should give the reader a overview over what TiddlyWiki consists of give a brief introduction to the topics of the main documentation.

Microkernel Architecture

The heart of TiddlyWiki is it's microkernel. This section will describe what bare mechanisms are included in the microkernel and how additional modules are loaded.

TiddlyWiki Core Application

After the microkernel has been explained this section focuses on the core plug-in. It describes the central modules of the core and explains how they work together to build a single file application.

Conclusion

A briefly summary about this documentation bringing out the advantages, disadvantages and experiences with TiddlyWiki.

With Messages a widget is able to put some kind of events into the TiddlyWiki application which is one part of the dynamic behaviour of widgets. The other part is selective updating. Widgets are often dependant on tiddler states. The ListWidget for example can be configured to list all tiddlers which are tagged with "important". Now, when such a tiddler is changed and it's "important" tag is removed, this change should reflect in the ListWidget. To allow widgets to react on such changes, each widget can provide a function refresh(changedTiddlers). The RootWidget is registered to the wiki store, using the Event Mechanism. When an change event occurs it starts to call the refresh function of its children with a list of the changed tiddlers. Each widget can then decide if it has to change or re-render its DOM representation and call the refresh function of its own children. This way every time a tiddler or the wiki store itself changes, each widget can instantly react on these changes or ignore them.

Another way of updating are text reference attributes (text references explained in Transclusion and TextReference):

{type: "indirect", textReference: <textReference>}

When a widget got a attribute which is a text reference and the refresh function is called, it can check if the text reference references a changed tiddler and update accordingly.

Nearly every state of the UI is stored in tiddlers. A search mechanism for example would use a EditTextWidget which shows an text input field for the search string and a ListWidget to show the results. The EditTextWidget is bound to a tiddler $:/temp/search. Meaning when editing the text in the input field the tiddlers content is changed to this text. On the other hand when the tiddler changes the RootWidget is notified. It then starts calling its childrens refresh methods. Eventually the refresh method of the EditTextWidget is called and it can re-render itself if necessary. This way TiddlyWiki can re-use an already existing data structure which is not only convenient because we don't need to introduce an additional structure but tiddlers managed in the wiki store are already a pretty powerful data structure, supporting an Event Mechanism (so we don't need an additional observer pattern), Caching, Metadata by additional tiddler fields and a way to obtain specific tiddlers.

Modules with module-type: startup have to export a function named startup and may export a name property to identify this module. The startup function will be executed by the boot kernel at the end of the boot process.

// From boot.js:
// Gather up any startup modules
$tw.boot.remainingStartupModules = []; // Array of startup modules
$tw.modules.forEachModuleOfType("startup",function(title,module) {
	if(module.startup) {
		$tw.boot.remainingStartupModules.push(module);
	}
});
// Keep track of the startup tasks that have been executed
$tw.boot.executedStartupModules = Object.create(null);
$tw.boot.disabledStartupModules = $tw.boot.disabledStartupModules || [];
// Repeatedly execute the next eligible task
$tw.boot.executeNextStartupTask();

executeNextStartupTask() will execute the remaining startup modules in remainingStartupModules. A startup module can export the variables before and/or after, each containing an array of names of other startup modules. executeNextStartupTask() will use this information to execute the modules in the correct order.

Startup modules can be marked as synchronous by exporting synchronous = true. If synchronous is set to false, the startup function is executed with an callback function as the first argument. The startup function has to call this function to allow subsequent startup modules to get executed. This is necessary when the startup function itself uses asynchronous calls.

A module with module-type: syncadaptor provides functionality to get a list of tiddlers (this list is provided as SkinnyTiddlers, which are normal tiddlers without the text field) and to load, save and delete single tiddlers. A syncadaptor can also provide functions to login and logout so that syncadaptor modules can be used to synchronise tiddlers with a remote server.

The syncer module only uses one syncadaptor and honours a special system tiddler $:/config/SyncFilter containing a filter string. Tiddlers matching this filter string are saved to the server with a syncadapter. It uses the WebServer API to load modified tiddlers from the server, which returns only non-system tiddlers.

Tags

The core plug-in extends the store by a simple mechanism to tag a tiddler. This provides the functionality to

• retrieve tiddlers with a specific tag
• retreive a hashmap of { tag: [tiddler1, tiddler3, tiddler3] }
• list or iterate tiddlers not tagged with a specific tag

The tags are stored directly in the tiddler. Each tiddler can have a field named "tag", which contains an array of its tags. The above functions use this field to calculate their results and cache them in a global cache.

Organising information with tags is easy, intuitive and is common throughout the web. In most cases the functions mentioned above are not enough and a more sophisticated way of querying is needed. But instead of focusing only on tags TiddlyWiki introduces a querying system that isn't bound to tags or single tiddlers.

Filter mechanism

This filter mechanism is build on the idea of a pipeline of single filter operators. Filters are noted as strings and can look like

[tag[task]!tag[done]interesting[very]]

This example would (implicitly) put all available tiddlers into the pipe. The first operator tag[task] would only pass tiddlers which are tagged with "task". The !tag[done] operator is negated and only passes tiddlers which are not tagged with "done". The last filter operator passes only tiddlers with a field "interesting" set to "very". So as a result this filter would be used to obtain all tiddlers which are marked as task, aren't already done and are very interesting.

There are many filter operators already build into the core plug-in including the mentioned tag- and field operators, filter operators to sort the tiddlerlist by a specified field, etc. But more sophisticated operators are possible, too. An example would be the search-operator. This filter operator looks for the searched string in the text and in the tags of the tiddlers. If the provided filter operators are not enough, a developer can add new filters by adding a module with the filteroperator type.

System Tags

Tags and the filter mechanism are used throughout the core plug-in for internal puproses. Tags which start with $:/ are normally hidden from the casual user, similar to System Tiddlers

The filter mechanism is added to the wiki store with the wikimethod module type.

The boot-kernel is responsible for creating a bare-bones TW environment. It is running under Node.js or in a HTML5 Browser. The Bootkernel just loads enough functionality to load the modules containing the main logic of the application. This boot-kernel contains a few helper methods, the module mechanism as well as the function to create a tiddler and manage them. The boot-kernel also creates the bare-bones wiki store, which holds all the information of the wiki during the runtime. After creating the store, the boot-kernel is in charge of decrypting the encrypted tiddlers and extracting all the tiddlers e.g. the core module tiddlers embedded in the DOM structure of the HTML file. Furthermore the boot kernel offers the functionality to load tiddlers from a file, when you run TW with Node.js. All other functionality which is not a part of the boot kernel is added dynamically by modules and plugins. The boot kernel is able to load the core plugins and perform the startup plugins. The core contains the startup modules shown in the picture below.

Following the "anything is a tiddler" concept, even the UI consists of tiddlers. This is possible because tiddlers can not only contain plain text or JavaScript (modules) but they also can contain a special markup text called WikiText. By using WikiText the user can put markup elements like tables or images in a tiddler. To provide some more sophisticated UI elements, WikiText can also contain special widgets like text input fields, checkboxes, dynamic lists etc. In most cases, these widgets are used to directly modify or represent the information contained in other tiddlers. If a tiddler is changed and this change should reflect in an UI element e.g. a widget, a process called selective update takes place. Selective updating means when a tiddler or a set of tiddlers changes, each widget is asked, if changes to this tiddlers would affect its appearance. If so, the respective widget is re-rendered otherwise it remains unchanged.

By managing nearly every part of the application as tiddlers, the application is only needed to provide some basic functionality to manage the individual tiddlers, load and persist them, render them to HTML output and provide a way to register for the changes made to tiddlers. This way the whole wiki application can be build from these simple concepts. Plug-ins can be used to add new functionality to the existing modules or even to replace individual tiddlers/modules, enabling developers to build whole new applications on the TiddlyWiki base system.

A tiddler is the smallest unit of the TiddlyWiki system. It can contain any data like plain text, WikiText markup, JavaScript code (module tiddler), JSON structures (JSON structures might even contain additional tiddlers. Plug-ins are implemented this way to pack multiple tiddlers in a single plug-in tiddler), images in SVG format or even binary images encoded with base64. Internally Tiddlers are immutable objects containing a bunch of key:value pairs called fields. The only required field of a tiddler is the title field. The Standard fields of a tiddler are listed below. Nearly everything in TiddlyWiki is loaded as tiddlers. Plug-ins for example are a bunch of tiddlers that are distributed as a single JSON tiddler. The only exception is the microkernel which isn't a tiddler.

created

Timestamp number of milliseconds since 01.01.1970.

modified

Timestamp number of milliseconds since 01.01.1970.

tags

list of tags seperated by whitespace. Tags which contain whitespaces are wrapped by , e.g. example Tag.

type

Type of the Tiddler, e.g. text/plain or text/vnd.tiddlywiki .

title

Title of the Tiddler

list

An ordered list of tiddler titles associated with a tiddler

Tiddlers are used in multiple roles and on different levels. A developer uses tiddlers as the basic element containing application code, configuration values and even as a form of variable to save the current UI state. On a different level, a tiddler is also the basic unit of work for the wiki user, e.g. the individual wiki pages are implemented as tiddlers. This makes sense for multiple reasons: Because the UI of TiddlyWiki is build from tiddlers, the wiki user is able to edit the interface of his own TiddlyWiki just by editing a wiki page. For example to add a list of tiddler links to the sidebar, the user just needs to create a new tiddler, put the links into this tiddler and tag this tiddler with $:/tags/SideBar. This way the user can customise his work environment just by using mechanisms he already uses to manage his wiki pages. Tiddlers consist of fields. When using a tiddler as wiki page, the user can use these fields to store meta information, like tags.

Because fields for metadata and especially tags are an easy way for the user to organise his wiki pages, TiddlyWiki provides a special filter mechanism to choose tiddlers using their metadata. A filter string like [tag[learncard]topic[math]!tag[successful]] would filter all tiddlers tagged with "learncard", with the value "math" in the topic-field and are not tagged with "successful". A user could use this filter together with the <$list> widget to display a list of all math learncards which are not yet answered successfully in a wiki page.

Another example which shows how the "anything is a tiddler" concept leads to an environment where a single feature brings great benefit is the drag and drop feature. HTML5 standard comes with a native drag and drop feature. TiddlyWiki uses this feature and makes it possible to drag and drop a tiddler from one instance to another. And because anything is a tiddler, this brings the ability to drag and drop individual wiki pages, JavaScript modules, UI components and whole plug-ins between TiddlyWiki instances.

Traditional web applications are bound to ~HTTP-Concepts, including stateless requests to transfer data. In these applications a state is often emulated by the use of sessions, which need to be handled on the client and especially on the server side. These restrictions often lead to a fragmented user experience because the user interface is rebuilt on every data transfer. TiddlyWiki tries to overcome these restrictions and the resulting disadvantages by building on few but basic concepts which loosen the coupling of HTTP and the actual application, eliminating the need of state emulation and resulting in a wiki style single page application with the ability to run in an offline environment.

TiddlyWiki as Single Page Application

TiddlyWiki builds on some basic concepts. First, TiddlyWiki should not be perceived as a dynamic web page like traditional server-side generated web pages. Instead TiddlyWiki can be perceived as an application which is written entirely in JavaScript and uses HTML5 and CSS3 to render a GUI. This way TiddlyWiki can be executed in any JavaScript environment like a browser or a node.js instance, while keeping the advantages of a simple application. One of these advantages is, that TiddlyWiki has no need to emulate an application state, as a traditional web application would need to do.

A second idea concerns the storage of the application data. In contrast to a traditional web application, TiddlyWiki doesn't store the data in an external database but simply uses native data structures already existing in JavaScript to store tiddlers, the basic (atomic) element of the TiddlyWiki application, in the memory. Additional core modules provide a way to persist this storage in simple HTML div elements.

Just by building on these simple and basic concepts,

• TiddlyWiki is able to store application data in a single HTML page by using div elements as data container.
• TW is able to store application code (JavaScript) in the same single HTML page.
• TiddlyWiki can be executed in any JavaScript environment like a browser.

These points already enable TW to be used as an offline-enabled single file web application. Also, by using a server side node.js environment running the same TiddlyWiki application, TiddlyWiki can be used as an online web application. This is realized on server-side by providing an additional module to persist tiddlers into plain text files and on client-side by a module syncing the local data store with the node.js server.

Microkernel

In the universe of TiddlyWiki everything is a tiddler. Even the application logic is stored in tiddlers that are marked as "application/javascript". These tiddlers, which contain application logic, are called modules and a CommonJS compatible module system is responsible for assembling the individual modules into the TiddlyWiki application. The result is a tree representing the whole TiddlyWiki application containing module tiddlers, data tiddlers and some JavaScript functions and objects.

Only a small part of the TiddlyWiki is not managed as tiddlers, the microkernel. The microkernel is the first thing to run, when the application is started and it puts some initial objects and functions into the application tree, which are needed to load and manage tiddlers. After the microkernel built this initial application tree, the remaining parts of the application can be loaded as module tiddlers. Beside some utility functions the most important object that is contributed by the boot kernel is "$tw.wiki", consisting of JavaScript structures and functions that are used to store and manage the loaded tiddlers. Among other things this store can be used to add new tiddlers, remove tiddlers and retrieve tiddlers by name.

The microkernel constructs a initial $tw object containing the needed structures and functions.

Data Persistence

The next important part of the application are deserializers. Deserializers are responsible to load tiddlers from various sources. One of the deserializers provided by the microkernel for example can load new tiddlers from the current DOM tree. The counterpart of deserializers are saver. A saver is used to save the complete store or the complete TiddlyWiki for that matter at once. After a user made changes to some tiddlers, this method can be used to download the current state of the store as a completely new TiddlyWiki application.

Another way of persisting data are syncadaptors. Like deserializer and saver, a syncadaptor can load tiddlers into the store and can persist changed tiddlers. But in contrast to deserializer and saver it can load and persist single tiddlers. A syncadaptor can provide a list of loadable tiddlers registers at the store for changes. Now when a tiddler is changed and these changes are written to the store, the syncadaptor is triggered and persists the new changes, for each tiddler individually.

Tiddlers can be persisted from/to harddisk or synced with a remote server.

The User Interface

Following the "anything is a tiddler" concept, even the UI consists of tiddlers. This is possible because tiddlers can not only contain plain text or JavaScript (modules) but they also can contain a special markup text called WikiText. By using WikiText the user can put markup elements like tables or images in a tiddler. To provide some more sophisticated UI elements, WikiText can also contain special widgets like text input fields, checkboxes, dynamic lists etc. In most cases, these widgets are used to directly modify or represent the information contained in other tiddlers. If a tiddler is changed and this change should reflect in an UI element e.g. a widget, a process called selective update takes place. Selective updating means when a tiddler or a set of tiddlers changes, each widget is asked, if changes to this tiddlers would affect its appearance. If so, the respective widget is re-rendered otherwise it remains unchanged.

Modularization

The whole application is basically built from three parts. At first, the microkernel provides the basic functionality to handle tiddlers. The second part are tiddlers representing core functionality. These are for example modules which extend the store by more sophisticated functions, UI tiddlers and widget modules, a WikiText parser, sophisticated deserializers, savers, syncadapters, etc. These core modules are provided as plug-in to the microkernel. Consequently, a plug-in is a single tiddler which itself contains multiple tiddlers, forming the plug-in. Each of this tiddler might be a module providing new functionality (i.e. a module tiddler marked with "module-type: saver" can extend the application with new methods of saving the current wiki state.). Tiddlers provided in plug-ins are called shadow tiddlers. They are immutable and can not be edited or deleted but we can create a new tiddler with the same name to override a shadow tiddler.

Tiddler as Basic Element

By managing nearly every part of the application as tiddlers, the application is only needed to provide some basic functionality to manage the individual tiddlers, load and persist them, render them to HTML output and provide a way to register for the changes made to tiddlers. This way the whole wiki application can be build from these simple concepts. Plug-ins can be used to add new functionality to the existing modules or even to replace individual tiddlers/modules, enabling developers to build whole new applications on the TiddlyWiki base system.

A tiddler is the smallest unit of the TiddlyWiki system. It can contain any data like plain text, WikiText markup, JavaScript code (module tiddler), JSON structures (JSON structures might even contain additional tiddlers. Plug-ins are implemented this way to pack multiple tiddlers in a single plug-in tiddler), images in SVG format or even binary images encoded with base64. Internally Tiddlers are immutable objects containing a bunch of key:value pairs called fields. The only required field of a tiddler is the title field. The Standard fields of a tiddler are listed below. Nearly everything in TiddlyWiki is loaded as tiddlers. Plug-ins for example are a bunch of tiddlers that are distributed as a single JSON tiddler. The only exception is the microkernel which isn't a tiddler.

created

Timestamp number of milliseconds since 01.01.1970.

modified

Timestamp number of milliseconds since 01.01.1970.

tags

list of tags seperated by whitespace. Tags which contain whitespaces are wrapped by , e.g. example Tag.

type

Type of the Tiddler, e.g. text/plain or text/vnd.tiddlywiki .

title

Title of the Tiddler

list

An ordered list of tiddler titles associated with a tiddler

Tiddlers are used in multiple roles and on different levels. A developer uses tiddlers as the basic element containing application code, configuration values and even as a form of variable to save the current UI state. On a different level, a tiddler is also the basic unit of work for the wiki user, e.g. the individual wiki pages are implemented as tiddlers. This makes sense for multiple reasons: Because the UI of TiddlyWiki is build from tiddlers, the wiki user is able to edit the interface of his own TiddlyWiki just by editing a wiki page. For example to add a list of tiddler links to the sidebar, the user just needs to create a new tiddler, put the links into this tiddler and tag this tiddler with $:/tags/SideBar. This way the user can customise his work environment just by using mechanisms he already uses to manage his wiki pages. Tiddlers consist of fields. When using a tiddler as wiki page, the user can use these fields to store meta information, like tags.

Because fields for metadata and especially tags are an easy way for the user to organise his wiki pages, TiddlyWiki provides a special filter mechanism to choose tiddlers using their metadata. A filter string like [tag[learncard]topic[math]!tag[successful]] would filter all tiddlers tagged with "learncard", with the value "math" in the topic-field and are not tagged with "successful". A user could use this filter together with the <$list> widget to display a list of all math learncards which are not yet answered successfully in a wiki page.

Another example which shows how the "anything is a tiddler" concept leads to an environment where a single feature brings great benefit is the drag and drop feature. HTML5 standard comes with a native drag and drop feature. TiddlyWiki uses this feature and makes it possible to drag and drop a tiddler from one instance to another. And because anything is a tiddler, this brings the ability to drag and drop individual wiki pages, JavaScript modules, UI components and whole plug-ins between TiddlyWiki instances.

WikiText Markup

The WikiText is a markup language, created especially for the requirements of the TiddlyWiki application. It is based on Markdown, but extended with some TiddlyWiki specific features. On one hand its a text-to-HTML conversion language and on the other hand its used to provide the interactive features of TiddlyWiki. The aim of this language is to allow the user of the software to focus on the writing. The WikiText is used to format Tiddlers within the TiddlyWiki application. The tags of the WikiText syntax can be used within the standard text input field. During the saving process these tags renders to HTML elements for example:

WikiText:--- 
Renders as:
HTML:<hr>
WikiText:[img[https://tiddlywiki.com/favicon.ico]]
Renders as: TW
HTML:<img src="https://tiddlywiki.com/favicon.ico">

Furthermore the WikiText is used to access the widgets which are integrated in the application.These widgets are used to enhance the the WikiText with a rich functionality. Widgets are based on the ~HTML-Syntax but always starts with a $.

WikiText:
<$button message="tm-close-tiddler">Close Me!</$button> 

TiddlyWiki builds on some basic concepts. First, TiddlyWiki should not be perceived as a dynamic web page like traditional server-side generated web pages. Instead TiddlyWiki can be perceived as an application which is written entirely in JavaScript and uses HTML5 and CSS3 to render a GUI. This way TiddlyWiki can be executed in any JavaScript environment like a browser or a node.js instance, while keeping the advantages of a simple application. One of these advantages is, that TiddlyWiki has no need to emulate an application state, as a traditional web application would need to do.

A second idea concerns the storage of the application data. In contrast to a traditional web application, TiddlyWiki doesn't store the data in an external database but simply uses native data structures already existing in JavaScript to store tiddlers, the basic (atomic) element of the TiddlyWiki application, in the memory. Additional core modules provide a way to persist this storage in simple HTML div elements.

Just by building on these simple and basic concepts,

• TiddlyWiki is able to store application data in a single HTML page by using div elements as data container.
• TW is able to store application code (JavaScript) in the same single HTML page.
• TiddlyWiki can be executed in any JavaScript environment like a browser.

These points already enable TW to be used as an offline-enabled single file web application. Also, by using a server side node.js environment running the same TiddlyWiki application, TiddlyWiki can be used as an online web application. This is realized on server-side by providing an additional module to persist tiddlers into plain text files and on client-side by a module syncing the local data store with the node.js server.

This section shows a quick and short overview over the startup process of TW, from the first step of the boot mechanism until the loading of the different startup modules. The image shown below shall point out the main parts of this startup-process.

• Introduction
• TiddlyWiki - A quick Overview
  • TiddlyWiki as Single Page Application
  • Microkernel
  • Data Persistence
  • The User Interface
  • Modularization
  • Tiddler as Basic Element
  • WikiText Markup
• Microkernel Architecture
  • Microkernel
  • Datamodel
  • Module System
• TiddlyWiki Core Application
  • Startup Process
  • Extending the Store
    • Event Mechanism
    • Caching
    • System Tiddlers
    • Tags and Filter Mechanism
  • Extended Persistence
    • Deserializer
    • Saver
    • Syncadaptor
  • UI and Rendering Pipeline
    • WikiText
    • Parser
    • Widgets
      • Messages
      • Selective Update
    • Transclusion and TextReference
    • RootWidget and Rendering Startup
    • Navigator Widget
    • Draft Mechanism
• Conclusion

The previous parts about Widgets and the Parser explained how a block of WikiText is transformed into a DOM representation and how this presentation can react on changes to the wiki store. The previous chapters also describe that WikiText is saved in individual tiddlers, including the WikiText describing the UI components. This raises the question, how these multiple tiddlers are build up to a single UI. But before answering this question we need to introduce text references and transclusion.

TextReference

A text reference describes a special notation to indirectly refer to the contents of a specified tiddler field. The syntax of a text reference is:

<tiddlertitle>
<tiddlertitle>!!<fieldname>
!!<fieldname> - specifies a field of the current tiddlers

To obtain the actual text, the core plug-in adds a function to the wiki store getTextReference(textRef,defaultText,currTiddlerTitle). The "currentTiddlerTitle" is the title of a tiddler which is used when no tiddlerTitle is specified in the text reference. What the currentTiddler should be, depends on where the text reference is used. If it is for example used as a widget attribute, the current tiddler is the tiddler which contains this widget. Text references are used by widgets (attributes), filteroperators (parameter) and in transclusions. These elements use the getTextReference(textRef,defaultText,currTiddlerTitle) function.

Transclusion

Transclusion means including the contents of a different tiddler. This is realized with a transclude widget which parses the tiddler to be included and adds the resulting nodes as children of its own parse node. The trick with transclusion is, that it shows the content of a different tiddler but by default it does not change the current tiddler. This enables us to create tiddlers with text references and use them as a templates.

For example: Tiddler MyTask having the fields and the content of:

important: "very"
assoc.person: "Hans Dampf"

<$transclude tiddler="TaskHeaderTemplate" />

Hans needs some more Dampf.

And Tiddler TaskHeaderTemplate with a content of:

<$view field="assoc.person"/> has a <$view field="important"/> important task for us:

When showing tiddler MyTask it would result in:

Hans Dampf has a very important task for us:

Hans needs some more Dampf.

Transclusion and templates is one of the most important concepts in TiddlyWiki. It allows us to include other tiddlers using the metadata for our current tiddler. It also allows us to transclude a template tiddler with a third tiddler set as the currentTiddler with the TiddlerWidget:

<$tiddler tiddler="MyTiddler">
<$transclude tiddler="EditTemplate" />
</$tiddler>

This way we can create a different view on a tiddler which does not only show it's title and it's content but shows it's content and metadata in editable text fields and allows us to edit this tiddler. Also the template concept is used by the ListWidget to show the tiddlers through a specified template. Finally, when wanting to download the wiki as a new html file, this html file is created by binding a list of all tiddlers to a template and sending the resulting text to the syncer module described in Extended Persistence to save the current wiki.