Introduction

A tiddlywiki.files JSON file in a sub-folder within a TiddlyWiki folder overrides the usual logic for recursively scanning the folder for tiddler files. Instead, the tiddlywiki.files file specifies instructions for loading tiddlers from specific files and folders.

The format of the file is an object with two optional properties:

• tiddlers - an array of objects describing external files with the ability to override or modify any of the fields read from the file
• directories - an array of objects describing external directories, a filter determining which files within those directories should be processed, and the ability to override or modify any of the fields read from the file

Note that significant enhancements to tiddlywiki.files processing were introduced in Release 5.1.14.

Field overrides

Both the tiddlers and directories sections of tiddlywiki.files files include the ability to override or customise the values of fields with a fields object.

Each field can be specified as either a string or array value to be assigned directly to the field, or Introduced in v5.1.14 an object describing how to generate the value for the field. The object contains the following properties:

• source - (optional) a string specifying the source value for the field. If not specified, the existing value is used
  • filename the filename of the file containing the tiddler
  • filename-uri-decoded the filename of the file containing the tiddler, with URI decoding applied
  • basename the filename of the file containing the tiddler without any extension
  • basename-uri-decoded the filename of the file containing the tiddler without any extension, with URI decoding applied
  • extname the extension of the filename of the file containing the tiddler
  • created the creation date/time of the file containing the tiddler
  • modified the modification date/time of the file containing the tiddler
  • New in v5.3.0 filepath the path of the file containing the tiddler, relative to the path property of the directory (only usable in directories declarations)
  • New in v5.3.0 subdirectories an array of the subdirectories in the file's path relative, to the path property of the directory (only usable in directories declarations)
• prefix - (optional) a string to be prepended to the value of the field
• suffix - (optional) a string to be appended to the value of the field

Tiddlers section

The file specifications in the tiddlers array support the following properties:

• file: (required) the absolute or relative path to the file containing the tiddler data (relative paths are interpreted relative to the path of the tiddlywiki.files file)
• isTiddlerFile: (optional) if true, the file will be treated as a tiddler file and deserialised to extract the tiddlers. Otherwise, the raw content of the file is assigned to the text field without any parsing
• fields: (optional) an object containing values that override or customise the fields provided in the tiddler file (see above)
• prefix & suffix: (optional) strings to be prefixed and suffixed to the tiddler text field

  Note that providing a prefix here is equivalent to setting the text field of the fields object to {"prefix":"<prefixvalue>"}.

Directories section

Directory specifications in the directories array may take the following forms:

• a string literal, specifying the absolute or relative path to the directory containing the tiddler files (relative paths are interpreted relative to the path of the tiddlywiki.files file). The directory is recursively searched for tiddler files
• Introduced in v5.1.14 an object with the following properties:
  • path - (required) the absolute or relative path to the directory containing the tiddler files (relative paths are interpreted relative to the path of the tiddlywiki.files file). Note that by default the directory is not recursively searched; sub-directories are ignored unless the searchSubdirectories flag is set to true (see below).
  • filesRegExp - (optional) a regular expression that matches the filenames of the files that should be processed within the directory
  • isTiddlerFile - (required) if true, the file will be treated as a tiddler file and deserialised to extract the tiddlers. Otherwise, the raw content of the file is assigned to the text field without any parsing
  • isEditableFile - Introduced in v5.1.23 (optional) if true, changes to the tiddler be saved back to the original file. The tiddler will be saved back to the original filepath as long as it does not generate a result from the $:/config/FileSystemPath filters, which will override the final filepath generated if a result is returned from a filter.
  • searchSubdirectories - Introduced in v5.1.23 (optional) if true, all subdirectories of the path are searched recursively for files that match the (optional) filesRegExp. If no filesRegExp is provided, all files in all subdirectories of the path are loaded. Tiddler titles generated via the source attribute filename (see above) will only include the filename, not any of the subdirectories of the path. If this results in multiple files with loaded with the same tiddler title, then only the last file loaded under that tiddler title will be in memory. In order to prevent this, you can use the filepath attribute instead of filename. Alternately, you can include multiple directory objects and customise the title field with a prefix or suffix alongside the source attribute.
  • fields - (required) an object containing values that override or customise the fields provided in the tiddler file (see above)

Fields can also be overridden for particular files by creating a file with the same name plus the suffix .meta – see TiddlerFiles.

Examples

These example tiddlywiki.files must be placed in their own sub-directory of the wiki folder.

There are also several examples of tiddlywiki.files files in the main TiddlyWiki 5 GitHub repository.

Importing a folder of PDFs

This example retrieves all the files with the extension .pdf from a folder specified by a relative path. This path starts with "../../../" indicating 3 directory levels above the folder holding this config file. Each tiddler is set up for LazyLoading with the following fields:

• title - set to the URI decoded base filename of the PDF file. URI decoding allows characters like "/" to be included in titles by URI encoding them as "%2F"
• created - set to the creation date/time of the PDF file
• modified - set to the modification date/time of the PDF file
• type - set to application/pdf
• tags - set to $:/tags/AttachedFile
• text - set to an empty string
• _canonical_uri - set to the string "pdfs/" concatenated with the filename

{
    "directories": [
        {
            "path": "../../../input/pdfs",
            "filesRegExp": "^.*\\.pdf$",
            "isTiddlerFile": false,
            "fields": {
                "title": {"source": "basename-uri-decoded"},
                "created": {"source": "created"},
                "modified": {"source": "modified"},
                "type": "application/pdf",
                "tags": ["$:/tags/AttachedFile"],
                "text": "",
                "_canonical_uri": {"source": "filename", "prefix": "pdfs/"}
            }
        }
    ]
}

Importing a folder of text files

This example retrieves all the files with the extension .txt from a folder specified by a relative path. This folder is within the wiki's base directory, and the current config file is in a directory within the wiki's "tiddlers/" directory. So, in this case the path starts with "../../" to traverse upwards two directory levels, and then down into the "externalnotes/" directory. Each tiddler is set up with the following fields:

• title - set to the URI decoded base filename of the text file. URI decoding allows characters like "/" to be included in titles by URI encoding them as "%2F"
• created - set to the creation date/time of the text file
• modified - set to the modification date/time of the text file
• type - set to text/plain
• tags - set to [[note]] [[externalnote]] [[.txt]] (using array notation)
• text - not set, thus the content of the file is loaded as the text field

{
    "directories": [
        {
            "path": "../../externalnotes",
            "filesRegExp": ".+\\.txt",
            "isTiddlerFile": false,
            "isEditableFile": true,
            "fields": {
                "title": {"source": "basename-uri-decoded"},
                "created": {"source": "created"},
                "modified": {"source": "modified"},
                "type": "text/plain",
                "tags": ["note", "externalnote", ".txt"]
            }
        }
  ]
}

This will load all text files in the ../../externalnotes/ directory into the wiki as individual tiddlers. These can be a collection of snippets in various markup-languages. Then, the type field of each of these tiddlers can be changed to match their languages For example, "text/vnd.tiddlywiki" for wikitext, or "text/markdown" for markdown files. Then, using $:/config/FileSystemPaths and $:/config/FileSystemExtentions tiddlers with the following lines we can cause any changes to these tiddlers to be saved back to the directory they started from, and also as "*.txt" files with accompanying "*.txt.meta" files. These meta files will be generated as required, and will then over-ride any fields generated from the config tiddlywiki.files file (such as the tiddler's type field) when the server is restarted.

From the examples in Customising Tiddler File Naming we see that the final [!tag[externalnote]addprefix[wiki/]] filter in the $:/config/FileSystemPaths tiddler excludes all tiddlers tagged with externalnotes (that have not matched an earlier filter). These tiddlers have their filepath retrieved from the $:/config/OriginalTiddlerPaths generated upon boot startup.

Then, the [tag[.txt]then[.txt]] filter in the $:/config/FileSystemExtensions tiddler forces all these tiddlers to be saved back to disk as *.txt and accompanying *.txt.meta files (overriding the normal tiddler-type to file-type mapping). In this case, allowing the snippets of Tiddlywiki wikitext or markdown-text to be saved back to "text" *.txt files.

Importing and auto-tagging images

This example imports all the image files in the files directory and all its subdirectories as external-image tiddlers, and tags them based on their filepath. Each tiddler is set up with the following fields:

• title - set to the URI decoded base filename of the text file
• created - set to the creation date/time of the text file
• modified - set to the modification date/time of the text file
• type - set to image/jpeg. There is currently no way to infer the correct ContentType of the image tiddler from the file, but image/jpeg tiddlers should render correctly even with png or gif images. As an alternative, you could create separate definitions for jpg, png, and gif files with the image/jpeg, image/png, and image/gif types respectively.
• tags - generated based on the path of the image relative to the parent directory (files in this case). Eg, images in files/photos will be tagged with photos, those in files/photos/family will be tagged with both photos and family, and those in the root files directory will have no tags.
• text - set to an empty string
• _canonical_uri - set to the full filepath of the image relative to the wiki root

{
    "directories": [
        {
            "path": "../../files/",
            "filesRegExp": "^.*\\.(?:jpg|jpeg|png|gif)$",
            "isTiddlerFile": false,
	        "searchSubdirectories": true,
            "fields": {
                "title": {"source": "basename-uri-decoded"},
                "created": {"source": "created"},
                "modified": {"source": "modified"},
                "type": "image/jpeg",
                "tags": { "source": "subdirectories" },
                "text": "",
                "_canonical_uri": { "source": "filepath", "prefix": "files/" }
            }
        }
    ]
}