Have ideas to improve npm?Join in the discussion! »

@ndcode/jst_cache

0.1.0 • Public • Published

Build Cache wrapper for JavaScript Template system

An NDCODE project.

Overview

The jst_cache package exports a single constructor JSTCache(root, args, diag) which must be called with the new operator. The resulting cache object stores compiled JavaScript templates. Each template is, in general, a function that can be called with certain arguments, and which will generate the HTML text for a particular page or part of a page. The arguments can vary with each template.

The JSTCache object takes care of loading template source code, compiling it using the jst package, and then executing the resulting JavaScript code. This initial execution is intended to allow the template to load any resources it needs, and generally set things up so that it is ready to generate HTML. The initial execution returns the object which is stored in the cache. As mentioned this is normally a function, but could be something else, e.g. any JSON object.

See the build_cache, disk_build and jst packages for more information. The JSTCache object is essentially a wrapper object which routes the request between these packages, to ensure that the compiled template is retrieved from either RAM or disk if available, or is compiled and stored to RAM and to disk.

As well as wrapping the build_cache, disk_build and jst functionality into a convenient "point and shoot" interface for template loading, jst_cache also provides a dependency resolution service for the compiled template. If the compiled template wants to load further templates during initialization or run- time, it calls back into jst_cache, which recursively loads the dependency.

Template examples

See the jst package for more information about what can be in a template. Essentially, the templates are written in a dialect of JavaScript which allows HTML constructs to be specified directly, returning a string containing HTML.

HTML templates occuring at expression level will be completely rendered to a string, and the resulting string returned as the value of the expression.

HTML templates or strings (either single or double quoted strings, or template strings) occurring at statement level, become part of an enclosing template.

For example, the JavaScript code let text = p {'Hello, world'} sets text to the value <p>Hello, world</p>. This consists of an HTML template p { } occurring at expression level, then the string 'Hello, world' at statement level inside the p { } template (any valid JavaScript statement is allowed).

Here is a complete example showing template substitution and HTML expressions:

let lang = 'en'
let name = 'John'
console.log(
  html(lang=lang) {
    body {`Hello, ${name}`}
  }
}

Running the above program will generate the output:

<html lang="en"><body>Hello, John</body></html>

Template output buffer

It is also possible to use statement-level HTML templates or strings to build up an output buffer, which can be used for whatever purpose. The output buffer must be called _out and must be a JavaScript array of string fragments. As each output fragment (such as an opening or closing tag or an embedded string) is generated, it will be sent to the output buffer by an _out.push(...) call.

If there is dynamically generated text in the template, then it will be esacped by the sequence .replace("&", "&amp;").replace("<", "&lt;"), this is chosen so that no external dependency such as the html-entities package is needed, and so that unnecessary escaping of characters such as ' or > is not done. Attributes are done similarly, except that " is replaced instead of <. We assume a UTF-8 environment, so there is really little need for HTML entities.

For example, consider a realistic template which we use on our demo server:

html(lang=lang) {
  head {
    link(rel="stylesheet" type="text/css" href="css/styles.css") {}
  }
  body {
    p {`Hello, ${name}`}
  }
}

This compiles to the following plain JavaScript code:

_out.push("<html lang=\"" + lang.toString().replace("&", "&amp;").replace("\"", "&quot;") + "\">");
{
  _out.push("<head>");
  _out.push("<link rel=\"stylesheet\" type=\"text/css\" href=\"css/styles.css\">");
  _out.push("</head>");
}
{
  _out.push("<body>");
  {
    _out.push("<p>");
    _out.push(`Hello, ${name}`.replace("&", "&amp;").replace("<", "&lt;"));
    _out.push("</p>");
  }
  _out.push("</body>");
}
_out.push("</html>");

If invoked as an expression, the same code will be generated, but wrapped in an Immediately-Invoked Function Expression (IIFE) that creates the _out buffer, executes the above, then returns the buffer concatenated into a single string.

Template dependencies

Templates may depend on other templates. Templates are imported imported using the syntax _require(...), similarly to the syntax require(...) in CommonJS.

We use _require() for templates instead of the normal require() because:

(1) We do not want to interfere with other symbols in the namespace, you can also use require() in your templates to import ordinary CommonJS modules;

(2) Absolute paths have a different meaning in _require() versus require(), for _require() they will be taken relative to the webserver's document root;

(3) Using _require() says that the file is in *.jst format, and hence it should be parsed and any HTML constructs converted into regular JavaScript; and

(4) The template importing is an asynchronous operation, so you normally use await _require(...), whereas require() is synchronous. Because templates which have been edited are recompiled automatically by the running webserver, we don't want to stop the world, especially as templates are slow to compile.

A slight difference between CommonJS exports and JavaScript template exports, is that where CommonJS modules set the dedicated symbol module.exports to whatever is exported, JavaScript templates instead return what is exported.

Template paths and variables

Similarly to CommonJS, the variable _require is injected by jst_cache and is available to all templates. Other variables injected by jst_cache are _pathname which is the path to the currently executing template, and _root which is the root directory for absolute pathname references in _require().

Thus, paths sent to _require() are resolved relative to either _root if absolute (beginning with /) or the directory part of _pathname if relative (not beginning with /). Each template gets its own _require() function, which knows the _pathname and _root values of the calling template.

If further variables should be made available to the template, they can be included as a dictionary in the args parameter to JSTCache(). For example,

let jst_cache = new JSTCache(
  '/home/myname/www',
  {myvar: 'some value'}
)
let page = jst_cache.get('/home/myname/www/blog/page.jst')

parses and executes the template /home/myname/www/blog/page.jst, with the variables _pathname set to '/home/myname/www/blog/page.jst', _root set to '/home/myname/www', and myvar set to 'some value'. Suppose page.jst is:

let footer = _require('/site/footer.jst')

or

let footer = _require('../site/footer.jst')

then the template /home/myname/www/site/footer.jst is parsed and executed, with _pathname set to '/home/myname/www/site/footer.jst', and the other injected variables _root and myvar set from the original JSTCache() call.

Multi-part template examples

Templating with HTML text strings

Here is an example of a template which takes the body of a page (as text which may contain HTML tags), and wraps it in the standard html and body tags to create a complete page. The idea is to use this to generate every page, so that additional commands, such as viewport commands or script tags, can be set here.

Save this as page.jst, since it's the template for all pages in the system:

return body_text => html {
  head {
    link(rel="stylesheet" type="text/css" href="css/styles.css") {}
  }
  body {
    _out(body_text)
  }
}

Save this as index.html.jst, as it's the template to generate index.html:

let page = await _require('/page.jst')

return page(p {'Hello, world'})

Note that here we have somewhat hacked into the internals of the templating system by directly appending the body text onto the output buffer _out. It is intentional that you can access the internals for efficiency reasons, and that you can, for example, avoid HTML escaping of particular text where appropriate.

Templating with callbacks

Another way, which is more sophisticated, is to use a callback function to generate the body. If doing this, we pass the output buffer _out into the callback function, so that the entire HTML of the page can be generated in an unbroken stream, without having to concatenate the partial page repeatedly.

The page wrapper template page.jst which takes a body callback and calls it:

return body_callback => html {
  head {
    link(rel="stylesheet" type="text/css" href="css/styles.css") {}
  }
  body {
    body_callback(_out)
  }
}

Then the specific page template index.html.jst which provides the callback:

let page = await _require('/page.jst')

return html(
  _out => {
    p {'Hello, world'}
  }
)

Note that in the above example, the { } around the callback function body is essential, unlike in regular JavaScript, so that p {...} occurs in statement rather than expression context. As an expression it would return a string which would be ignored, since the page template is expecting output to be in _out.

File management

The JSTCache object a "point and shoot" interface which makes it very easy to manage on-disk templates. You just call JSTCache.get() stating a pathname to a template, normally a *.jst file, and the template is parsed, compiled to JavaScript, and evaluated, with the evaluation result cached for future reuse.

What the JSTCache.get() function returns depends on the template source code. Template exports are similar but slightly different to CommonJS module exports, as noted in examples above. Usually, JSTCache.get() returns a JavaScript function, which you call to generate HTML each time the page is to be served.

The HTML-generating function is re-useable in this way for efficiency reasons, since after the initial compilation, the cached version can be reused each time the page is served, but called with different arguments, resulting in different substitutions made on the page. For example, consider the following template:

return (lang, name) => html(lang=lang) {
  body {
    p {`Hello, ${name}`}
  }
}

Suppose the above is saved as /hello/index.html.jst under your document root /var/www/html. When you instantiate the JSTCache object giving the document root, and then call the get(pathname) instance function, you receive a function of two arguments lang and name, which returns the HTML string.

Note that _require() paths beginning with / are taken relative to the document root passed into the JSTCache() constructor call, whereas paths beginning with anything else are taken relative to the dirname extracted from the current value of _pathname as seen by the template. Each template has its own _pathname and thus its _require() calls are relative to its own source.

Memory vs disk caching of templates

Templates compiled using JSTCache.get() are cached in memory, as long as the same node interpreter is running, so that they can be retrieved using either JSTCache.get(), or equivalently _require() inside a template, and they will not be re-executed. In the above example, if you call JSTCache.get() twice, you get the same object twice (it is a 2-argument function returning String).

As well as this, the compiled templates are also cached on disk, which requires the document root to be writeable. For instance if you compile index.html.jst in a hello/ subdirectory of the document root, you get a hidden file with the same name prefixed by . in the same directory, here hello/.index.html.jst.

The main reason for the disk caching is really to fool the node.js require() system into using correct relative paths, if the template uses require() as well as _require(). However, it is also handy that the templates only need to be re-evaluated and not recompiled if the webserver is stopped and restarted.

Before using either a disk-cached or a memory-cached template, the modification times are checked and the template is recompiled if it is stale. Thus, you can edit your website while it is live, and each page will be recompiled as needed just before serving. Templates deleted on disk are not returned from the cache.

Note: If the document root is not writeable, a simple expedient is to make sure that all .*.jst files are up to date, so no write is attempted. You could do this by firing up a development instance of the server in a writeable directory and then indexing the whole site, for example by a recursive wget invocation.

To be implemented

It is intended that we will shortly add a timer function (or possibly just a function that the user should call periodically) to flush built templates from the cache after a stale time, on the assumption that the template might not be accessible or wanted anymore. For example, if the templates are HTML pages, the link structure of the site may have changed to make some pages inaccessible.

GIT repository

The development version can be cloned, downloaded, or browsed with gitweb at: https://git.ndcode.org/public/jst_cache.git

License

All of our NPM packages are MIT licensed, please see LICENSE in the repository.

Contributions

The jst system is under active development (and is part of a larger project that is also under development) and thus the API is tentative. Please go ahead and incorporate the system into your project, or try out our example webserver built on the system, subject to the caution that the API could change. Please send us your experience and feedback, and let us know of improvements you make.

Contact: Nick Downing nick@ndcode.org

Install

npm i @ndcode/jst_cache

DownloadsWeekly Downloads

1

Version

0.1.0

License

MIT

Unpacked Size

19.2 kB

Total Files

4

Last publish

Collaborators

  • avatar