The CL-WHO library is one of many that make it easy to generate HTML. When first checking out CL-WHO, I thought that it must be at least a couple thousand lines of code long. As it turns out, it is only several hundred. At the core of CL-WHO is with-html-output (hence the name “who”), which allows one to use a DSL for generating HTML. With-html-output works like all macros. At a high level, it takes your code in the DSL, and compiles it into Lisp code that will generate the desired HTML (here are some examples).
With-html-output does little by itself. Almost all of the work is done by three functions: tree-to-template, process-tag, and convert-tag-to-string-list. Most of the time these functions call one another recursively in order to process the entire DSL. It is possible to customize the control flow, but I will get to that later. Here is a link to a gist of the output after tracing all of the functions and using macroexpand-1 to expand a simple example. The example only shows what happens when using basic tags in CL-WHO. It doesn’t show what happens when you embed Lisp expressions in the DSL.
Tree-to-template is the entry point into the compilation process. It loops through the DSL tree, and builds up a “template”. A template is just a list of strings and expressions. The strings in the template contain HTML and are meant to be printed directly to the HTML stream. On the other hand, the expressions contain code that will print objects to that stream. Eventually all of this output put together will be the desired HTML. As tree-to-template loops through the code, if it sees a non-tag, it will just collect that into the list.1 When it does see a tag, tree-to-template calls process-tag to process it, and then concatenates the result of that into the template.
Process-tag will extract the tag as well as the attribute list. Everything after the attribute list makes up the “body” of the tag. How is the body processed? Well, process-tag takes an additional argument, body-fn, which specifies how to process the body. Process-tag will then call convert-tag-to-string-list with the tag, the attribute list, the body, and body-fn. The reason process-tag doesn’t process the body itself is that convert-tag-to-string-list is a generic function, making it possible to customize its behavior.
Convert-tag-to-string-list handles the semantics of the tag. It takes all of the arguments above and returns a list of strings and expressions. That list will become part of the template eventually returned by tree-to-template. Since convert-tag-to-string-list is a generic function, it is possible to extend it. The documentation for CL-WHO gives an example of how one could create a custom “red” tag which changes the font of the text to red, even though there is no such HTML tag. In the default case, convert-tag-to-string-list takes the result from calling body-fn on body and surrounds that with strings for the opening and closing tags. Since convert-tag-to-string-list is customizable, it is possible to change the control flow and ultimately how the body is processed. If one wanted, they could make a call to process-tag, but with a different body-fn argument, changing how the code is processed further up (down?) the tree.
With the help of these functions with-html-output converts the DSL into a template. The template is then turned into a list of valid Lisp code.2 With-html-output then wraps the body with a macrolet which binds several local macros. These macros are: htm, fmt, esc, str. These macros make it easier to print objects to the stream used for output. Check out the documentation for CL-WHO for a more detailed description of what these macros do.
I really like CL-WHO. It is a great example of an embedded DSL. A Lisp hacker still has full access to Lisp from within what is a great DSL. The only problem I have with CL-WHO is the inability to have macros expand into code for the DSL. This decreases the flexibility of CL-WHO somewhat. The only way I can see to fix this problem would be to use a library such as :hu.dwim.walker to expand all of the macros in advance.
- It actually wraps it with a let that binds *indent* in order to handle indentation properly.
- The first thing with-html-output does is pass the DSL tree to tree-to-commands. Tree-to-commands is the function that handles the entire process of converting the tree into valid Lisp code. Tree-to-commands just calls tree-to-template to do most of the work.