Declarative Programming with XSLT and LLMs

I do not believe that we are that close to true "self-coding" apps, not because of the limitations of LLMs (though these are significant) but because of limitations of human beings - the more complex the applications, the more likely that a lack of understanding about how to design a program will make a difference.

At the same time, it's worth noting that this has always been true. CASE tools were all the rage in the 1990s. In the 2000s, it seemed like you couldn't turn around without hitting yet another Javascript framework, and it seems like every time we end up switching dominant programming languages (such as from Javascript to Python), you inevitably get the growth of "Just Add Water" toolkits that will let you create simple apps with little coding expertise, but that get truly gnarly once you reach a certain level of complexity.

Sometime (indeed, many times) this is fine. Despite the fundamental sense of self-importance that many business people have, business apps for the most part are very simple: create an interface to retrieve this information in this format, create another interface to make changes to the data in a visual manner, then persist. Rinse and repeat. They are hypercard apps that look and act enough like applications to get the job done, without having to involve expensive (and maddeningly insolent) programmers.

We are entering the era of throwaway apps. This goes against the grain of every programmer out there, because the impulse of most programmers is to try to build generalized applications that can be reused for a variety of situations, but it is very common among subject domain experts who are not themselves programmers in a formal sense. However, my expectation is that for this to truly take off, you need to have the capability for managing versions.

To give an example, at some point, you want to be able to tell an agent - save this as a checkpoint, but fork it. Later, if that branch doesn't work out, you can always go back to the checkpoint and build out from there.

However, to do this, it becomes important to be able to serialize the application. Now this is a novel concept (though not that novel) - programmers are used to serializing the state of an application, but seldom the application itself. However, again, this is where HTML-like apps become more important. If you can describe functionality within established libraries but use tagged structures (such as is the case with XML or JSON) to indicate the behavior of the applications, then you can build out persistable applications.

It should be noted that building out such applications in XML or JSON is not something most developers WANT to do - not because you couldn't, but because most programmers are trained to work imperatively - we have gravitated to an Algol/C based design framework over the years, but this may actually be an impedent for AI. LLMs are much more declaratively oriented, because they are pattern matching systems, and declarative architectures (such as XSLT) are recursive pattern-centric templating languages.

For instance, consider a transformation that takes a concept and a level, and generates a tree involving subordinate concepts to the indicated level.

Thus, if you passed in the node

What you'd get back would be something like the following:

This gets presented as:

There are several key advantages to this approach. The first is that you can persist the transformation module (here XSLT, but it doesn't have to be) then load it back in at a later point. Because the code is declarative, it makes it easier for the LLM to construct internal templates for managing this output, and there are fewer dependencies on different code versions.

Additionally, you'll notice that this is agentic in that I am calling an LLM within the XSLT:

which invokes the xsl:http-request method:

It incorporated both loops and conditional logic, as well as works well recursively:

The code can also be validated and tested in scenarios without necessarily "executing" anything (this is a major advantage of declarative programming in general, by the way - you can always preflight your code.

Note also that you can use the xslt json-to-xml() function here to convert retrieved JSON content and convert it into XML, so while you still need to be mindful of namespaces, you don't need to be as mindful of which format you're dealing with - XSLT 3.0 can handle it just fine, and most LLMs are capable of understanding XSLT 3.0.

XSLT as Orchestrator and Compiler

Another key advantage of XSLT (as a declarative transformation language), is that you can chain XSLT processes together, and you can use XSLT to generate XSLT. I routinely use a two or even three pass system, where I use one process on a schema to generate an XSLT that then takes a data stream and generates an output.

This multipass approach then illustrates some of the additional benefits that can come from a transformation oriented application system.

Autogeneration

If I have a schema (let's say in SHACL, converted into XSD), I can use that with an XSLT to generate a baseline XSLT for transforming content that conforms to that schema - and I can do it dynamically. This XSLT can additionally be specialized by adding subordinate transformation match templates that can handle special cases. This in turn can generate views, tables, forms, and other visualizations.

Autogeneration

If I have a schema (let's say in SHACL, converted into XSD), I can use that with an XSLT to generate a baseline XSLT for transforming content that conforms to that schema - and I can do it dynamically. This XSLT can additionally be specialized by adding subordinate transformation match templates that can handle special cases. This in turn can generate views, tables, forms, and other visualizations.

Orchestration

XSLT 3.0 is designed for streaming. This makes it ideal for orchestration at multiple levels, first because it can be designed to handle inputs asynchronously, and because it is able to communicate with external services (that can in turn be parameterized from XML or JSON configurations). This can be especially useful when dealing with LLM-based agentic services of knowledge graph calls, or for converting knowledge graph requests into LLM friendly token streams. You can also use it in a similar manner for event-driven activities

Parsing

One of the most useful capabilities of XSLT is the <analyse-string> function. This is like a regex on steroids, as it makes it possible to take a stream and parse it in various ways in order to generate nodal data structures. Moreover, this can be used with the invisible-XML (ixml) specification to generate relevant parsers/translators.

For instance, the following iXML grammar:

can be applied to a URL such as:

to generate the XML output:

This can handle everything from parsing JSON for XML conversion to parsing WGS-84 lat-long data to parsing of emails, transcripts and similar text documents that have a semi-regular structure.

The Relationship Between XSLT and LLMs

XSLT is a tool that takes a sequence of tokens, recursively transforms them through a recursive network of responses along a graph of node templates to generate a sequence of output tokens that form one or more response documents.

An LLM is a tool that takes a sequence of tokens, recursively transforms them through a recursive network of responses along a graph of narrative threads to generate a sequence of output tokens that form one or more response documents.

Notice the similarities here.

Forget, for the moment, the term XML. That is not relevant here. I could just have readily made use of some form JSON transformer, if there were any as robust as XSLT. There aren't, but that's still not the point. XSLT is a language designed to match a particular context of information, and to then transform this context into some other form. I have used XSLT to both parse and generate old police report codes that were first in used in the 1960s. I have used XSLT in conjunction with knowledge graphs (one of the oldest graph formats was XML based, because XML actually lends itself quite well with graphs. I've used ixml with XSLT for email processing.

Here's the point I want you to consider. XSLT and LLMs are both contextual recursive descent transformers. They both create paths in graph-like spaces. They both act upon a sequence of tokens (nodes) against an information space to generate a resulting stream of tokens that represent a serialized form of graph structures. I can replace the service call in the above XSLT to one that makes a query against a knowledge graph, and you would not in fact tell that there is otherwise any difference between the two kinds of operations (other than the fact that the knowledge graph query is much, MUCH faster).

By itself this isn't necessarily that huge a factor, but this needs to be put into another context. There is a schema that specifies XSLT. This means that there are already pre-existing rules for making valid XSLT. Let me state this again to make the point:

There are already pre-existing rules for making valid XSLT.

Neural networks are pattern matchers - they work best when they have enough examples to determine what the implicit rules are. If you give neural networks the rules, you don't need anywhere near as much training data. A typical trained LLM transformer is anywhere from 7GB to around a terabyte (500 GB or more). Even a few thousand schemas - ontologies and taxonomies together, will weigh in well under 10 MB, or about 1/5,000th of the size of Literally Ludicrous Model (or LLM).

If you have schema for XSLT, the number of permutations needed to do most processing becomes much smaller, especially if that XSLT can, DNA-like, generate it's own specialized protein factories (in the form of other XSLTs that act like transfer RNAs). It is something that LLM transformers can do very well, because it is essentially the same kind of operation that are used to do recursive descent on narrative structures.

The reason that LLMs don't code very well is because they are making guesses about what the expected structure of an algorithm looks like, so they have to work on the basis of inferred metadata from all of the GitHub repositories that they have ingested (and especially from the public GitHub repositories that are filled with BAD code). They are trying to infer the behavior of imperative code without any intrinsic understanding of what imperative code does. LLMs are not Turing machines. They are approximations of Turing machines as designed by people who were trained as statisticians. Sometimes those approximations are pretty close. Sometimes they ... hallucinate.

XSLTs are perfect Turing Machines. XSLT was written by James Clarke in the late 1990s, who is a contemporary of mine (he's actually a year younger), and is a brilliant computer scientist in the old school sense of someone who understood algorithms at a very deep level. Most of the changes to XSLT since then (primarily due to the work of Jeni Tennison and, especially, Michael Kay) have refined the language considerably. XQuery emerged primarily as a response from the largely Algol/C trained programming community that XSLT was simply too weird and counterintutive to the way that they were taught to program; XSLT is simply too declarative, and has very little in the way of imperative design in it. XSLT and XQuery do, however, share a lot of the same DNA - think of them simply as different expressions of the same underlying graph/tree navigational approach.

However, imperative code, while natural for many programmers, has too much variable state locked up in various objects that simply does not work the way that LLMs work. Counting is an arduous process for an LLM, (they in essence have to create a queue of objects and reinject those objects one at a time from an external process just to create an indexed for loop) and they have to go to external services to do their underlying logic.

Declarative code, on the other hand, is a natural activity for an LLM: Create a structure, name it, when that object is referenced later, then use the underlying structure in a recursive manner. ultimately you end up with a result set that can be serialized. This means that XSLT is a very natural way for an LLM to think, because that's precisely what XSLT does. By identifying the XSLT namespace and its associated ontology, LLMs can limit the "narrative" space they have to work with dramatically. By having access to the ontological schemas and taxonomies of other structures to determine THEIR shape envelopes (what's called a Markov Blanket), this similarly means that they can perform multiple recursions to create transformers that generate transformers that generate transformers to handle ever deeper levels of abstractions.

Human beings would never do this, because they are used to giving commands. Programming in human terms is reductionist: telling computers what they should do, or providing intent. For an LLM (or XSLT on the other hand), programming involves taking a stream of data, transforming that data in some way, and generating an outgoing stream of data. Everything else is plumbing.

In Media Res,

Kurt Cagle

Editor, The Cagle Report

If you want to shoot the breeze or have a cup of virtual coffee, I have a Calendly account at https://calendly.com/theCagleReport. I am available for consulting and full-time work as an ontologist, AI/Knowledge Graph guru, and coffee maker.

I've created a Ko-fi account for voluntary contributions, either one-time or ongoing. If you find value in my articles, technical pieces, or general thoughts about work in the 21st century, please contribute something to keep me afloat so I can continue writing.