Justin Fagnani

What should a native DOM templating API look like?

Justin Fagnani — Mon, 30 Jun 2025 08:00:00 GMT

If you read my previous post, The time is right for a DOM templating API, you might be wondering what such an API would look like.

Let's dive into that question now.

What are we building?

First, let's clarify what we're trying to design here, because when people hear the abstract template API idea described, before there's a concrete proposal or examples, they can sometimes think of very different things.

In webcomponents/1069 I propose that we add a "declarative JavaScript templating API"

Declarative means that the developer is describing what they want, not how to achieve it. What exactly constitutes "declarative" is often debatable, and under every declarative API is an imperative implementation, but hopefully people can align on the spirit of the word.

By "templating" we mean the ability to describe the DOM that the developer wants to create and update in a form that resembles the output. This includes syntaxes like JSX and lit-html, which for some people count as something other than templating. For our purposes, they are.

Taken together, "declarative DOM templating" is something that is very, very common. All major frameworks today have declarative templating at their core.

But we're going to narrow our focus a bit JavaScript APIs.

By "JavaScript API" we mean methods, classes, etc. that are part of the overall DOM JavaScript API surface. Something that's a sibling and alternative to innerHTML, importNode(), and a bunch of other DOM APIs that web developers and frameworks use to create and update DOM. But also, we mean that script-like abilities will be left to JavaScript, not added to markup.

It's also important to not that we're not talking about components at all, just templated DOM creation and updating. Most frameworks intertwine the two concepts, so some people aren't aware that there can be a distinction. One of the use cases for this proposal is to be a primitive that frameworks can use, so it's necessary that components can be added on top of templates, but any such feature to do that should be as generic and minimal as possible.

What's guiding us?

In my experience, we can derive the general API shape for this feature from its constraints and requirements, with a dash of opinion about good APIs on top. Listing these helps us see what things must be a certain way vs which things can flex as part of the decision space for the problem.

Constraints

Because we're designing a native DOM API, and not a framework, we have more constraints to deal with. A native API has to work within the current platform, can't break existing sites, and has to be viable within the standards process.

Web compatibility: The proposal can't introduce any breaking changes to the platform.
Standard file types: While a proposal could introduce a new file type just for templates, that's unlikely to make it through the web standards process without incredibly compelling reasons. We'll take using the existing file types as a hard constraint.
Standard JavaScript: It's also possible to try to simultaneously introduce changes to both the DOM and JavaScript, but this is also less likely to succeed because we would be dealing with two separate standards bodies and a much larger overall addition to the platform.
JavaScript is not just a web language anymore, and the JavaScript side would likely have to be useful for non-web use-cases to move forward, dramatically raising the complexity of a proposal.
So we'll also take it as a hard constraint that this API works with today's standard JavaScript.
Low implementation security risk: The proposal should not require drastic and potentially-unsafe HTML parser changes. Implementors are just extremely reluctant to make large HTML parser changes, since they are prime targets for attackers and previous changes (like <template>) did introduce security bugs.
No performance regressions on existing sites. Slowing down existing sites, or benchmarks like Speedometer, will sink a proposal when it's implemented.
Security: New APIs have a much higher security bar to clear than previous platform changes. They need to be secure by default.
No required compiler: This is not universal, but many web platform maintainers insist that web APIs should be directly usable by developers and not require a compiler to be useful. This does not preclude an API from being a good compile target for alternate syntaxes.
Coherence with the platform. Also not universal, but it will be drastically easier to get a proposal accepted if it's coherent with the existing platform. This would prevent us from introducing a new "better DOM", for instance.

Requirements

Next we need to look at our requirements. What does this proposal have to do to be useful and successful? The actual proposal will need to detail its target use cases, but we can list some requirements now:

Ergonomic API: This API should be nice to use directly for developers.
Usable from JavaScript libraries: Libraries should be able to adopt this API as an internal implementation helper.
JavaScript for control flow and expressions: We don't want to introduce a new JavaScript-like expression language. That would increase the size of the proposal, be less likely to get agreement on, and introduce security concerns.
It would also introduce a different lexical environment that developers would have to get their data into. It's much easier on developers and tooling if template expressions exist within the same lexical scope as their data.
Ability to set attributes, properties, and event listeners: DOM elements have three major key/value style APIs that developers are used to setting declaratively from templates. Because these APIs are truly independent in the DOM, we require the ability to set them each unambiguously.
Support full HTML syntax: elements, attributes, children, comments, etc. We need to be able to create any structure that can be created with HTML.
Security: More specific than the platform constraint, we need templates to be safe from XSS and gadget attacks, etc.
Performance: The API must be fast for initial rendering and updates with DOM stability.
Composability: Developers should be able to build templates from pieces, and abstract over templates and rendering.
Extensibility: One template API is not going to be able to have every feature that all developers need. It needs to have hooks for extending the system with new behavior.
Open to evolution and extension of HTML and the DOM: The system should assume that the platform will evolve and that elements, attributes, and events can be added in userland. It should avoid special casing specific DOM elements and attributes.

Opinions

Then we have opinions, which people might or might not agree with. Things that make the API good, elegant, maximally general and flexible. These are my personal opinions, but I hope they're very reasonable and shared by a lot of other people.

The APIs should work with functional-style programming. One of the motivating reasons to have a declarative template API is to reduce the amount of imperative code need to build UIs. An API compatible with function-style programming can still be used in imperative contexts.
Composition should be a core feature that other features are built on. For example, instead of specific and special constructs for conditionals and loops, we should be able to rely on JavaScript and composition to dynamically construct a template.
The underlying DOM creation approach should be HTML template cloning. HTML templates - the <template> tag has a few performance advantages over other DOM creation approaches. <template>s can use a simpler HTML parser, and cloning nodes is faster than re-parsing HTML as needed with innerHTML.
The API should support support multiple models of reactivity and DOM updates, possibly through userland extension.
- Template re-rendering should be a core supported feature because it is extremely fast and generalizes to many types of data.
- Fine-grained reactivity should be supported for native observable data types, which will hopefully soon include Signals.
- DOM diffing with various algorithms should be possible to add with userland utilities.
The API should have good layering. DOM update mechanisms should utilize lower-level platform APIs, like DOM Parts and a tree-aware task scheduler, that can be used by libraries that aren't using the templating API.
The API should not introduce any new component abstractions, but it should have stateful hooks that allow a framework to easily attach their own component instances to the DOM.

Template syntax

In my previous post I made the claim that we know what good template syntax looks like. While some people took exception with that claim, I stand by it because of how fundamentally similar all the popular template syntaxes are. The claim is even stronger when considering JavaScript-based APIs.

The core similarity is that HTML templates are all markup with interpolations and control flow.

Compare:

React

<>
  <h1>Hello {name}</h1>
  <button onClick={handleClick}>Click Me</button>
  <ul>
    {items.map((i) => (<li>{i}</li>))}
  </ul>
</>

Vue

<template>
  <h1>{{ name }}</h1>
  <button @click="handleClick">Click Me</button>
  <ul>
    <li v-for="item in items">
      {{ item.message }}
    </li>
  </ul>
</template>

Angular

  <h1>{{ name }}</h1>
  <button (click)="handleClick()">Click Me</button>
  <ul>
    @for (item of items) {
      <li>
        {{ item.message }}
      </li>
    }
  </ul>

Lit

html`
  <h1>Hello ${name}</h1>
  <button @click=${handleClick}>Click Me</button>
  <ul>
    ${items.map((i) => html`<li>${i}</li>`)}
  </ul>
`

Polymer

<template>
  <h1>{{ name }}</h1>
  <button on-click="handleClick">Click Me</button>
  <ul>
    <template is="dom-repeat" items="{{ items }}"">
      <li>
        {{ item.message }}
      </li>
    </template>
  </ul>
</template>

FAST

html<MyElement>`
  <h1>Hello ${x => name}</h1>
  <button @click=${(x, c) => x.handleClick(c.event)}>Click Me</button>
  <ul>
    ${repeat(x => x.items, html<string>`<li>${i => i}</li>`)}
  </ul>
`

To me, these are in fact very similar:

The HTML portion is described in markup
Bindings can appear in text and attribute value positions
Bindings expressions are (usually) delimited by some kind of separator
Control flow can conditionally render a subtree/sub-template

Yes, there are some differences.

Different expression delimiters
Some systems delimit all expressions, some don't in certain cases (bound attributes)
Expression language differences
Property and event listener delineation
Control flow: attributes (directives), templates, non-markup syntax, control-flow-is-JS-expressions
Some template-literal-based systems require bindings to contain closures.

I personally don't think these differences are actually all that big. Most of them are surface syntax.

The biggest difference in my mind are:

Whether the syntax is HTML-based or JavaScript-based
Control flow syntax and placement among the HTML-based syntaxes
Whether bindings require closures in JavaScript-based syntaxes

When we narrow our choices based on the fact that we and proposing a JavaScript API, most of the differences fall away. Expressions are JS, control flow is just expressions, expression delimiters are fixed.

Template expressions

One of my strongly held opinions in this area is that templates should be side-effect free expressions that return a value that describes the intended DOM. This is a functional-style API that's extremely flexible and simple. This is what most JSX transforms (ie, React's) do, what Lit does, and most other templates-in-JS systems do.

In addition, I believe that template expressions should be able to be re-evaluated to generate a new description of DOM, also how React and Lit work. This ensures that any data available in the lexical scope of the template can be consumed by templates, and any trigger that indicates that data has changed can be used to initiate a template re-evaluation.

The approach of expressions generating descriptions of DOM lends itself to a wide variety of powerful techniques.

Abstraction: template results can be evaluated and returned by functions, accepted by functions, and computed over. It enables patterns like higher-order templates and render props.
Transformation: Utilities can walk over the DOM description and transform it into a different description.
Lazy evaluation: template expressions naturally work with lazily evaluated JavaScript constructs like ternaries, short-circuiting operators, defaults for parameters and object destructuring.
Alternate render functions: The process of rendering a DOM description isn't locked off to platform code. Alternate ways of applying a description to live DOM, like DOM morphing libraries, can take these DOM descriptions and render them in userland. This can even be a means of polyfill future new features of the template system.

Alternatives to template expressions

Static template definitions. This is how FAST, Polymer, Angular, and some others work. A template is a property of a component class, and is interpreted by the component instances. I believe this ties templates and components too closely together for a native API, and instead of just accessing variables in a JavaScript lexical scope (ie, function scope or class instance scope in a method) expressions have to be bound to some scope object, which make this not play very well with plain functions and functional style programming.
Side-effectful statements. These mutate DOM as they are evaluated. Google's internal framework, Wiz, worked this way when I left the company. It seemed to me to cause a lot friction for abstraction, as there was no way to programmatically access the template definition or result. The best you could do is pass around an opaque closure with the instructions in it.

Template container syntax.

Aside from the internal template syntax, like bindings and such, we need a container for templates. Let's look at the relevant constraints and requirements:

No compiler required, only standard syntax
Templates resemble output, ie markup
Full support for HTML

This first constraint rules out using JSX (until and unless JSX becomes a standard). And without JSX, the next two items pretty much require that our templates be contained in strings. Nested function calls (like React's createElement() or Hyperscript's h()) or object-based builder-style APIs just don't look enough like HTML.

So we want the HTML portion of a template to be strings, like:

'<h1>Hello World</h1>'

But we also need a way to include expressions, and one of our requirements is that we use JavaScript instead of a new expression language.

This rules out something like:

'<h1>Hello {{ name }}</h1>'

because this {{ name }} bit is inside the string and can't reference variables in the containing JavaScript scope. You could try to do a format-string style of API with a separate data bag of named parameters:

render('<h1>Hello {{ name }}</h1>', {name: 'World'})

but that's pretty ugly, and JavaScript has much a nicer syntax for doing this same thing already!

Tagged template literals

The most straightforward way to embed markup in standard JavaScript is with tagged template literals. In fact, embedding a DSL like HTML in JavaScript is exactly what tagged template literals were designed for - and they have some special features to make doing so efficient and safe.

We'll call our tag the most obvious thing: html. Templates expressions are then template literals tagged with the html tag:

html`<div>Hello World</div>`

This ends up looking a lot like JSX, but with the markup inside of a string instead of as part of the outer language grammar.

With tagged template literals, our expression delimiter syntax is chosen for us: ${}. So expressions look like:

html`<div>Hello ${name}</div>`

Benefits of tagged template literals

It's worth taking a moment to talk about tagged template literals, because I think they're one of the least appreciated and understood parts of ES2015.

Like I said above, tagged template literals were specifically designed to enable embedding of external DSLs (like HTML, SQL, or GraphQL) into JavaScript, and they have a few features that make this possible.

A tagged template literal is prefixed by a reference to a tag function, and a tag function has this signature:

(strings: TemplateStringsArray, ...values: Array<unknown>) => R;

(where R is the return type of the specific tag function)

The important features for DSLs:

The tag function receives the static strings and values separately.
This means that we can process the static strings and values separately, which is great for both performance and security. We can skip most work for the static strings, and use them to make a cloneable <template> element. We trust the static strings as developer-authored and allow them to create HTML; we distrust the values as potentially user-controlled and make the browser escape them before inserting into the DOM.
The tag function can return any type of value, not only strings.
This means that while template expressions may look a lot of string interpolation, they are quite a bit more powerful. Our html tag will simply return an object holding the static strings and the values.
The static strings array passed to the tag function is the same array instance every time a tagged template literal is evaluated.
This is a little subtle. If you have a tagged template literal expression that can be evaluated multiple time, say if it's in a function that's called multiple times, that the tag function will receive the same strings array on every evaluation.
Here's a simple example:
```
// A template tag function that simply returns the strings array
const tag = (strings) => string;
// A function that evaluates returns the result of tag
const run = (x) => tag`abc ${x} def`;
run(1) === run(2); // true
```
This means that we can perform one-time setup work based on the strings, and then use the strings array as a cache key to retrieve that work on future evaluations. We can also use the strings array as a key to check whether a template has already been been rendered to a DOM location so that we only update the changed values.
The tag can identify the embedded language. Any template literal using the tag html contains HTML. Lots of IDEs and other tools key off of the tag name to automatically give developers syntax highlighting and other checks.

Downsides of tagged template literals

Of course, tagged template literals aren't perfect, otherwise there's a good chance that the React team would have used them instead of inventing JSX. Two things that may or may not matter much depending on your personal preferences are related to verbosity:

More symbols than JSX for templates and bindings. The html tag and backticks are 6 more characters per template expression than JSX with one root element, though only one more character for a JSX expression wrapped in a fragment. The binding delimiter of ${} is one more character than {}.
Verbose for userland components. This one is more significant. In tagged template literals, element names are strings in the static side of the template. This means they can't hold JavaScript references to component definitions. String tage names work well with custom elements because components, like <my-element>, are registered by name. For JSX-style references you would need to use binding syntax like <${MyComponent}>.
This is why it's important that this template API can be a good compile target for JSX. If we want userland component systems to be able to use the API, they will need to allow their users to write templates in their most preferred syntax, and can then compile to something that may require more characters.
By the way, JSX's convention of capitalized tag names being JS references and lower-cased names being string literals is one of my biggest worries about the standardization of JSX. I haven't heard anyone on TC39 opine on this yet, but it seems like the kind of odd rule that works well enough in practice, but that some people will object to. It's entirely possible that to make it through standardization a proposal would need a more explicit differentiator between strings and references anyway.

Aren't strings bad?

Some people critique lit-html and similar APIs as being too "string based". The complaint being that a string can contain anything and isn't checked for correctness by JavaScript parsers. I think this complaint is usually overblown. All programming languages are strings. JavaScript, HTML, and CSS are strings. What makes them structured languages are their grammars and tools that understand them.

This holds true for these tagged template literals. There is a syntax and grammar to them, which is just standard HTML and can be checked by tools and runtimes. Developers who use these style of templates today usually add editor and compiler extensions to their toolchain to give them all the benefits of good PL tools support: syntax highlighting, syntax checking, type-checking, intellisense, linting, and formatting.

The biggest difference between an embedded syntax like this and an extension syntax like JSX is that in the actual browser runtime syntax errors will happen at template expression evaluation time rather than module parse time. This difference should be mitigated by edit and built-time tools that check template syntax.

Bindings and template parts

Because we have static template vs dynamic expression separation, we can mark exactly which portions of the DOM will change and which won't. Expressions in templates - really the gaps where expressions go - create DOM Parts that we can update with new values.

DOM Parts are a proposal for a new DOM object that can be attached to a specific location in the DOM and updated over time. It's a lower-level templating feature that will need to be worked on as part of any proposal here. A goal with DOM Parts is being usable by frameworks and template libraries. If a framework can't take advantage of the template API for some reason, hopefully it can use the DOM Parts APIs directly.

Attributes, properties, and events

After we have our container syntax, we have another bit of template system to figure out: how to differentiate between attribute, properties, and events.

Attribute, properties, and events are three separate key/value-style namespaces on elements that developers expect to be able to set declaratively from templates.

Frameworks and templating libraries take different approaches to this. Some try to merge them into a single namespace and differentiate them with runtime introspection, conventions, and/or special case lists. Others disambiguate with prefixes or sigils on the attribute name to signify properties and events.

Some examples:

React: special case lists, runtime introspection, and on prefix for events.
Vue: runtime introspection and specific syntax.
Angular: specific syntax
Lit: specific syntax

Runtime introspection is potentially error-prone, and there an element could potentially have an attribute, property, and event of the same name and require specific disambiguation to setup properly.

Vue and Lit both use a similar shorthand syntax for properties and events:

. prefix for setting properties.
@ prefix for setting event listeners

In addition Lit has a useful ? prefix for opting into boolean attribute behavior.

I think this is a nice, concise, and pretty intuitive syntax. A proposal will go into more detail on the rationale for this choice, but here's what it would look like in a template:

html`
  <input
    name="foo"
    .value=${v}
    ?required=${isRequired}
    @input=${handleInput}>
`

Isn't this not "just HTML" anymore?

These sigils do give special meaning to attribute that plain HTML doesn't have, but they entirely valid standard HTML. So the syntax is standard while the semantics have been augmented for our DOM templating purposes.

Composition, conditionals, and looping

Any template system needs control flow and composition capabilities. One thing that React and JSX taught us is that control flow can be implemented in terms of composition, leaving all control flow features to JavaScript.

Start with simple composition: a template expression can contain a reference to another template:

const hello = html`<span>Hello</span>`;
const message = () => html`<h1>${hello} World!</h1>`;

And now you can use JavaScript for conditionals:

const hello = html`<span>Hello</span>`;
const goodbye = html`<span>Goodbye</span>`;

const message = (isLeaving) => html`<h1>${isLeaving ? goodbye : hello} World!</h1>`;

Like JSX, ternaries aren't the only conditional you can use. Any conditional expression or function works.

Then we need to support nesting lists of nested-templates as well:

const renderList = () => html`
  <ul>
    ${[
      html`<li>One</li>`,
      html`<li>Two</li>`
    ]}
  </ul>
`;

And we get looping:

const renderList = () => html`
  <ul>
    ${items.map((i) => html`<li>${i}</li>`)}
  </ul>
`;

Updating and reactivity

Once we have a way to express templates we need a way to actually render them to the DOM, and like my last post pointed out, it's critical that we have ways to efficiently update that DOM from the same template later.

The React and Lit render() API for this is pretty simple, and would work well for a built-in API.

Those APIs take a renderable value and a container to render into. But since we're imagining a native API, we can introduce a new render() method on Element that takes a template result:

const go = (name) => containerEl.render(html`<h1>Hello ${name}!</h1>`);

Updates can work in two ways: template re-rendering, and fine-grained updates.

Template re-rendering

Template re-rendering works by recording what template was rendered to a DOM location, and checking any new template to be rendered there against the previous template. If they're different, then the DOM is cleared and the new template is rendered. If they're the same, then just the dynamic values that changed are updated.

This is quite often the same result you get from VDOM diffing, especially if you key your top-level elements in a JSX fragment. It's essentially keying the fragment by the template identity.

This approach is a lot more efficient than VDOM diffing because it's a single identity check for the template instance, then a dirty check for each value. We're diffing on the value side, and by running through a linear list of new and old values rather than traversing a VDOM tree.

Fine-grained reactivity

Fine-grained reactivity approaches, like signals, can be even more efficient in some cases. And they can have ergonomic benefits if your shared data is inside of observable containers as far-flung uses of that data will all update when the data changes.

There's a proposal to add Signals to JavaScript. If that moves forward, signals could be easily supported within templates for fine-grained reactivity:

const name = new Signal.State('Fred');

containerEl.render(html`<h1>Hello ${name}!</h1>`);
// DOM: <h1>Hello Fred!</h1>

name.set('Ambrose');
// DOM: <h1>Hello Ambrose!</h1>

There are a lot of important details to work out around batching and scheduling of updates and efficient list updating, but I think it's important to have a path forward to built-in fine-grained reactivity. I've seen a lot of web developers asking for something like this.

Why not JSX again?

The big reason not to pursue standardizing JSX as part of this proposal is that it would make the proposal much larger and much harder to move forward.

I do hope that TC39 tackles some addition to JavaScript that meets the JSX demand, and in that case the templating API should accept whatever data type JSX expressions produce. I hope that if a templating API is added to the platform before JSX that it would really help inform the capabilities we need out of a JSX-like addition.

In the meantime, people can develop and use JSX transforms for the template API, and that's an important goal of this API idea. This has already been done for Lit, and Preact's html library should that you can make the JSX->template literal association in the other direction too.

Why not templates in HTML itself?

We already have the <template> element, but <template> only supports cloning its inert contents, and a lot of developers want something more powerful like data-binding support.

I also think that we can do this, but it's a larger task than a JavaScript API - we'd have to specify binding syntax, expressions, and control flow that are already done for us in JavaScript.

Isn't this just standardizing lit-html?

Honestly? Yes, and no. Though I think it's more accurate to say that this proposal and lit-html look similar because they share similar constraints and goals, and this API shape is the most obvious way to accomplish them.

The reason why lit-html looks the way it does is precisely because the Polymer team had these same constraints and requirements that the web platform does. This was partly the disposition of the team members at the time, but largely because as part of the Chrome organization, that was our mission. A large part of our job was to polyfill and prototype upcoming standards, and experiment with things that potentially could be standards proposals one day.

Once Polymer moved from using HTML Imports to JavaScript modules, its HTML-based template system became a big drag. Compared to a more JavaScript-oriented solution, it was more code to implement, had more custom concepts and cognitive overhead, slower in many cases, and created a division between JavaScript and template lexical environments that added friction to development.

lit-html was our response to those issues, still working within our constraints of no required compiler, no forking the web's core languages, and potentially standardizable features and API shapes. This simply led us to the same place that this proposal is going.

And we weren't the only ones. Preact's htm library, Microsoft's FAST, and HyperHtml look extremely similar, for similar reasons.

In fact, I'm not aware of a client-side templating library that meets these constraints and requirements and doesn't look a lot like lit-html (or HyperHTML or htm or FAST). I would love to see examples if there are any.

So it seems like just a natural way to do things given the way JavaScript and the DOM work. I think that's a strong indicator of the style of API being ready for standardization.

Not everything in lit-html is standardizable in my opinion, and any proposal here should be derived on its own from its own requirements, not copied from lit-html, so I don't think this proposal would look exactly like lit-html. But if this API were added to the platform I do think it would largely obsolete lit-html. But that's also one of the goals of the Lit née Polymer team: to eventually be obsoleted by the platform itself.

Summary

I think that's a pretty good overview of the type of API I think that the web should standardize on. The end result is something that looks and feels like the templating portions of React and other popular frameworks in a lot of ways, but works with the standard JavaScript we have today.

This can be a controversial area of web development to wade into, so any proposal here is going to have to be extremely well researched, argued, socialized, and detailed early to have any hope of success. I hope to have time to push these ideas forward over the next few months.

If you're interested in collaborating, please let me know by commenting on webcomponents#1069, or reach out to me on Bluesky!

likes on Bluesky

The time is right for a DOM templating API

Justin Fagnani — Thu, 26 Jun 2025 08:00:00 GMT

Update: See my follow up post, What should a native DOM templating API look like?.
TL;DR: I want to propose adding a declarative templating API to the web platform. Here's why...

The web platform is the most successful application runtime of all time. While the largest reason for this is the web's reach, it wouldn't be possible without the DOM API, which turns a mostly static document viewer into a highly dynamic and expressive runtime.

For as much hate as the DOM sometimes receives (some of that deserved, but some really not!) the DOM is undeniably a very powerful API. This is easily shown by all the amazing and highly dynamic web applications out there - even an app as complex as Photoshop is available as a web app, and its entire UI is built with web components!

Yet the DOM is missing one of the most important and popular features in modern web development: templating. Currently there is no ergonomic way in the standard DOM APIs to create a chunk of DOM nodes, from data, event listeners added, properties set on elements, safe against XSS attacks; and then to efficiently update that chunk with new data.

This kind of templating is the cornerstone of all modern web frameworks and rendering libraries these days, all of which let you declaratively combine markup with data. This includes React, Vue, Angular, Preact, Lit, Svelte, SolidJS, Stencil, Quik, Ember, FAST, Polymer, Marko, and just about every other framework out there.

The reasons that declarative templating are so popular and foundational may be obvious, but let's recount them anyway:

The ergonomics are far superior to imperative DOM APIs. Declarative interpolation - plus features that go beyond plain HTML, like event listeners and property setting - help preserve the locality of behavior, making templates much easier to write and read than imperative code.
It's easier to be secure against XSS attacks, since templates can automatically escape interpolations.
Performance can be faster than all but the best hand-written code. Good template systems take great care to update only the DOM that needs to change.
Static analysis is easier. Most template systems support some kind of type-checking and intellisense. Many similar static analysis passes are much more difficult to do with imperative APIs.
Efficient server-side rendering becomes possible, due to the ability to interpret declarative template definitions outside of the full browser environment.

It's rare to see a technique that wins so completely on so many axes; that's good for DX and UX, for security, and portability, and is so extremely popular as to be ubiquitous.

So templating is great, but what's the problem with the current situation?

The biggest thing to me is that there's an extremely core need of nearly all web applications that's just not being met by the platform. The web platform should adapt to meet such clearly demonstrated developer needs, and in the case of templating it hasn't yet. And it's such an obvious thing to add, similar to how internationalization and a great datetime API are now part of the platform, or how scheduler and sanitizer APIs are hopefully coming soon too.

But we also have real consequences of the lack of templating, it's bad for everyone:

Users suffer from longer app download times, rendering overhead, and/or insecure apps. They often need to wait for library code to download before their application is usable. For templating, this can range from only a few kB in the best cases, to upwards of 100kB or more in the worst. A few kB might not seem that bad, but it can be a pretty large portion of an ideal budget for initial interactive rendering of an app.

Developers need to reach for a library, and thus tools like npm or a CDN, to do many basic things. This adds to the overhead of getting started. It makes simple static files and devtools less useful than they could be. There's no fundamental templating knowledge that's portable between stacks, and native DOM creation APIs like innerHTML are unsafe by default.

Frameworks are harder to build because they have to implement templating. Templating libraries are challenging to build and maintain because they're under massive constraints: they must be ergonomic, fast to render, fast to update, secure, and as small as possible. Libraries very often have to make hard trade-offs between features, performance, and code size.

And the platform suffers from this friction as it competes with native platforms that are far less sensitive to a few kB here and there. Competing platforms like Flutter, SwiftUI, Jetpack Compose, etc. not only all have templating-like systems built-in, but they work on app installs, use compiled languages, and also don't have to worry as much about things like a few kB in app bundle size.

Why right now is a great time to add templating

There have been proposals to add things close to templating to the platform before. E4X was a big one that actually shipped in Firefox and Flash. E4H was a simplification of that idea by Ian Hixie that I can't find any references for anymore. At some point around 2012 there were even prototypes of a built-in `html` template-literal tag. Those failed, and for the better I think, because they might have helped with creating DOM, but not with updating it.

But now I think we can revisit this feature. In fact, I think that right now is a particularly good time to address this situation and add a templating API to the DOM.

A big reason why I think we can do this now is that frameworks have worn great cowpaths to pave. And while it's true that there's a lot of variety in userland template solutions, they're actually very similar in a lot of ways, and they seem to be getting more similar over time. We understand ergonomic template syntaxes and reactivity models a lot better than 13 years ago.

But also, not everyone uses frameworks and there's a lot of pent-up developer demand for ergonomic DOM manipulation and reactivity APIs from so-called "vanilla" developers and the web components communities.

There are in-flight proposals for very low-level DOM update primitives, like DOM Parts, which target framework implementations, but I think higher-level APIs like full declarative templating can take even more load off, help prove out and complete the lower-level API proposals, and be really impactful for developers and users.

We know what good syntax for templating looks like

The popular client-side template systems out there all share fundamentally similar syntax of markup with bindings. While there are small differences in expression delimiters, control flow constructs, binding type designators; the similarities are far greater than the differences. Even across the HTML-based (Vue, Angular, Svelte) and JavaScript-based (React, Lit, Solid) spectrum.

Things get even more similar when you look at JavaScript-based APIs (which any new DOM API would be): templates are usually expressions, composition is done via nesting template expressions or references to them, and control flow is just JavaScript.

There's also a lot of similarity in semantics. The most popular JavaScript-API-based systems return a description of the DOM from a template expression, then apply that description with a separate render function call. Most of those also update the DOM with the same render function call (though some eschew that altogether for fine-grained reactivity).

Previous platform proposals didn't match these semantics. Both E4X and E4H added new syntax to JavaScript to embed markup in JavaScript similar to JSX, but they critically didn't return a description of DOM, but a fragment of DOM itself. This made them good for initial DOM creation, but bad for updates, which were still imperative.

We can build an API for today's JavaScript

It's been here for a while, but it's worth reminding readers: we have a built-in JavaScript feature for embedded DLS like HTML: tagged template literals.

So we can describe templates in a DOM API without having to add new features to JavaScript. And we have popular libraries that prove this approach works well. That's a big deal for realistically being able to move a proposal forward.

We can also plan for tomorrow's JavaScript

Tagged template literals are great - they're an often unsung hero feature of ES2015 - but they have a bit more syntactic overhead than JSX, especially with userland component systems. And, people love JSX.

So some people might ask: why don't we just add JSX to JavaScript too?

Well, the problem with JSX is that it's all syntax and no semantics. This means that in order to standardize it we can't just add new syntax to JavaScript, we have to define the semantics. Treating JSX as a call to an ambiently-defined createElement() call is out of the question. Creating a tree of objects is possible, but that's one of the least efficient ways to use JSX as it irretrievably mixes static unchangeable parts of the tree with dynamic parts of the tree, precluding more efficient update patterns. If the Records and Tuples proposal were progressing, JSX could maybe create Records with boxes, but that proposal has been stalled, especially on the record identity and box parts that would make it suitable for a JSX semantics.

Nailing down the semantics is already probably the hardest part of a hypothetical JSX proposal, but there's also a chicken-and-egg problem when it comes to templating. In order to ensure that a JSX-like syntax and semantics is suitable for full-expressive DOM templating, we need a target templating system to verify it against. It might seem like React already provides this, but in some ways React's templating is less powerful and expressive than other systems. For instance, React doesn't provide a way to explicitly bind to properties and events of DOM elements, or provide directives that apply to an element.

But again, people love JSX, and JSX has absolutely proven the popularity of embedding markup in JavaScript. So I do believe that the platform should eventually take a look at adding something like it. I'm just a realist about how much work that is and think that we can do templating with today's JavaScript much sooner.

Luckily, the good thing about today's non-standard JSX is that it's pure syntax with no runtime semantics. This means that we can build JSX-to-tagged-template-literal compilers, as many have done for things like JSX-to-Lit. A transform for JSX to a native template API would make the template API a compile target, where developers could choose a syntax they prefer, with little-to-no runtime overhead charged against users.

Then, if work to standardize JSX is picked up by someone, it'll have a concrete system to target. The template system may have to adapt to accept the data type that a JSX expression evaluates to, but this is eminently possible, and a better direction to adapt in that having a native JSX syntax and semantics that we later realize are insufficient for native templating.

We can lay the groundwork for HTML-based templating

Many web developers have been asking for a full-features HTML-based templating system. This is across communities like web components users, framework users, and "vanilla" developers. Many developers expected this kind of templating from the `<template>` element, or from declarative shadow DOM, or web components in general, and are disappointed that it's not the case. And a very large fraction of frameworks use HTML-based templates instead of JavaScript-based templates: Vue, Angular, Polymer, Svelte…

So they might ask why we don't pursue an HTML-based API first instead of a JavaScript-based API?

My answer would be that an HTML-based system is just a much, much larger undertaking. The beauty of React and JSX is that it leaves so many things to plain JavaScript while the JSX part only concerns itself with markup and bindings. Control flow and expressions are just script. An HTML-based template system would need to invent binding syntax, an expression language, control flow constructs, plus all of the template update mechanisms that a JavaScript-API has to.

We also don't have a native way to load HTML-based template definitions. The HTML Imports feature of the web components v0 spec is gone, and HTML Modules aren't here yet. This means that we'd have to embed HTML templates inside JavaScript modules, and at that point it's nicer to have your expressions just be JavaScript in the same lexical scope as your data. In fact, this is exactly the reason that Polymer 3, (which moved the HTML templates of Polymer 2 into JavaScrip) led directly to Lit. It was less code and a better experience to just let JavaScript handle all the script-like things.

So a JavaScript API is a subset of an HTML-based API, and will be a huge stepping stone towards an HTML-based API. Once the JavaScript API is in and proven, the HTML API only takes defining bindings, expressions, control flow, and a module system (ok, yes, that's still a massive amount of work).

We've explored the reactivity landscape

While early DOM templating proposals didn't include updating, userland systems have thoroughly
explored the landscape by now, and discovered good mental models and better and worse implementation approaches. I think we can now zero-in on a system that combines the best features from the different approaches.

The most popular reactivity approaches can be grouped into a few main categories:

VDOM and diffing as used in React
Template identity as used in Lit
Signals and similar fine-grained reactivity as used in Solid (and an increasing number of frameworks like Svelte, Vue, Angular)

Diffing is great because on the surface it's a very comprehensible model, and it works with any data. Unfortunately, it's slow compared to other approaches, and variations in diffing algorithms could be observable. I think it'd be difficult to standardize one diffing algorithm, and difficult to convince platform engineers to accept an approach with that much overhead.

Template identity - which updates the DOM if it was rendered with the current template, replaces it otherwise - is fast and gives very similar results to diffing in a majority of use cases, and it's easy to understand. It fits very well with tagged-template literals, alternate JSX transforms, and HTML-based templates, and has a nice simple semantics. It would be straightforward to specify.

Signals… are great, IMO. I strongly support the proposal to add them to JavaScript, and think that they should be a part of any new reactive DOM APIs including a template API.

Signals aren't a native part of the platform yet, and not guaranteed to be in. Signals-based systems also require all your data to be wrapped in signals - they work best when you're all in on them. But there's a lot of data that's not in signal form, and a lot of platform APIs that don't produce signals. Re-rendering is an easy way to pick up changes to non-signal data.

It's pretty easy to have a system that supports both template updating and fine-grained signals updating as co-existing modes of reactivity. A signal-pure template - one that only passes signals to bindings - would never need to be "re-rendered" - it could always only update via signals. A template that possibly references non-signal data needs to be re-rendered. Typically re-rendering is done automatically by the component or framework layer which will have reactivity, like reactive component properties.

I think this makes template identity with signals a winning combination that's fast, comprehensible, flexible, and possible to to specify.

A path to a proposal

I brought up the topic of a native declarative JS templating API in this GitHub issue, and I think what's vaguely outlined there is a logical and valuable next step for DOM creation APIs; for vanilla web developers, web components developers, and future framework evolution.

It's immediately useful to vanilla web developers as an ergonomics and secure DOM creation primitive.
It's immediately useful to many existing web component libraries, such as Lit, FAST, HyperElement, etc.
It's immediately useful to raw web component API users, and a stepping stone to a more complete ergonomic reactive component definition API.
New frameworks can be built around it.
Existing frameworks could use it as a compile target, runtime backend, or possibly support the templating API directly.
It's useful for reactivity - both for "re-rendering" techniques and for use with Signals
It's a good stepping stone to declarative HTML templating and declarative custom elements.
- It's a much smaller API to agree on and specify
- It helps define the semantics for templates
It will require the completion of the DOM Parts proposal, and help define it and prove its usefulness.

This is no small undertaking though. The surface API and syntax aren't huge, but the behavior space and the underlying DOM Parts API are large. Fine-grained reactivity also requires some kind of DOM scheduler. So there are a large number of things to enumerate, agree on, specify, and add tests for.

It's going to have to be a collaborative effort, but I think it can be done and deliver huge value for web developers and users. I'll be working on a proposal as much as I have time for, and hopefully I can get other developers, especially some platform engineers interested too.

Please comment on webcomponents#1069, or reach out to me on Bluesky if you're interested!

likes on Bluesky

Stop Using CustomEvent

Justin Fagnani — Wed, 25 Jun 2025 08:00:00 GMT

I see a lot web developers out there firing CustomEvents from their component code. So much that it seems like many developers think that CustomEvent is the only way for their code to fire custom (with a little "c") events, and maybe even the only way to fire their own events at all.

It's understandable. It's right in the name: "Custom" event. It feels like the designated tool for the job. It even sounds like it goes right along with "custom element". But I always tell developers to not use CustomEvent. There's no reason for it. Why?

What's wrong with CustomEvent?

First, let me be honest here: using CustomEvent isn't the end of the world. In terms of bad practices in web developer, this is hardly a nine alarm fire. It's barely even leaving a candle burning. CustomEvent works, and using it won't harm the users that matter most: application end users.

That said, this is apparently something I rant a bit in various web dev circles, so it's probably better I put my arguments down in one place.

CustomEvent shouldn't exist!

CustomEvent only exists because subclassing native classes wasn't well-supported when it was introduced. Adding a details object via the event init options was the platform-blessed way to add a custom payload to an Event. I presume you could also patch properties onto any Event instance, but maybe some quirk of the spec or IE meant that you couldn't rely on the same Event instance arriving at every listener.

Then ES2015 came out and suddenly we could subclass native classes like Array, HTMLElement, and Event! Had ES6 (what we used to call ES2015) come out earlier, CustomEvent would never had happened. I've heard this directly from browser engineers responsible for the DOM APIs at that time.

CustomEvent is a leaky implementation detail

When you fire a CustomEvent, you're forcing the user (the developer user) of your component to deal with the detail property. They have to know that the data they need is tucked away in this arbitrary container.

// The consumer experience with CustomEvent
someElement.addEventListener('my-event', (e: CustomEvent<MyEventDetail>) => {
  // Why the extra step?
  const { foo, bar } = e.detail;
  // ...
});

This is a leaky abstraction. Your users shouldn't have to care about the implementation detail that you chose CustomEvent. They should be interacting with an event that feels as native and intuitive as a MouseEvent or a KeyboardEvent.

What to use instead of CustomEvent

So, what should you use instead, and why?

Obviously, since CustomEvent was added due to a lack of subclassing, the main alternative is to subclass the native Event class. But I'll get into that in one bit. First, let me suggest alternatives that don't involve subclassing: using a plain Event or using a built-in event subclass.

Do you really need custom data?

The Event constructor takes an event name and options (for bubbling, composed, etc). If you don't need additional data with your event, you can just new up a plain Event. And I think that many CustomEvents don't actually need their detail object.

Consider that many platform events and simply a notification that something happened, and a reference back to the thing it happened to: the event target. For instance, when an <input> element fires a 'input' event, the event doesn't carry a new value. Instead you look at the event target and get the current value directly from that.

The nice thing about getting data this way is that it's robust against the value changing after the event is handled. You can take code that works synchronously with the 'input' event, wrap it in a debounce, and it'll still work.

Payloadless events are also more general and reusable. You could fire an 'input' event from an element that has a different data type, or multiple pieces of relevant data, without changing the event. Many platform events are payloadless and just a plain Event instance, like 'input', 'change', 'select', 'load'...

So, first, I would consider if you can rely on event.target instead of a payload.

Now, this is a subjective choice, because many people find having data right on the event to be easier to use. And maybe you do need to record the data as it was when the event was fired. These are fine reasons to use a custom payload.

Also, you may want to subclass Event even if you don't have a custom payload, for some of the other benefits I mention below.

Is there a native event that meets your needs?

Second, I would consider if a native event has the semantics you need. This could be a native event that uses a generic Event instance, or a native subclass of Event. The advantage of using a built-in event name and class is that your users and their tools may already be familiar with that event.

For example, when a component loads something, you can fire a load event. Or if you want to report an error from a component you can fire an ErrorEvent. If your component has open and close states, consider firing a ToggleEvent.

Just make sure that your event's semantics match the platform event's semantics exactly. They should be fired for the same reasons, at the same timing, with the same options (eg, bubbles).

Subclassing Event

Since we can subclass Event when we need to, this is my main and most general recommended alternative to CustomEvent. Event subclasses are better all-around.

This is an example of my basic pattern for subclassing Event:

/**
 * An event that's fired when foo happens.
 */
export class MyEvent extends Event {
  static readonly eventName = 'my-event';

  readonly foo: number;
  readonly bar: string;

  constructor(foo: number, bar: string) {
    super(MyEvent.eventName, { bubbles: true, composed: true });
    this.foo = foo;
    this.bar = bar;
  }
}

Look at what this gives us.

First, it's easier to use and plain nicer. The event's data is directly on the event object. It feels just like a native event.

// The consumer experience with a proper Event subclass
someElement.addEventListener(MyEvent.eventName, (e: MyEvent) => {
  // So much cleaner.
  const { foo, bar } = e;
  // ...
});

Second, the event subclass serves as its own documentation. The class definition is a natural place to use JSDoc to explain what the event is for and what its properties mean.

Third, and the most important for program correctness, it provides a single source of truth for the event's creation. The constructor ensures that every time MyEvent is instantiated and dispatched it has the same name and the same options, like bubbles and composed. You could achieve something similar with a factory function for CustomEvent, but at that point, you're just writing a poor man's class.

Finally, the Event subclass is its own type. This is a huge win for TypeScript users. Instead of fumbling with CustomEvent<MyDetailType>, you just use the event class itself as the type. It's simpler and more accurate.

In many cases it even makes sense to register the event type globally too:

declare global {
  interface GlobalEventHandlersEventMap {
    'my-event': MyEvent;
  }
}

Then your users will automatically get correctly-typed event handlers when using addEventListener(). Of course, you can do this with CustomEvent too, so should do this in the same cases, either way.

Composite Map Keys in JavaScript with Bitsets

Justin Fagnani — Sat, 09 Nov 2024 08:00:00 GMT

Sometimes you need to store and retrieve information related to groups of objects.

In JavaScript, Maps seem like they should be the right tool for this, but it's trickier to do than you might think. JavaScript has very minimal object and collection interfaces. Unlike some languages, like Java, there is no built-in concept of custom equality or hashcodes.

Maps and Sets in JavaScript work based on reference equality for objects. You can use an object as a key, but you can only set and get the data for an object if you use the exact same object instance - that is, if they are === to each other. Two different object instances, even with the same values for the same properties, are always different, unique map keys. If you set a value with an array key, like map.set([1, 2, 3], 'foo'), you can not then retrieve that value with map.get([1, 2, 3]).

The Records and Tuples language proposal is supposed to help with this problem by adding objects with value semantics to the language. Then map.set(#[1, 2, 3], 'foo') and map.get(#[1, 2, 3]) would work how we want. But that proposal is taking a long time and not here yet, and it might never be.

So what do we do today?

The two main approaches that I've mainly used and nested Maps and generating composite string keys, and another interesting technique came up in the Lit Discord: using BigInts bitsets as composite keys.

Nested Maps

Nested Maps work by doing successive lookups with your keys, like map.get(o1)?.get(o2). The problem is that you need a lot of Map instances. You're building a tree of maps, so there can be reuse due to branching, but at the limit, for a set of key size of K objects and N keys you need up to N * (K - 1) Maps. That can cause a lot of memory overhead.

You also have to be able to use your keys in a stable order. You need to read and write o1 then o2 always, because o2 then o1 would be a different key.

I use nested Maps occasionally, but not usually for what I would consider the composite key use cases. It's more for when my data already has a hierarchical relationship - but then you can often use the leaf nodes as keys, or roll the data you need to store into your data model.

Composite string keys

More commonly, I've uses strings as composite keys instead. Strings work well because they have value semantics. You can create the same string contents twice and they will act as the same key for Maps and Sets (and plain objects). So if you can generate the same string for a group of objects, you have yourself a composite key.

When your keys are already strings

The simplest case is when your keys are already strings. Then you can just concatenate the strings with a separator to make a composite key:

const compositeKey = (...keys: Array<string>) => keys.join('--');

The joiner string ('--') should not appear in your keys. If if could, escape the keys.

If you need to be able to supply keys in any order, you can sort them before joining:

const compositeKey = (...keys: Array<string>) => keys.toSorted().join('--');

There is a performance concern here when dealing with long string keys and/or many objects in your key sets: you may be frequently generating new string instances and causing native Map code to do full string comparisons rather than reference checks. Be aware of your data and profile as needed!

When your keys are objects

If your keys are objects, a nice pattern is to generate synthetic unique IDs for objects, then concatenate those to build the composite key.

// Stores the sequential ID we generate for each object
const ids = new WeakMap();
let nextId = 1;

const getId = (o: object) => {
  let id = ids.get(o);
  if (id === undefined) {
    ids.set((id = nextId++));
  }
  return id;
};

Then we can sort and join those IDs to build the composite key:

const compositeKey = (...keys: Array<object>) =>
  keys.map(getId).sorted().join('--');

And use the composite key in a regular Map:

const map = new Map();

const setValue = (o1, o2, o3, v) => {
  return map.set(compositeKey(o1, o2, o3), v);
};

const getValue = (o1, o2, o3) => {
  return map.get(compositeKey(o1, o2, o3));
};

BigInt Bitsets

Strings work well as composite keys because we can generate them and they have value semantics. Number also have those properties, so another technique we can use is to use bitsets represented as numbers as our composite Map keys.

Each object's presence in the key would be represented by a binary digit in the key. So instead of a composite key of 0--1--2 for the first three objects seen, you would have 0b111 or 7.

This will make for much more compact keys than strings and remove the need for sorting. The only problem is that you could run out of digits. Using bitwise operations on plain numbers in JavaScript limits you to 32 bits, so you could only store 32 objects with plain numbers.

But in modern JavaScript we now have BigInts that can store any number of bits! So we'll assign sequential IDs to objects as before, but now we'll use those as number places, generate a bitmask for each object with a single bit turned on, then combine the per-object masks with a bitwise OR to generate our bitset key.

const ids = new WeakMap();
let nextId = 1n;

const getBitMask = (o: object) => {
  let id = ids.get(o);
  if (id === undefined) {
    ids.set((id = nextId++));
  }
  // Generate a mask with a single bit set in the `id`-th place
  return 1n << id;
};

const compositeKey = (...keys: Array<object>) =>
  keys.reduce((k, o) => k | getBitMask(o), 0n);

This is a really cool technique because it got a lot simpler in ES2020 with the addition of BigInt. You could encode bitsets in strings, but it's not as easy or obvious how to do so.

Using BigInts as bitsets and Map keys was actually suggested by Claude (as relayed by our Lit teammate Steve), so I presume someone is already doing this. I couldn't find anything written up about it in a quick search though, so I hope this is useful to people out there.

You can like and discuss this post on BlueSky.

Happy hacking!

likes on Bluesky

I'm Going to Need to Blog a Lot More

Justin Fagnani — Wed, 16 Oct 2024 08:00:00 GMT

In January 2017, after having witnessed how much terrible disinformation spread on Facebook and certainly affected the presidential election, I made one final post calling for people to donate to some worthwhile charities and signed off. I haven't posted or commented on Facebook since.

I tried to do something similar with Twitter when Musk bought it. He seemed clearly intent on using the Twitter for his personal alt-right, trans- and xenophobic, conspiracy theory speading mission.

That much was right. But after trying to use Mastodon for the type of tech, urbanist, and cultural interactions I got from Twttier, to very limited success, I decided that there was simply too much in my industry circles still happening on Twitter to fully abandon it. So I limped back on to the site to at least be able to talk about web development and such. Slowly I slipped back into just as much activity as I had pre-Musk.

But that might not be so tenable for me anymore. Musk changing how blocking works, de-ranking posts with external links, and obviously promoting pro-Trump and right-wing posts makes it morally questionable to me to continue to use the site. I do not want to be making the richest man on the planet, someone as repulsive and wrong as Musk, more money be providing content and eyeballs for his personal disinformation system.

The problem is that I still find the Twitter alternatives to be severely lacking. The biggest problem is that they simply don't have comparable levels of community or activity. I hear Mastodon is doing ok, but I find discoverability of people and conversations there to be really, really bad. Threads? Bluesky? I'm not sure what has the critical mass I'm looking for. Everything seems so fragmented.

So with wanting to drastically cut back on my Twitter usage and not seeing a viable new homebase to migrate to, where that leave me? The most obvious answer to me is to cut back on social media in general and to own more of my own content. Then I can promote my content on several sites, and not be very attached to them.

Luckily, I already have a space for that here!

It won't be as easy as Tweeting, but maybe that's good. Maybe we could all use a little more time and space to put our thoughts together before publishing them to the whole world.

These things do get easier with practice. I'll need to get in the habit of quickly writing and publishing a post when I feel the need, instead of working on something for weeks and never finishing as I often do. This post here, in fact, is a bit of that practice.

Compared the the last 9 years though, I'm going to need to blog a lot more. I think you should too. It'll be good for us all.

Reactive State with Signals in Lit

Justin Fagnani — Wed, 09 Oct 2024 08:00:00 GMT

Yesterday the Lit project released the new @lit-labs/signals package that I've been working for a few months. @lit-labs/signals integrates the Signals TC39 Proposal with Lit 🎉 so that elements that use signals will automatically update when they change.

I'm really excited about this one! Partly because it's the first big new feature I've landed in Lit since leaving Google, but mostly because I think standardized signals can be a game-changer for web components, Lit, and all kinds of state management libraries.

So, what’s the big deal?

Lit Reactivity Overview

Well, first let's take a look at the current state of reactivity in Lit.

Lit has one main reactive environment - the ReactiveElement base class's "reactive update lifecycle". This is an asynchronously enqueued set of steps that perform the update of an element, including rendering in LitElement's case.

A reactive update can be triggered in any number of ways, but the main way is via reactive properties: class instance properties that request an update when they change.

This is a simple and quite effective system that composes quite well.

When using Lit elements in declarative template systems, a parent element setting properties on a child element will cause that child to enqueue an update microtask. Setting several properties, which is quite common, will enqueue just one task. Deep "property drilling" will cause a cascade of async microtasks that update elements in tree-order, each in their own batch.

Change detection in the reactive property setters means that elements will only enqueue an update if their data actually changed. This means that only the part of a Lit element subtree that needs re-rendering will even try to re-render. Change detection in the template bindings means that only DOM that's bound to changed data will update.

It's a very efficient and pretty simple system that limits the effect scope of data changes and performs minimal DOM updates, all for a small amount of code. On top of that, it's not centrally coordinated and it's interoperable. Any other web component that responds to property changes by enqueuing microtasks will benefit from the same, emergent, in-order, batched, naturally tree-pruning behavior!

Lit dataflow demo

Here's a demo that shows how components only update when their own data changes, and how that makes data flow down the tree.

Each node has a baseValue and rightValue and displays their sum, but the rightValue is only bound to the right child. The elements flash when they re-render, and delay their updates by 100ms.

You can see that changing baseValue updates the whole tree because that value flows to every node, but changing rightValue only updates the right-most branch of the tree.

Edit the files in this demo

Great, so what's missing?

Lit's system is awesome (I know... I'm biased), but it purposefully doesn't handle a lot of data management cases, and it's shallow.

Lit, in the spirit of interoperable web components, is inherently local. Elements only control their own API surface. They don't dictate what kind of data you use, they don't patch other objects or globals. They don't interject Proxies into your data objects.

This means that Lit's reactive properties have a sometimes limited view of state changes - they only see changes on the elements themselves. This is similar to React which can only "see" top-level changes to props and state from useState(). Observing deep changes is out of scope.

For developers, there are two main implications:

Elements may not update automatically in response to deep property changes.
Lit may do more work than strictly necessary to render a template because it assumes any state could have changed. (Though no-op updates are very fast)

So how do elements respond to data changes deep within objects or collections?

The answer is it's left up to developers.

And this isn't a bad answer in my opinion. There are a lot of ways to try to get deep observability of data, with few clear winners, and a lot of downsides to various approaches.

You have systems that lean on immutability, like Redux, and actually work pretty great with Lit's reactivity out of the box. Old proposals like Object.observe(). Systems that use Proxies to wrap data and detect mutations. Systems that patch a bunch of globals like Array, Map, Set., etc. to detect mutations. Patterns based on EventTarget, or Observables. Systems that let you build complex graphs of observable objects that work like generic versions of Lit's reactivity.

What's nice about Lit (via the ReactiveElement base class) is that all of these observation systems can be plugged into Lit in the same way: use their custom observation API, and call .requestUpdate() when something changes to provoke a Lit updates. There are Lit adapters for MobX, Redux, Preact Signals, RxJS, XState, etc., etc.

And of course, element authors can just call this.requestUpdate() if an element knows some data has changed, like in an event handler that performs a mutation.

Lit + Signals

Before rambling on more, let's check out the new @lit-labs/signals package.

The main thing is exports is the SignalWatcher mixin. Using it is incredibly simple: Apply it to your element, use some signals, and your element will update when the signals change. That's it!

Here's a very basic demo of a shared signal that can be read and written to by multiple elements:

This is just the simplest demo. You can get much more involved: put signals on instance fields of elements, use computed signals, build complex objects out of signals, share signals with the WCCG Context protocol, etc...

Cool, but why am I so excited about signals? 🤔

Fundamentally, the new signals integration is just more of the same: a helper that wraps the Lit update lifecycle to watch it and call this.requestUpdate() when it sees that signals have changed. There are some subtle niceties around skipping full re-renders for pinpoint DOM updates and such, but the approach is the same as with, say, MobX.

What's different with the TC39 Signals proposal is that it advocates for adding a reactive primitive directly to JavaScript.

This means that we won't have to bring along a library to use signals, but three things are even more impactful, in my opinion:

Any library based on the JavaScript standard signals API will be interoperable.
JavaScript will have a universal reactive primitive that can be pervasively used by libraries and relied on by other parts of the web platform, like the DOM.
Other web standards can use signals

Interoperable signals

A future MobX based on standard signals will produce objects that are deeply observable by any library or framework that supports watching standard signals. No special MobX integration needed. Signal-backed collections, like those in the package signal-utils, can be mixed and matched from different libraries and seamlessly used in components, effects, etc.

The interoperability of standard signals is perfectly aligned with web components and will increase web components interoperability as well. Now multiple web components built with multiple libraries, or none at all, can share the same observable data and react to it, without having to use the same non-standard library.

Universal reactive primitive

As a universal reactive primitive, more libraries will choose to make signals part of their APIs. A remote data-syncing library might make all of its data object be backed by signals, so they then just plug into this ecosystem of observers.

I also think there's huge potential for using signal-based constructs to build very rich, observable, headless data models that are easily usable from any UI layer.

Many developers currently use utilities on the UI side of an app for dealing with async data, for instance. Think things like TanStack Query or Lit's Task helper. These are extremely useful, but I think even better patterns emerge when your data model itself can handle fetching more data, caching, metadata, and synchronization. You can make very nice APIs that abstract away details that are unimportant for components (usually the how) and focus on what is important (the what), and the data can be portably used in any signal-aware environment.

Example: AsyncComputed

In my own code I've been moving away from using Lit's Task, which is a reactive controller that's tied to a single element, and towards using the AsyncComputed utility from signal-utils (a utility I contributed).

AsyncComputed lets you construct signals based on async computations. It's actually very similar to Lit's Task, but instead of hooking Lit's reactive lifecycle, it reads from signals and publishes its state as various signals.

This is a port of the Lit Task example to use AsyncComputed.

The interesting thing here is that the async work was moved into a UI-layer agnostic data model object. All the component has to do is read and write to the object. As data is fetched, the signals of the AsyncComputed that hold the result are updated, and the components that use it are automatically updated as well.

I find this pattern especially useful for applications that navigate large object graphs with lazy loaded edges. I also have a wrapper on AsyncComputed for lazy-loaded collections that helps with metadata like collection size and pagination. Using those collections from the UI layer is super easy.

A reactive type for web standards

One of the biggest changes that signals could bring, and only if they're standardized, is integration into other web standards like the DOM.

Imagine being able to assign a signal to an attribute or some text content and have the DOM automatically update when the signal changes!

const userName = new Signal.State('amy123');
const userNameSpan = document.querySelector('#user-name`);
// An imagined new API, but bear with me...
userNameSpan.setTextContent(userName);

// Later...
userName.set('joe456');
// And the <span> updates to <span>joe456</span>

I'm not sure about this setTextContext() API I just dreamed up above, but I do think we have a potential lead on this type of reactivity with the DOM Parts proposal which aims to add markers and objects that directly represent dynamic segments of DOM - perfect for signals integration.

<template parseparts>
  <p>Current user: {{}}</p>
</template>

const {node, parts} = template.content.cloneWithParts();
const userPart = parts[0];
userPart.setValue(userName);

// Later...
userName.set('joe456');
// And the <p> updates to <p>Current user: joe456</p>

If the DOM ever gets a declarative JavaScript template API, like I'm proposing in WICG/webcomponents#1069, we could do this with great ergonomics:

const {html} = HTMLElement;

const userName = new Signal.State('amy123');
document.body.render(html`<p>Current user: ${userName}</p>`);

// Later...
userName.set('joe456');
// And the <p> updates to <p>Current user: joe456</p>

There's a huge potential for built-in reactivity in the DOM, that could be unlocked by standard signals.

Looking forward with `@lit-labs/signals`

If you can't tell, I'm pretty thrilled by everything related to standardized signals right now: the TC39 signals proposal, our integration with Lit, and the seemingly endless possibilities ahead of us.

Like all standards work, it's going to take a little while. So we have some time to shore up how things work on the Lit end.

A few things I'd like to see added soon:

Support for auto-updating templates outside of LitElement and ReactiveElement: A lot of use cases are satisfied by simple templates, without the need for full-blown components. We'll need a scheduler interface in lit-html to make this work.
Signal aware repeat() directive: Signal-oriented frameworks have optimizations built into their loop constructs that we can copy over to Lit.
Signal-backed @property(): We can replace Lit's reactive property storage with signals and make integration into signals patterns even more powerful. Combined with effects would also address requests for property observers in Lit. Instead of a special Lit feature, you just read LitElement properties in a regular effect.
Batched effects in the standard signals API: I've made a feature request to the signals proposal to allow for batching of signal updates, with an immediate flush of batched effects. This would allow us to move to synchronous rendering in Lit, by using signal-backed @property()s and rendering in a batched effect, retaining our efficient DOM batching and tree-order rendering.
A tree-aware DOM scheduler API: Shared signals can introduce out-of-tree-order rendering without a central scheduler, but web components don't share a common scheduler, outside of built-ins like the microtask and task queues or animation frames, adn those aren't tree aware. I want to propose a tree-aware task scheduler for the DOM so that all web components, frameworks, and general DOM updating code can ensure that updates run in batched, top-down order.

On the Lit team, we'll be working hard on our libraries and standards proposals to help bring about our glorious, interoperable, reactive future 😎

In the meantime, please try out the new @lit-labs/signals package, signal-utils, and build your own projects with signals and let us know how things work, or don't, for you!

Find the Lit team and community on Discord or GitHub Discussions.

Links:

How to Publish Web Components to NPM

Justin Fagnani — Fri, 01 Nov 2019 18:58:10 GMT

Publishing JavaScript to npm is a controversial affair. What should be a simple process, in my opinion, is fraught with a large number of correlated choices that are dependent not just on technical factors, but social factors like what users are used to, and how web-centric their point-of-view is.

Web components, being a superset of JavaScript, bring their own set of choices on top of this.

What follows is my personal checklist for publishing web components to npm. This checklist attempts to maximize compatibility, standards compliance, flexibility, and usefulness to your users.

These are my personal opinions, yes, but they are hard-won, toiled over, and hopefully well-reasoned opinions, so I (a bit obviously) think they're really the right way to go.

Justin's Checklist for Publishing Web Components to NPM™

Publish standard ES2017
Publish standard JavaScript modules
Do not use .mjs file extensions
Only publish a single build
Important package.json fields:
- Set "type" to "module"
- Set "main" to the main entry point module
- Set "module" to the same file as "main"
- Include polyfills in devDependencies, not dependencies
Do not bundle
Do not minify
Always self-define elements
Export element classes
Do not import polyfills into modules
Import dependencies with "bare" or "named" import specifiers
Always include file extensions in import specifiers
Publish a custom-elements.json file documenting your elements
Include good TypeScript typings

Now let's get into the *why *for each recommendation.

Publish standard ES2017

Modern JavaScript is smaller, faster, and more capable than the same code transpiled to an old version of JS like ES5, and the vast majority of users have browsers that support it.

So it's very preferable to send the most modern JavaScript to users that you can. But you can't send modern JS to browsers if you don't have it to begin with. Consumers of your packages can compile the code to a lower language level if they need to, but they can't un-compile your code to a newer language version.

Another way to frame this is that only applications know their exact browser support levels. Some applications need to support very old browsers. Some, maybe running in Electron, only need to support the latest Chrome. Reusable libraries can't know what the browser requirements are, and should publish modern JS to give the most flexibility to applications.

But how modern? This is the tricky part. I choose ES2017 because it's very widely supported across Chrome, Safari, Firefox, and Edge. This means that for most browsers you don't need to compile at all.

I once had tried to come up with a convention that would describe what language version to publish, like "ESY-1", meaning in 2019 publish ES2018, but Edge has slipped a bit and doesn't support object spread/rest, so this didn't hold. I think the situation will be a little clearer when Edge ships with the Chromium backend, hopefully soon.

What do users do who need to support IE11? They need to compile dependencies inside node_modules/. This is something that should be more common, because it solves a lot of problems with JS distribution. If this is slow with some tools, my opinion is that those tools need to cache intermediate build results better because you should only recompile dependencies when they change.

Publish standard JavaScript modules

All modern browsers and tool-chains support standard JavaScript modules, and browsers don't natively support any other module format. This means that if you publish modules, they can load uncompiled.

This is especially nice for development, where the less transforms on code you do the better the debugging experience is. Native modules are great to use during development without bundling too. A static file server that properly sends 304 response codes for unmodified files will take maximum advantage of the browser's cache and only send the files that changed.

Most applications are not serving native modules to browsers yet, but the application's tool chain can certainly handle modules as input and transform and bundle them to whatever format the app is using.

Do not use .mjs file extensions

The .mjs file extension is as useless for browsers as it is controversial. There's really no benefit at all from using it. Browsers only care about the mime type of files, not the file extension, so .mjs does nothing there. And tools look to package.json to determine if a package contains modules, so there's no benefit there.

There are downsides to .mjs though: not every tool in the world understands it. Some static file servers may not send the right mime-type header, meaning that the file won't load as a module in browsers.

The best thing is to just avoid it altogether and always write and publish modules, and always use the .js extension.

Only publish a single build

It's pretty common on npm to publish multiple builds. This is a bad and outdated practice that can lead to bloat in application bundles. The reason is that multiple libraries may share a common dependency, but if they import different builds, the bundles will end up with slightly different duplicates of the dependency.

In the case of web components, this is especially dangerous, since we really need there to be a single definition of a component. Multiple versions here is a bigger headache than just bloat.

And again, if the application consumes standard JS modules in its build pipeline, it can transform that to whatever single format it needs. There is really no need to publish multiple builds ever.

I really recommend holding this line against the inevitable issues and PRs that will ask you to add an ES5 UMD build.

Important package.json fields

Set "type" to "module"

The "type" field is the standard way to indicate that JavaScript files in an npm package are modules. Tools like CDNs and bundlers can use this to properly parse files with module parse goal.

With this field you do not need to use the .mjs extension, even in Node.

Set "main" to the main entry point module

This is just standard practice with the slight difference that most packages today publish some kind of non-modules build and point "main" to that. It'll be unusal to some to point this field to a module, but it's semantically correct, goes along with "type": "module", and npm requires a "main" entry.

Set "module" to the same file as "main"

"type": "module" is the most correct way to specify that a package contains modules, but tools have supported the "module" field for a while, and we'll have to use this for a while yet before all tools support the "type" field.

Include polyfills in devDependencies, not dependencies

Polyfills are an application concern, so the application should depend directly on them. Packages may need to depend on polyfills for tests and demos, so if they're needed, they should only go in "devDependencies".

Do not bundle

This one has a little bit of wiggle room depending on the structure of your package.

The most important advice here is to not bundle dependencies. This keeps you from causing bloat by duplicating dependencies into multiple package bundles.

As long as you don't bundle dependencies, you may decide to bundle your package in order to hide implementation modules. If you do, make sure you preserve all the valid entry points of your package as separate entry point files into a set of bundles with shared chunks.

That is, if you support importing 'my-library/element-a.js' and 'my-library/element-b.js don't bundle them together. The browser's module loader acts as a natural tree-shaker, in that it only loads the modules it needs. It's important then to keep modules relatively small and single-purposed. Let consumers import just the parts of your package that they need. Bundlers will have an easier time of creating small bundles too. So don't let per-package bundling lead to bloat for applications.

I find it simpler to just not bundle libraries at all. The application's build pipeline will take of it how it sees fit.

Do not minify

Just like the recommendations to not publish builds less than ES2017 and to not bundle, minification should be an application concern. It's much easier to debug non-minified code, and minifiers get better over time, so don't bake this into your published files.

Always self-define elements

The module that declares the web component class should always include a call to customElements.define() to define the element.

This is a contentious point, but currently there is no other practical choice. Some people would like to allow consumers of components to choose the tag name, so they advocate leaving out the customElements.define() call, or putting it in a separate module, but this doesn't actually work out well.

Custom elements currently require that there be a strict one-to-one relationship between tag names and classes. This means that if a custom element doesn't self-define, it leaves open the possibility that multiple consumers try to register the element and only the first will succeed.

If consumers really want to choose a different tag name, they can create a trivial subclass of the element and register that:

import {SomeElement} from 'some-element';
    
customElements.define('my-some-element', class extends SomeElement{});

This will all change when we finally get scoped custom element registries. Then consumers will be able to register elements into a scope that they fully control and can choose whatever that names they want.

Export element classes

In order to support the trivial subclassing pattern described above and subclassing in general, you should export the element class:

export class MyElement extends LitElement {
  // ...
}
customElements.define('my-element', MyElement);

Do not import polyfills into modules

This is important general advice, but it's worth repeating for web components.

To go along with the "Only the application knows" mantra, only the application knows what polyfills are necessary for it's target environments. Most users don't need the web components polyfills, so a well-constructed application will not serve them to those users, or will use the webcomponentsjs polyfill loader to only dynamically load them when necessary.

If your library directly imports polyfills, it makes it pretty tricky for applications to pull them out when they don't need them.

Import dependencies with "bare" or "named" import specifiers

It's common practice in Node and on npm to import external dependencies by package name.

Node natively supports this in CommonJS modules with require() and node module resolution:

const otherLib = require('other-lib');

And developers these days are pretty used to writing standard JS module syntax that uses node module resolution:

import * as otherLib from 'other-lib';

These are called "bare import specifiers" or "named imports", and browsers don't actually support them! At least not yet. Browsers only support importing by URL. So all imports must by a full URL (http://) or be a relative URL starting with /, ./ or ../.

What's happening to make bare specifiers work in browsers is that tools like Rollup and Webpack are performing node module resolution at build time and transforming these paths while they're building the whole application.

So if we want to publish standard modules to npm so that they can load without compilation, and we have dependencies that we need to import what do we do?

We could try to use relative paths to point to dependencies, like this:

import * as otherLib from '../other-lib/index.js';

But this requires that we know exactly where other-lib is located on the file system relative to the importing module. The problem is that npm can install modules in a number of different locations, so we can't possibly know the right path to use. Paths will be different when your package is the top package, and you dependencies will likely be at ./node_modules/, and when your package is installed as a dependencies might be at ../node_modules. Servers and build-systems could move files around too.

An even bigger problem is that most tools do not understand these kind of relative paths that reach out of package boundaries. VS Code and TypeScript get mightily confused in my experience.

So right now the best option is to import dependencies by name and let tools rewrite the import specifiers before they reach the browser. This works really well in practice. Since basically all tools support named specifiers, compatibility is good there, and a number of tools have popped up that only rewrite import specifiers and leave individual modules otherwise untouched to be loaded natively by browsers.

We pioneered this on my team with the Polymer CLI which rewrites specifiers on the fly in it's polymer serve command, and other tools like [es-dev-server](https://open-wc.org/developing/es-dev-server.html) and the unpkg.com CDN with the ?module query parameter do this.

In the near future, browsers will support "import maps" that will tell them how to translate names into URLs, so they too will support named imports. Yay!

Always include file extensions in import specifiers

Classic Node module resolution doesn't require file extensions because it does a search of the file system looking for one of several file extensions if one isn't given. When you import some-package/foo, Node will import some-package/foo.js if it exists. This isn't feasible over a network, so browsers will not do this kind of searching.

Import maps will allow mapping names to URLs, but they only have two type of mappings: exact and prefix. Here's an example for lodash:

{
  "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js",
    "lodash/": "/node_modules/lodash-es/"
  }
}

This means that we can easily map a bare specifier like lodash, or a prefix + full file path, like lodash/forEach.js, etc., but to support extensionless imports like lodash/forEach, we'd have to map every one to the full path, like:

{
  "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js",
    "lodash/": "/node_modules/lodash-es/",
    "lodash/forEach": "/node_modules/lodash-es/forEach.js"
  }
}

We would have to do this for every extensionless import in the entire app. lodash-es has 341 modules that could be imported. Creating entries for each one that's imported without extensions would make for a bloated import map, so it's much better to just use extensions in imports and have only a prefix import map entry.

Publish a custom-elements.json file documenting your elements

Web components tools, like IDE plugins and catalogs, are starting to converge on a common format for describing the custom elements publish in an npm package.

The custom-elements.json file describes the tag names, attributes, properties, events, etc., that an element supports. With this information IDEs can provide auto-completion, hover-over docs, etc; linters can check that you're using defined properties; type checkers can ensure property bindings are of the correct type; documentation viewers can display the information for human consumption; and catalogs like Storybook can generate "knobs" for components automatically.

Rune Mehlsen, an amazing developer who maintains the lit-plugin VS Code plugin, also maintains web-component-analyzer that outputs this format.

Including this file will greatly enhance the developer experience of your users.

Include good TypeScript typings

I'm a huge fan of TypeScript, and one of my favorite things about its type system is how it's able to type APIs that use string keys.

One example is the document.createElement() method. It's return type depends on the value of the string parameter passed to it, so that document.createElement('div') returns an HTMLDivElement and document.createElement('img') returns an HTMLImageElement. This let's code like this type-check:

document.createElement('div').src = './image.jpg'; // error!
document.createElement('img').src = './image.jpg'; // fine :)

The best part about this is it's based on a mapping from tag name to class that you can extend. It's called the HTMLElementTagNameMap. To extend it to add your element, use TypeScript's interface augmentation.

A full custom element definition in TypeScript should look something like this:

export class MyElement extends LitElement {
  // ...
}

customElements.define('my-element', MyElement);

declare global {
  interface HTMLElementTagNameMap {
    "my-element": MyElement,
  }
}

Of course you can just add this to your typings too if you're not writing your element in TypeScript. The benefits to your TypeScript users will be huge.

Disagree? Comments and Questions?

I know some of these recommendations are controversial, and the finer points can be open to interpretation and debate, but I firmly believe that this is the best way to publish web components in late 2019.

I don't have comments enabled on this blog for good reasons, but you can find me on Twitter at @justinfagnani. I can't promise I'll have the time or inclination to debate these recommendations, but I try to answer questions. Of course, if you seriously disagree you can publish your own recommendations too! 😎

Enhancing Mixins with Decorator Functions

Justin Fagnani — Thu, 07 Jan 2016 08:16:00 GMT

Last time, in "Real" Mixins with JavaScript Classes, we saw how we can create powerful ES6 mixins by using class expressions as subclass factories.

Now, let's look at some enhancements that make our mixins more powerful, in a way still requires no framework for mixin users, and is still easy to use for mixin authors.

First, we'll look at how subclass-factory-style mixins can be wrapped, or decorated, to add aditional features. Then we'll enable the caching of mixin applications, so that the same mixin definition applied to the same superclass multiple times reuses the same mixin application, and implement ES2015's @@hasInstance[^1] method so that instanceof works with mixins!

The Technique and Some Groundwork

We're going to add these enhancements at the mixin definition site, so that only the mixin author needs to do anything to get them.

Recall a mixin definition from the last post:

const MyMixin = (superclass) => class extends superclass {
  /* ... */
};

Our new enhancements are built as functions that wrap the mixin function. Mixin declaration will now look like this:

const MyMixin = BaseMixin((superclass) => class extends superclass {
  /* ... */
});

BaseMixin wraps the author's subclass factory in a function that does a little extra work before calling the factory. This is very much like ES.next decorators, just without the special @BaseMixin syntax, so I'll call them decorator functions.

All kinds of behaviors can be added to mixins by adding more wrappers. Since we're going to implement caching and @@hasInstance support, our mixin definitions might end up looking like this:

const MyMixin = Cached(HasInstance(BaseMixin((superclass) => class extends superclass {
  /* ... */
})));

This should be fairly easy for authors to use, and mixin users don't need to do anything different. Mixin application still just works as function invocation:

class A extends MyMixin(MyBaseClass) {}

class B extends MyMixin(MyBaseClass) {}

Or we can use mix().with():

class A extends mix(MyBaseClass).with(MyMixin) {}

class B extends mix(MyBaseClass).with(MyMixin) {}

mix().with() is still just plain and optional sugar around function invocation.

If we implement Cached and HasInstance correctly, the mixins and classes suddenly have new powers:

let a = new A();
let b = new B();

a instanceof A; // true
b instanceof B; // true
Object.getPrototypeOf(a) === Object.prototypeOf(b); // true

We can use standard function composition to create a convenience decorator for the common behaviors:

const Mixin = (mixin) => Cached(HasInstance(BaseMixin(mixin)));

const MyMixin = Mixin((superclass) => class extends superclass {
  /* ... */
});

Wrapping with Care

We have to be a little more careful with wrapping mixin functions than usual because we want the final result of calling the decorators to be an object with certain properties, like @@hasInstance. We don't know what decorators might be applied to our mixin, and we'd like to avoid requiring a specific ordering of decorators, so we don't want to obscure properties added by one wrapper with a subsequent wrapper.

To solve this we'll create a wrap utility that sets the prototype of the wrappers, and stores a property pointing to the original mixin function so that we have a canonical identity for a mixin:

const _originalMixin = Symbol('_originalMixin');

const wrap = (mixin, wrapper) => {
  Object.setPrototypeOf(wrapper, mixin);
  if (!mixin[_originalMixin]) {
    mixin[_originalMixin] = mixin;
  }
  return wrapper;
};

This makes our function wrapping behave a lot more like inheritance.

Next, we'll create a base mixin decorator that applies the mixin and stores a reference from the mixin application back to the mixin definition for later use in caching:

const BaseMixin = (mixin) => wrap(mixin, (superclass) => {
  let application = mixin(superclass);
  application.prototype[_mixinRef] = mixin[_originalMixin];
  return application;
});

Notice the call to wrap. This sets things up so that other mixin decorators can add properties to the mixin and have them be visible after wrapping.

Caching Mixin Applications

The whole purpose of mixins is to apply them to different superclasses so that using a mixin doesn't require a specific inheritance hierarchy. However, mixins may still be repeatedly applied to the same superclass.

Consider two classes, B and C which both extend A and mixin M1 and M2:

class B extends mix(A).with(M1, M2) {}

class C extends mix(A).with(M1, M2) {}

The simplest mixin implementation will produce two identical A-with-M1 and two identical A-with-M1-with-M2 prototypes.

Instead we would like to share identical applications:

We can do this by caching mixin applications on classes:

const _mixinRef = Symbol('_mixinRef');

const Cached = (mixin) => wrap(mixin, (superclass) => {
  // Create a symbol used to reference a cached application from a superclass
  let applicationRef = mixin[_cachedApplicationRef];
  if (!applicationRef) {
    applicationRef = mixin[_cachedApplicationRef] = Symbol(mixin.name);
  }

  // Look up an cached application of `mixin` to `superclass`
  if (superclass.hasOwnProperty(applicationRef)) {
    return superclass[applicationRef];
  }

  // Apply the mixin
  let application = mixin(superclass);

  // Cache the mixin application on the superclass
  superclass[applicationRef] = application;

  return application;
});

Hopefully the comments are easy to follow. The approach is to create a unique symbol per mixin, then store mixin applications in a property keyed by that symbol on the superclass of a mixin application. That allows us to look up and reuse previous applications of a mixin to a superclass.

If this seems like overkill, consider cases where you must extend a certain class, yet libraries would like to offer help in customizing those subclasses.

I'm thinking of custom elements here, and specifically, libraries like Polymer. Custom elements need to extend one of the built-in elements - at the very least HTMLElement - but Polymer does a lot of work by being on the prototype chain. Mixins make this easy to express:

class MyElement extends mix(HTMLElement).with(Polymer) {
  // ...
}

But with naive mixins a new identical superclass of HTMLElement-With-Polymer is created for every custom element class, wasting time and memory. With mixin application caching this can be easy and efficient.

Adding instanceof Support

ES2015 has support for overriding the instanceof operator via the @@hasInstance method, which should be implemented by objects that appear of the righthand side of an instanceof operator[^2].

Ideally an expression like o instanceof MyMixin would work as expected: if o is an instance of a class that has mixed in MyMixin, the expression should return true.

With mixins, the "type" is one of our subclass factory functions, so we need to patch the @@hasInstance method into the mixin. The prototype setting we did in the groundwork section will ensure that this method is available even after subsequent wrapping.

Out patch looks like this:

const HasInstance = (mixin) => {
  if (!Symbol.hasInstance) {
    return mixin;
  }
  mixin[Symbol.hasInstance] = function(o) {
    const originalMixin = this[_originalMixin];
    while (o != null) {
      if (o.hasOwnProperty(_mixinRef) && o[_mixinRef] === originalMixin) {
        return true;
      }
      o = Object.getPrototypeOf(o);
    }
    return false;
  }
  return mixin;
};

I can't find that this is actually implemented in any production VM yet, but it looks like Webkit has initial support, along with Babel under a flag.

Writing Your Own Mixin Decorators

mixwith.js includes the wrap utility and some common symbols that help with writing mixin decorators. You can use these to write decorators that play well with each other. What kind of decorators might you want to write? Maybe a de-duplication, so that a mixin applied twice to a prototype chain only appears once in the chain, or a traits-like system that disallows overriding.

There are probably two main types of mixin decorators: those that need to wrap the mixin function, and those that just need to patch it.

If you need to wrap the mixin function, make sure you call wrap and usually you want to invoke the mixin function. Sometimes you may want to conditionally invoke the mixin, like with de-duplication:

import { wrap } from 'mixwith';

const DeDupe = (mixin) => wrap(mixin, (superclass) => {
  if (mixinAlreadyApplied(superclass)) {
    return superclass;
  }
  return mixin(superclass);
});

If you only need to patch the mixin function, just patch and return it:

const Fooify = (mixin) => {
  mixin.foo = "Foo";
  return mixin;
};

Follow the conversation

reddit.com/r/javascript

Footnotes

[^1]: @@foo is shorthand for Symbol.foo in the JavaScript specification

[^2]: This is fabulous for extensibility, since new types can accept an instance without having to modify the instance. You can implement any kind of type checking you want, say structural typing. It's terrible for static type checking, however.

"Real" Mixins with JavaScript Classes

Justin Fagnani — Mon, 21 Dec 2015 11:14:00 GMT

Mixins and Javascript: The Good, the Bad, and the Ugly.

Mixins and JavaScript are a like the classic Clint Eastwood movie.

The good is that composing objects out of small pieces of implementation is even possible due to JavaScript's flexible nature, and that mixins are fairly popular in certain circles.

The bad is a long list: there's no common idea of what the concept of a mixin even is in JavaScript; there's no common pattern for them; they require helper libraries to use; more advanced composition (like coordination between mixins and prototypes) is complicated and does not fall out naturally from the patterns; they're difficult to statically analyze and introspect; finally, most mixin libraries mutate objects or their prototypes, causing problems for VMs and some programmers to avoid them.

The ugly is that the result of all this is a balkanized ecosystem of mixin libraries and mixins, with often incompatible construction and semantics across libraries. As far as I can tell, no particular library is popular enough to even be called common. You're more likely to see a project implement its own mixin function than see a mixin library used.

JavaScript mixins so far have simply fallen well short of their potential, of how mixins are described in academic literature and even implemented in a few good languages.

For someone who loves mixins, and thinks they should be used as much as possible, this is terrible. Mixins solve a number of issues that single-inheritance languages have, and to me, most of the complaints that the prototypal crowd has against classes in JavaScript. I would argue all inheritance should be mixin inheritance - subclassing is just a degenerate form of mixin application.

Luckily, there's light at the end of the tunnel with JavaScript classes. Their arrival finally gives JavaScript very easy to use syntax for inheritance. JavaScript classes are more powerful than most people realize, and are a great foundation for building real mixins.

In this post I'll explore what mixins should do, what's wrong with current JavaScript mixins, and how simple it is to build a very capable mixin system in JavaScript that plays extremely well with classes.

What, Exactly, are Mixins?

To understand what a mixin implementation should do, let's first look at what mixins are:

A mixin is an abstract subclass; i.e. a subclass definition that may be applied to different superclasses to create a related family of modified classes.
Gilad Bracha and William Cook, Mixin-based Inheritance

This is the best definition of mixins I've been able to find. It clearly shows the difference between a mixin and a normal class, and strongly hints at how mixins can be implemented in JavaScript.

To dig deeper into the implications of this definition, let's add two terms to our mixin lexicon:

mixin definition: The definition of a class that may be applied to different superclasses.
mixin application: The application of a mixin definition to a specific superclass, producing a new subclass.

The mixin definition is really a subclass factory, parameterized by the superclass, which produces mixin applications. A mixin application sits in the inheritance hierarchy between the subclass and superclass.

The real, and only, difference between a mixin and normal subclass is that a normal subclass has a fixed superclass, while a mixin definition doesn't yet have a superclass. Only the mixin applications have their own superclasses. You can even look at normal subclass inheritance as a degenerate form of mixin inheritance where the superclass is known at class definition time, and there's only one application of it.

Examples

Here's an example of mixins in Dart, which has a nice syntax for mixins while being similar to JavaScript:

class B extends A with M {}

Here A is a base class, B the subclass, and M a mixin. The mixin application is the specific combination of M mixed into A, often called A-with-M. The superclass of A-with-M is A, and the actual superclass of B is not A, as you might expect, but A-with-M.

Some class declarations and diagrams might be helpful to show what's going on.

Let's start with a simple class hierarchy, with class B inheriting from class A:

class B extends A {}

Now let's add the mixin:

class B extends A with M {}

As you can see, the mixin application A-with-M is inserted into the hierarchy between the subclass and superclass.

Note: I'm using long-dashed line to represent the mixin declartion (B includes M), and the short-dashed line to represent the mixin application's definition.

Multiple Mixins

In Dart, multiple mixins are applied in left-to-right order, resulting in multiple mixin applications being added to the inheritance hierarchy:

class B extends A with M1, M2 {}

class C extends A with M1, M2 {}

Traditional JavaScript Mixins

The ability to freely modify objects in JavaScript means that it's very easy to copy functions around to achieve code reuse without relying on inheritance.

Mixin libraries like Cocktail, traits.js, and patterns described in many blog posts (like one of the latest to hit Hacker News: Using ES7 Decorators as Mixins), generally work by modifying objects in place, copying in properties from mixin objects and overwriting existing properties.

This is often implemented via a function similar to this:

function mixin(source, target) {
  for (var prop in source) {
    if (source.hasOwnProperty(prop)) {
      target[prop] = source[prop];
    }
  }
}

A version of this has even made it into JavaScript as Object.assign.

mixin() is usually then called on a prototype:

mixin(MyMixin, MyClass.prototype);

and now MyClass has all the properties defined in MyMixin.

What's So Bad About That?

Simply copying properties into a target object has a few issues. Some of this can be worked around with smart enough mixin functions, but the common examples usually have these problems:

Prototypes are modified in place.

When using mixin libraries against prototype objects, the prototypes are directly mutated. This is a problem if the prototype is used anywhere else that the mixed-in properties are not wanted. Mixins that add state can create slower objects in VMs that try to understand the shape of objects at allocation time. It also goes against the idea that a mixin application should create a new class by composing existing ones.

`super` doesn't work.

With JavaScript finally supporting super, so should mixins: A mixin's methods should be able to delegate to an overridden method up the prototype chain. Since super is essentially lexically bound, this won't work with copying functions.

Incorrect precedence.

This isn't necessarily always the case, but as often shown in examples, by overwriting properties, mixin methods take precedence over those in the subclass. They should only take precedence over methods in the superclass, allowing the subclass to override methods on the mixin.

Composition is compromised

Mixins often need to delegate to other mixins or objects on the prototype chain, but there's no natural way to do this with traditional mixins. Since functions are copied onto objects, naive implementations overwrite existing methods. More sophisticated libraries will remember the existing methods, and call multiple methods of the same name, but the library has to invent its own composition rules: What order the methods are called, what arguments are passed, etc.

References to functions are duplicated across all applications of a mixin, where in many cases they could be bundled in a shared prototype. By overwriting properties, the structure of protytpes and some of the dynamic nature of JavaScript is reduced: you can't easily introspect the mixins or remove or re-order mixins, because the mixin has been expanded directly into the target object.

If we actually use the prototype chain, all of this goes away with very little work.

Better Mixins Through Class Expressions

Now let's get to the good stuff: Awesome Mixins™ 2015 Edition.

Let's quickly list the features we'd like to enable, so we can judge our implementation against them:

Mixins are added to the prototype chain.
Mixins are applied without modifying existing objects.
Mixins do no magic, and don't define new semantics on top of the core language.
super.foo property access works within mixins and subclasses.
super() calls work in constructors.
Mixins are able to extend other mixins.
instanceof works.
Mixin definitions do not require library support - they can be written in a universal style.

Subclass Factories with this One Weird Trick

Above I referred to mixins as "subclass factories, parameterized by the superclass", and in this formulation they are literally just that.

We rely on two features of JavaScript classes:

class can be used as an expression as well as a statement. As an expression it returns a new class each time it's evaluated. (sort of like a factory!)
The extends clause accepts arbitrary expressions that return classes or constructors.

The key is that classes in JavaScript are first-class: they are values that can be passed to and returned from functions.

All we need to define a mixin is a function that accepts a superclass and creates a new subclass from it, like this:

let MyMixin = (superclass) => class extends superclass {
  foo() {
    console.log('foo from MyMixin');
  }
};

Then we can use it in an extends clause like this:

class MyClass extends MyMixin(MyBaseClass) {
  /* ... */
}

And MyClass now has a foo method via mixin inheritance:

let c = new MyClass();
c.foo(); // prints "foo from MyMixin"

Incredibly simple, and incredibly powerful! By just combining function application and class expressions we get a complete mixin solution, that generalizes well too.

Applying multiple mixins works as expected:

class MyClass extends Mixin1(Mixin2(MyBaseClass)) {
  /* ... */
}

Mixins can easily inherit from other mixins by passing the superclass along:

let Mixin2 = (superclass) => class extends Mixin1(superclass) {
  /* Add or override methods here */
}

And you can use normal function composition to compose mixins:

let CompoundMixin = (superclass) => Mixin2(Mixin3(superclass));

Benefits of Mixins as Subclass Factories

This approach gives us a very good implementation of mixins

Subclasses can override mixin methods.

As I mentioned before, many examples of JavaScript mixins get this wrong, and mixins override the subclass. With our approach subclasses correctly override mixin methods which override superclass methods.

`super` works.

One of the biggest benefits is that super works inside methods of the subclass and the mixins. Since we don't ever overwrite methods on classes or mixins, they're available for super to address.

super calls can be a little unintuitive for those new to mixins because the superclass isn't known at mixin definition, and sometimes developers expect super to point to the declared superclass (the parameter to the mixin), not the mixin application. Thinking about the final prototype chain helps here.

Composition is preserved.

This is really just a consequence of the other benefits, but two mixins can define the same method, and as long as they call super, both of them will be invoked then applied.

Sometimes a mixin will not know if a superclass even has a particular property or method, so it's best to guard the super call:

let Mixin1 = (superclass) => class extends superclass {
  foo() {
    console.log('foo from Mixin1');
    if (super.foo) super.foo();
  }
};

let Mixin2 = (superclass) => class extends superclass {
  foo() {
    console.log('foo from Mixin2');
    if (super.foo) super.foo();
  }
};

class S {
  foo() {
    console.log('foo from S');
  }
}

class C extends Mixin1(Mixin2(S)) {
  foo() {
    console.log('foo from C');
    super.foo();
  }
}

new C().foo();

prints:

foo from C
foo from Mixin1
foo from Mixin2
foo from S

Improving the Syntax

I find applying mixins as functions to be both elegantly simple - describing exactly what's going on, but a little bit ugly at the same time. My biggest concern is that this construction isn't optimized for readers unfamiliar with this pattern.

I'd like a syntax that was easier on the eyes and at least that gave new readers something to search for to explain what's going on, like the Dart syntax. I'd also like to add some more features like memoizing the mixin applications and automatically implementing instanceof support.

For that we can write a simple helper which applies a list of mixins to a superclass, in a fluent-like API:

class MyClass extends mix(MyBaseClass).with(Mixin1, Mixin2) {
  /* ... */
}

Here's the code:

let mix = (superclass) => new MixinBuilder(superclass);

class MixinBuilder {
  constructor(superclass) {
    this.superclass = superclass;
  }

  with(...mixins) { 
    return mixins.reduce((c, mixin) => mixin(c), this.superclass);
  }
}

That's it! It's still extremely simple for the features and nice syntax it enables.

Constructors and Initialization

Constructors are a potential source of confusion with mixins. They essentially behave like methods, except that overriden methods tend to have the same signature, while constructors in a inheritance hierarchy often have different signatures.

Since a mixin likely does not know what superclass it's being applied to, and therefore its super-constructor signature, calling super() can be tricky. The best way to deal with this is to always pass along all constructor arguments to super(), either by not defining a constructor at all, or by using the spread operator: super(...arguments).

This means that passing arguments specifically to a mixin's constructor is difficult. One easy workaround is to just have an explicit initialization method on the mixin if it requires arguments.

Further Exploration

This is just the beginning of many topics related to mixins in JavaScript. I'll post more about things like:

Enhancing Mixins with Decorator Functions new post that covers:
Caching mixin applications so that the same mixin applied to the same superclass reuses a prototype.
Getting instanceof to work.
How mixin inheritance can address the fear that ES6 classes and classical inheritance are bad for JavaScript.
Using subclass-factory-style mixins in ES5.
De-duplicating mixins so mixin composition is more usable.

The mixwith.js project

I started a project on GitHub to build a library around this pattern: https://github.com/justinfagnani/mixwith.js It can be found on npm: https://www.npmjs.com/package/mixwith

Follow the conversation

Getting Things Done... Like This Blog!

Justin Fagnani — Mon, 21 Dec 2015 10:55:00 GMT

I've intended to launch a blog at justinfagnani.com for a very, very long time now. Prevailing over my procrastinating and complexifying nature I've finally done it, but only by cutting almost every corner possible.

After a brief flirtation with blogging about Dart at dartery.blogspot.com, I wanted a better writing experience for myself, reading experience for my visitors, and broader scope than just Dart programming. My main frustration was with Blogger and the difficulty of customizing it and writing posts with richer content, like code snippets, or embeds from other sites.

This first lead me down a path of building my own blogging engine in Dart (server and client, no less!) which included advanced routing, authentication middleware, custom elements in Polymer.dart way before Polymer.dart was close to ready, wiring in non-Dart services like syntax highlighting and on, and on, and on...

Eventually and mercifully I declared bankruptcy on that idea (even thought I still hold out hope for a clean and simple Dart-based blogging engine), but my next attempt wasn't any more productive.

I researched blogging platforms, looked at their source code, their table layouts, theming engines, and confidently settled on Ghost. Great, but did I mention my complexifier nature? Sure enough, it got the better of me and suddenly was I neck deep in Docker, running Ghost unconventionally (as an Express module), Google Compute Engine options, all trying to achieve immutable infrastructure nirvana. To top it off I started my own Ghost theme using Polymer elements.

Whoa...

Realizing that way madness lied, at least if I ever wanted to actually get my blog up, I took a step back and asked myself:

What's the minimal amount of work I can do to get up and running, while leaving myself just enough flexibility to build what I want in the future?

Run my own server, so I can easily customize the blog, as well as run my own programs.
Control my URLs, and keep them from changing over time. I want to mount my blog at /blog so that I can put other things at the top level without fear of collisions.
Analytics

And... that's about it. Themes, syntax highlighting, even comments, can all wait.

With that minimal goal, I looked at Google Compute Engine's Cloud Launcher and the Bitnami Ghost installer. I was a little wary that there would be some catch with Bitnami: maybe they've modified Ghost in some unscrupulous way, or charge extra money. I don't think this is the case, so I clicked the "Get Started" button, and 10 minutes later I had this blog.

Sweet!

Justin Fagnani

What should a native DOM templating API look like?

What are we building?

What's guiding us?

Constraints

Requirements

Opinions

Template syntax

Template expressions

Alternatives to template expressions

Template container syntax.

Tagged template literals

Benefits of tagged template literals

Downsides of tagged template literals

Aren't strings bad?

Bindings and template parts

Attributes, properties, and events

Isn't this not "just HTML" anymore?

Composition, conditionals, and looping

Updating and reactivity

Template re-rendering

Fine-grained reactivity

Why not JSX again?

Why not templates in HTML itself?

Isn't this just standardizing lit-html?

Summary

likes on Bluesky

The time is right for a DOM templating API

Why right now is a great time to add templating

We know what good syntax for templating looks like

We can build an API for today's JavaScript

We can also plan for tomorrow's JavaScript

We can lay the groundwork for HTML-based templating

We've explored the reactivity landscape

A path to a proposal

likes on Bluesky

Stop Using CustomEvent

What's wrong with CustomEvent?

CustomEvent shouldn't exist!

CustomEvent is a leaky implementation detail

What to use instead of CustomEvent

Do you really need custom data?

Is there a native event that meets your needs?

Subclassing Event

Further reading

likes on Bluesky

Composite Map Keys in JavaScript with Bitsets

Nested Maps

Composite string keys

When your keys are already strings

When your keys are objects

BigInt Bitsets

likes on Bluesky

I'm Going to Need to Blog a Lot More

Reactive State with Signals in Lit

Lit Reactivity Overview

Lit dataflow demo

Great, so what's missing?

Lit + Signals

Cool, but why am I so excited about signals? 🤔

Interoperable signals

Universal reactive primitive

Example: AsyncComputed

A reactive type for web standards

Looking forward with @lit-labs/signals

How to Publish Web Components to NPM

Justin's Checklist for Publishing Web Components to NPM™

Publish standard ES2017

Publish standard JavaScript modules

Do not use .mjs file extensions

Only publish a single build

Important package.json fields

Set "type" to "module"

Set "main" to the main entry point module

Set "module" to the same file as "main"

Include polyfills in devDependencies, not dependencies

Do not bundle

Do not minify

Always self-define elements

Export element classes

Looking forward with `@lit-labs/signals`

`super` doesn't work.

`super` works.