ideas@sullice.com

A New Era for Drupal's JSON:API

20 September, 2019

On Monday (2019-9-16), I published the 8.x-1.0 version of the JSON:API Hypermedia module (hypermedia is just a fancy name for things with links). It took lots of work in my current role at Acquia and I depended on the help of my friends (and colleagues) Peter Weber (zrpnr), Wim Leers (same), and Mateu Aguiló Bosch (e0ipso).

I believe this opens a new era in the decoupled Drupal ecosystem.

What does this module do? It provides an API for modules to add links to JSON:API responses.

Neat, right?!

Oh… why does that matter, you ask? Well, before I answer that, let’s think a little more deeply about the web that we know and love and the links that drive it.

Consider what it would be like to build a website without links. It would have no menus, no login button, no links between related pages, and no pagination. Already, that sounds like a fairly terrible website.

Yet links on the web are far more prevalent than the ubiquitous <a> element. If we look a little deeper, there are many, many more. Without links, the web would be little more than an internet-connected file browser.

Each of these HTML elements represents a kind of link on the web too:

Of course, let’s not forget that we use links in CSS as well. What’s background-image: url(...) if not a link?

If we couldn’t build a website with links, we couldn’t build much of a website at all! Beyond helping a user navigate between websites and webpages, links enrich every page we visit. They add images, videos and audio; they add user interactivity; they make a webpage visually appealing and intuitive.

Maybe you paused before and asked yourself, “is <form> really a link?” Sure it is. A <form> element’s action attribute takes a URL and its method attribute takes an HTTP method (GET or POST). Similarly to how your web browser knows that when a user clicks on an <a> element that:

Your web browser also knows that when a user clicks on a <form> element’s inner <button> or <input type=”submit”> element that:

While the browser’s steps to follow the <form> link are slightly more complicated, we can see that fundamentally, the <a> and <form> elements both represent instructions to the browser and each represents a relationship between one resource on the web and another (a location that the user can visit and a location where a form can be submitted, respectively).

Likewise, an <img> element tells your browser:

However, unlike the previous two examples, processing an <img> link doesn’t refresh a browser tab. Instead, it updates the existing rendering with the newly fetched data.

Going further, the <script> link is the most powerful of them all. It tells your browser:

If that link is not awesome enough already, consider that the script itself can add links to the DOM (say, to lazy load images or fetch other scripts).

Finally, links are not just for browsers. Other utilities, like a search engine crawler, can process links for purposes not visible to a user at all. The <link rel="canonical"> element is a familiar example. This link is often used to establish a “canonical URL” for a search result. Under the hood, the link is giving instructions to the crawler. It tells the crawler:

How does this understanding of links relate to decoupled Drupal and a new contributed module? I believe that when many of us think of REST APIs, we don’t realize that all of the richness provided by links in HTML is also available to our JSON-based APIs. Moreover, little prevents those links in JSON from being just as functional as a <form> element. In fact, we’ll see later that they can be made more functional than anything we’ve already seen. That’s because links in our own APIs can be extended and customized for our particular applications.

Let’s stop for a moment and internalize this: our web browsers are, at their core, highly sophisticated REST clients that understand responses of Content-Type: text/html.

Now, what if our JSON:API clients could take as much advantage of links as browsers do? Could our single-page JavaScript applications mature into sophisticated REST clients that understand responses of Content-Type: application/json, or, in Drupal’s JSON:API case, Content-Type: application/vnd.api+json?

Absolutely! In the same way that a browser “knows” how to parse and process links in a text/html document (based on whether the link is an <img> or <form> element) a client that understands application/vnd.api+json documents can parse and process many different kinds of links too. This means that if we can enrich JSON:API responses with more dynamic and powerful links, we can unlock a new realm of sophistication for our decoupled projects.

A hyperdrive engine for decoupled applications

As Drupal modules start to provide those more dynamic and powerful links, Drupal will be able to power native applications, kiosks, web apps and more in ways that it previously hasn’t been able to. Drupal will grow from being a mere content repository for those decoupled applications into an engine that drives those decoupled applications.

What I find most exciting about this new era is that decoupled Drupal implementations will be able to do much more than replicate the HTML links we examined above, they’ll be able to create new types of links of their own!

So, let’s do it! Let’s define a format for JSON link objects—and don’t worry if these definitions seem dense, there’s an interactive example further below that will bring it all together. Let’s say that a link object can have the following properties:

As a concrete example, let’s imagine that we operate an online store and users can purchase a product if it’s in stock or save it to their wishlist otherwise.

If we write a client that understands the generic link object defined above, we’d be able to add an action link to every product. If a product is in stock, the server would set the title to “Buy now!” and, if not, it’d set the title to “Save for later”. Take a look at the example below and try changing the value of the Stock field, observe how the link adapts. Notice that when the title property changes, the button adapts accordingly. Go ahead and click on the link. When the product is out of stock, the confirm property becomes false and the button processes the link immediately. When the confirm property is a string, there’s an extra confirmation button to click.

There’s something very exciting hiding in plain sight: because the link adapts to product availability, the client doesn’t have to. In fact, the client doesn’t need to know anything about how a product is stocked at all! Without that dynamic link, the client would otherwise have had to check some stock field or to have had to make an HTTP request to determine if the product was available.

This pattern means that months or years later, when some developers realize that products can be discontinued, all that must be changed is the link. By updating the link to direct the client to a different product and without changing a single byte of client code, the application could evolve to support an entirely new scenario! Try checking the Discontinued checkbox above. The new feature just works.

Had the client been checking the availability of the product, not only would the server have had to make a new endpoint or field to communicate that a product is discontinued, the client would need to been updated in lock-step with that change 😨.

You don’t have to take my word for it, the button-generating code in the example doesn’t “know” anything about products, wishlists, stocks or carts. It only knows how to process the generic link we defined. Here’s a link to the demo’s source code.

When you have complete control over both the client and the server (e.g. in a progressively decoupled block) this pattern might seem nice, but not necessary. However, consider that as applications—and teams—grow in size and complexity, this pattern helps you move more quickly. What if your client were a single-page JavaScript application? What if it were a native mobile application on a phone that rarely gets updated? What if you had to maintain both? Finally, even if you do have complete control over the client and the server, what if links could be defined by contrib modules? Wouldn’t you want them to use a similar pattern?

A toolbox, not an engine

I began this post by asking what the JSON:API Hypermedia module does. The answer was simple: “It provides an API for modules to add links to JSON:API responses.” That answer hasn’t changed. However, I hope that by seeing all of what this actually enables, you’re as excited as I am!

It would certainly have been possible to add lots of new, hardcoded links to the JSON:API module in order to create a tighter integration with Drupal core’s features (like publishing and unpublishing content), but Drupal shines at its brightest when users can combine specialized modules to serve their exact needs. Now, modules have a place to provide those specialized behaviors for the decoupled ecosystem too.

That’s why I believe this opens a new era in the decoupled Drupal ecosystem.

Imagine if the Flag module starts decorating JSON:API responses with links to create bookmarks, or if the Commerce module started decorating them with links to purchase products, or that your module might start decorating responses with links to drive your own custom components!

Wrapping up

Over the coming weeks, I will be publishing more posts that explore the possibilities that the JSON:API Hypermedia module unlocks in greater detail. Each post will dive deeper into the ideas that informed this blog post. To do that, the posts will be broken up into different categories—links that navigate, links that instruct, and links that describe—and each will give examples of when, where and how you can use those links to solve otherwise difficult and inelegant problems. My goal will be to show that many of the “hardest problems” in decoupled Drupal development are simplified through the use of hypermedia—be it decoupled menus, access control and user interactions, or schemas and developer tools.