Hypermedia to the Rescue

Linking has always been a key component of REST architecture. Of course, links are familiar in UI contexts such as browsers; for example, consider a “See details” link for obtaining the details of a given product in a catalog. But what about computer-to-computer scenarios where there’s no UI or user interaction? The idea is that you can use hypermedia artifacts in these scenarios as well. 

With this new approach, the server doesn’t only return data. It returns data and hypermedia artifacts. The hypermedia artifacts give the client a way to determine the available set of actions that can be performed at a given point based on the state of the server application workflow.

This is one area that typically differentiates a regular Web API from a RESTful API, but there are other constraints that also apply, so discussing whether an API is RESTful or not probably doesn’t make sense in most cases. What matters is that the API uses HTTP correctly as an application protocol and leverages hypermedia when possible. By enabling hypermedia, you can create self-discoverable APIs. This is no excuse for not providing documentation, but the APIs are more flexible in terms of updatability.

Which hypermedia artifacts are available is mainly determined by the chosen media types. Many of the media types we use nowadays for building a Web API, such as JSON or XML, don’t have a built-in concept for representing links or forms, as HTML does. You can leverage those media types by defining a way to express hypermedia, but this requires that clients understand how the hypermedia semantics are defined on top of those. In contrast, media types such as XHTML (application/xhtml+xml) or ATOM (application/atom+xml) already support some of these hyper­media artifacts such as links or forms.

In the case of HTML, a link is made up of three components: an “href” attribute pointing to a URL, a “rel” attribute for describing how the link relates to the current resource, and an optional “type” attribute for specifying the expected media type. For example, if you want to expose a list of products in a catalog using XHTML, the resource payload might look like what’s shown in Figure 1.

<div id="products">
  <ul class="all">
    <li>
      <span class="product-id">1</span>
      <span class="product-name">Product 1</span>
      <span class="product-price">5.34</span>
      <a rel="add-cart" href="/cart" type="application/xml"/>
    </li>
    <li>
      <span class="product-id">2</span>
      <span class="product-name">Product 2</span>
      <span class="product-price">10</span>
      <a rel="add-cart" href="/cart" type="application/xml"/>
    </li>
  </ul>
</div>

In this example, the catalog of products is represented with standard HTML elements, but I used XHTML because it’s much easier to parse with any existing XML library. Also, as part of the payload, an anchor (a) element has been included, representing a link for adding the item to the current user cart. By looking at the link, a client can infer its usage via the rel attribute (add a new item), and use the href for performing an operation over that resource (/cart). It’s important to notice that the links are generated by the server based on its business workflow, so the client doesn’t need to hardcode any URL or infer any rule. This also offers new opportunities to modify the workflow during runtime without affecting the existing clients at all. If any of the products offered in the catalog are out of stock, the server can simply omit the link for adding that product to the cart. From the client perspective, the link isn’t available so the product can’t be ordered. More complex rules related to that workflow might apply on the server side, but the client is totally unaware of that, as the only thing that matters is that the link isn’t present. Thanks to hypermedia and links, the client has been decoupled from the business workflow on the server side.

Moreover, the evolvability of an API design can be improved with hypermedia and links. As the business workflow on the server evolves, it can offer additional links for new functionality. In our product catalog example, the server might include a new link for marking a product as a favorite, like this:

<li>
  <span class="product-id">1</span>
  <span class="product-name">Product 1</span>
  <span class="product-price">5.34</span>
  <a rel="add-cart" href="/cart/1" type="application/xml"/>
  <a rel="favorite" href="/product_favorite/1" type="application/xml"/>
</li>

While existing clients might ignore that link and remain unaffected by this new functionality, newer clients can start using it right away. In this way, it wouldn’t be crazy to think of having a single entry point or root URL for your Web API that contains links to discover the rest of the functionality. For example, you could have a single URL “/shopping_cart” that returns the following HTML representation:

<div class="root">
  <a rel="products" href="/products"/>
  <a rel="cart" href="/cart"/>
  <a rel="favorites" href="/product_favorite"/>
</div>

Analogous functionality is also found in OData services, which expose a single service document in the root URL with all the supported resource sets and links for getting the data associated to them.

Links represent a great way to connect servers and clients, but there’s an evident problem with them. In the previous example with the product catalog, a link in HTML offers only the rel, href and type attributes, which implies some out-of-band knowledge about what to do with that URL expressed in the href attribute. Should the client use an HTTP POST or HTTP GET? If it uses a POST, what data should the client include in the request body? While all that knowledge could be documented somewhere, wouldn’t it be great if the clients could actually discover that functionality? For all these questions, using HTML forms is the answer that makes a lot of sense.

Forms in Action

When you interact with the Web using a browser, actions are typically represented with forms. In the product catalog example, pressing the Add to Cart link implies an HTTP GET sent to the server to return an HTML form that can be used to add the product to the cart. That form could contain an “action” attribute with a URL, a “method” attribute representing the HTTP method, and some input fields that might require input from the user, as well as some readable instructions to move forward.

You can do the same for a machine-to-machine scenario. Rather than having a human being interacting with a form, you might have an application running JavaScript or C#. In the product catalog, an HTTP GET to the “add-cart” link for the first product would retrieve the following form represented in XHTML:

<form action="/cart" method="POST">
  <input type="hidden" id="product-id">1</input>
  <input type="hidden" id="product-price">5.34</input>
  <input type="hidden" id="product-quantity" class="required">1</input>
  <input type="hidden" id="___forgeryToken">XXXXXXXX</input>
</form>

The client application has now been decoupled from certain details related to adding the product to the cart. It just needs to submit this form using an HTTP POST to the URL specified in the action attribute. The server can also include additional information in the form—for example, a forgery token to avoid cross-site request forgery (CSRF) attacks or to sign the data that’s prepopulated for the server.

This model allows any Web API to freely evolve by offering new forms based on different factors such as user permissions or the version the client wants to use.

Hypermedia for XML and JSON?

As I mentioned earlier, the generic media types for XML (application/­xml) and JSON (application/json) don’t have a built-in support for hypermedia links or forms. While it’s possible to extend those media types with domain-specific concepts such as “application/vnd-shoppingcart+xml,” this requires new clients to understand all the semantics defined in that new type (and it would probably also generate a proliferation of media types) so it’s generally not considered a good idea.

For that reason, a new media type that extends XML and JSON with linking semantics and is called Hypertext Application Language (HAL) has been proposed. The draft, which simply defines a standard way to express hyperlinks and embedded resources (data) using XML and JSON, is available at stateless.co/hal_specification.html. The HAL media type defines a resource that contains a set of properties, a set of links and a set of embedded resources, as shown in Figure 2.

Figure 3 shows an example of how a product catalog would look in HAL using both the XML and JSON representations. Figure 4 is the JSON representation for the sample resource.

<resource href="/products">
  <link rel="next" href="/products?page=2" />
  <link rel="find" href="/products{?id}" templated="true" />
  <resource rel="product" href="/products/1">
    <link rel="add-cart" href="/cart/" />
    <name>Product 1</name>
    <price>5.34</price>
  </resource>
  <resource rel="product" href="/products/2">
    <link rel="add-cart" href="/cart/" />
    <name>Product 2</name>
    <price>10</price>
  </resource>
</resource>

{
  "_links": {
    "self": { "href": "/products" },
    "next": { "href": "/products?page=2" },
    "find": { "href": "/products{?id}", "templated": true }
  },
  "_embedded": {
    "products": [{
      "_links": {
        "self": { "href": "/products/1" },
        "add-cart": { "href": "/cart/" },
      },
      "name": "Product 1",
      "price": 5.34,
    },{
      "_links": {
        "self": { "href": "/products/2" },
        "add-cart": { "href": "/cart/" }
      },
      "name": "Product 2",
      "price": 10
    }]
  }
}

Supporting Hypermedia in ASP.NET Web API

So far I’ve discussed some of the theory behind hypermedia in the design of Web APIs. Now let’s see how that theory can actually be implemented in the real world using ASP.NET Web API, with all the extensibility points and features this framework provides.

At a core level, ASP.NET Web API supports the idea of formatters. A formatter implementation knows how to deal with a specific media type, and how to serialize or deserialize it into concrete .NET types. In the past, support for new media types was very limited in ASP.NET MVC. Only HTML and JSON were treated as first-class citizens and fully supported across the entire stack. Furthermore, there was no consistent model for supporting content negotiation. You could support different media type formats for the response messages by providing custom ActionResult implementations, but it wasn’t clear how a new media type could be introduced for deserializing request messages. This was typically solved by leveraging the model-binding infrastructure with new model binders or value providers. Fortunately, this inconsistency has been solved in ASP.NET Web API with the introduction of formatters.

Every formatter derives from the base class System.Net.Http.For­matting.MediaTypeFormatter and overrides the method CanReadType/ReadFromStreamAsync for supporting deserialization and the method CanWriteType/WriteToStreamAsync for supporting serialization of .NET types into a given media type format.

Figure 5 shows the definition of the MediaTypeFormatter class.

public abstract class MediaTypeFormatter
{
  public Collection<Encoding> SupportedEncodings { get; }
  public Collection<MediaTypeHeaderValue> SupportedMediaTypes { get; }
  public abstract bool CanReadType(Type type);
  public abstract bool CanWriteType(Type type);
  public virtual Task<object> ReadFromStreamAsync(Type type, Stream readStream,
    HttpContent content, IFormatterLogger formatterLogger);
  public virtual Task WriteToStreamAsync(Type type, object value,
    Stream writeStream, HttpContent content, TransportContext transportContext);
}

Formatters play a very important role in ASP.NET Web API in supporting content negotiation, as the framework can now choose the correct formatter based on the values received in the “Accept” and “Content-Type” headers of the request message.

The ReadFromStreamAsync and WriteToStreamAsync methods rely on the Task Parallel Library (TPL) for doing the asynchronous work, so they return a Task instance. In case you want to explicitly make your formatter implementation work synchronously, the base class, BufferedMediaTypeFormatter, does it for you internally. This base class provides two methods you can override in an implementation, SaveToStream and ReadFromStream, which are the synchronous versions of SaveToStreamAsync and ReadFromStreamAsync. 

Developing a MediaTypeFormatter for HAL

HAL uses specific semantics for representing resources and links, so you can’t use just any model in a Web API implementation. For that reason, one base class for representing a resource and another for a collection of resources are used to make the implementation of the formatter much simpler:

public abstract class LinkedResource
{
  public List<Link> Links { get; set; }
  public string HRef { get; set; }
}
public abstract class LinkedResourceCollection<T> : LinkedResource,
  ICollection<T> where T : LinkedResource
{
  // Rest of the collection implementation
}

The real model classes that the Web API controllers will use can derive from these two base classes. For example, a product or a collection of products can be implemented as follows:

public class Product : LinkedResource
{
  public int Id { get; set; }
  public string Name { get; set; }
  public decimal UnitPrice { get; set; }
}
...
public class Products : LinkedResourceCollection<Product>
{
}

Now, with a standard way to define HAL models, it’s time to implement the formatter. The simplest way to start a new formatter implementation is to derive from either the MediaTypeFormatter base class or from the BufferedMediaTypeFormatter base class. The example in Figure 6 uses the second base class.

public class HalXmlMediaTypeFormatter : BufferedMediaTypeFormatter
{
  public HalXmlMediaTypeFormatter()
    : base()
  {
    this.SupportedMediaTypes.Add(new MediaTypeHeaderValue(
      "application/hal+xml"));
  }
  public override bool CanReadType(Type type)
  {
    return type.BaseType == typeof(LinkedResource) ||
      type.BaseType.GetGenericTypeDefinition() ==
        typeof(LinkedResourceCollection<>);
  }
  public override bool CanWriteType(Type type)
  {
    return type.BaseType == typeof(LinkedResource) ||
     type.BaseType.GetGenericTypeDefinition() ==
       typeof(LinkedResourceCollection<>);
  }
  ...
}

The code first defines in the constructor the supported media types for this implementation (“application/hal+xml”), and overrides the CanReadType and CanWriteType methods to specify the supported .NET types, which have to derive from Linked­Resource or LinkedResourceCollection. Because it was defined in the constructor, this implementation only supports the XML variant of HAL. Another formatter could optionally be implemented to support the JSON variant.

The real work is done in the WriteToStream and ReadFromStream methods, shown in Figure 7, which will use an XmlWriter and XmlReader, respectively, to write and read an object into and out of a stream.

public override void WriteToStream(Type type, object value,
  System.IO.Stream writeStream, System.Net.Http.HttpContent content)
{
  var encoding = base.SelectCharacterEncoding(content.Headers);
  var settings = new XmlWriterSettings();
  settings.Encoding = encoding;
  var writer = XmlWriter.Create(writeStream, settings);
  var resource = (LinkedResource)value;
  if (resource is IEnumerable)
  {
    writer.WriteStartElement("resource");
    writer.WriteAttributeString("href", resource.HRef);
    foreach (LinkedResource innerResource in (IEnumerable)resource)
    {
      // Serializes the resource state and links recursively
      SerializeInnerResource(writer, innerResource);
    }
    writer.WriteEndElement();
  }
  else
  {
    // Serializes a single linked resource
    SerializeInnerResource(writer, resource);
  }
  writer.Flush();
  writer.Close();
}
public override object ReadFromStream(Type type,
  System.IO.Stream readStream, System.Net.Http.HttpContent content,
  IFormatterLogger formatterLogger)
{
  if (type != typeof(LinkedResource))
    throw new ArgumentException(
      "Only the LinkedResource type is supported", "type");
  var value = (LinkedResource)Activator.CreateInstance(type);
  var reader = XmlReader.Create(readStream);
  if (value is IEnumerable)
  {
    var collection = (ILinkedResourceCollection)value;
    reader.ReadStartElement("resource");
    value.HRef = reader.GetAttribute("href");
    var innerType = type.BaseType.GetGenericArguments().First();
    while (reader.Read() && reader.LocalName == "resource")
    {
      // Deserializes a linked resource recursively
      var innerResource = DeserializeInnerResource(reader, innerType);
      collection.Add(innerResource);
    }
  }
  else
  {
    // Deserializes a linked resource recursively
    value = DeserializeInnerResource(reader, type);
  }
  reader.Close();
  return value;
}

The last step is to configure the formatter implementation as part of the Web API host. This step can be accomplished in almost the same way as in ASP.NET or ASP.NET Web API Self-Host, with a single difference in the needed HttpConfiguration implementation. While Self-Host uses an HttpSelfHostConfiguration instance, ASP.NET typically uses the HttpConfiguration instance available globally in System.Web.Http.GlobalConfiguration.Configuration. The HttpConfiguration class provides a Formatters collection into which you can inject your own formatter implementation. Here’s how to do this for ASP.NET:

protected void Application_Start()
{
  Register(GlobalConfiguration.Configuration);
}
public static void Register(HttpConfiguration config)
{
  config.Formatters.Add(new HalXmlMediaTypeFormatter());
}

Once the formatter is configured in the ASP.NET Web API pipeline, any controller can simply return a model class derived from LinkedResource to be serialized by the formatter using HAL. For the product catalog example, the product and the collection of the products representing the catalog can be derived from LinkedResource and LinkedResourceCollection, respectively:

public class Product : LinkedResource
{
  public int Id { get; set; }
  public string Name { get; set; }
  public decimal UnitPrice { get; set; }
}
public class Products : LinkedResourceCollection<Product>
{
}

The controller ProductCatalogController, which handles all the requests for the product catalog resource, can now return instances of Product and Products as shown in Figure 8  for the Get method.

public class ProductCatalogController : ApiController
{
  public static Products Products = new Products
  {
    new Product
    {
      Id = 1,
      Name = "Product 1",
      UnitPrice = 5.34M,
      Links = new List<Link>
      {
        new Link { Rel = "add-cart", HRef = "/api/cart" },
        new Link { Rel = "self", HRef = "/api/products/1" }
      }
    },
    new Product
    {
      Id = 2,
      Name = "Product 2",
      UnitPrice = 10,
      Links = new List<Link>
      {
        new Link { Rel = "add-cart", HRef = "/cart" },
        new Link { Rel = "self", HRef = "/api/products/2" }
      }
    }
  };
  public Products Get()
  {
    return Products;           
  }
}

This example uses the HAL format, but you can also use a similar approach to build a formatter that uses Razor and templates for serializing models into XHTML. You’ll find a concrete implementation of a MediaTypeFormatter for Razor in RestBugs, a sample application created by Howard Dierking to demonstrate how ASP.NET Web API can be used to create hypermedia Web APIs, atgithub.com/howarddierking/RestBugs.

Formatters make it easy to extend your Web API with new media types.    

Better Linking Support in the Web API Controllers

Something is definitely wrong with the previous ProductCatalog­Controller example. All the links have been hardcoded, which might cause a lot of headaches if the routes change often. The good news is that the framework provides a helper class called System.Web.Http.Routing.UrlHelper for automatically inferring the links from the routing table. An instance of this class is available in the ApiController base class through the Url property, so it can easily be used in any controller method. This is what the UrlHelper class definition looks like:

public class UrlHelper
{
  public string Link(string routeName,
    IDictionary<string, object> routeValues);
  public string Link(string routeName, object routeValues);
  public string Route(string routeName,
    IDictionary<string, object> routeValues);
  public string Route(string routeName, object routeValues);
}

The Route methods return the relative URL for a given route (for example, /products/1), and the Link methods return the absolute URL, which is the one that can be used in the models to avoid any hardcoding. The Link method receives two arguments: the route name and the values to compose the URL.

Figure 9 shows how, in the previous product catalog example, the UrlHelper class could be used in the Get method.

public Products Get()
{
  var products = GetProducts();
  foreach (var product in products)
  {
    var selfLink = new Link
    {
      Rel = "self",
      HRef = Url.Route("API Default",
        new
        {
          controller = "ProductCatalog",
          id = product.Id
        })
    };
product.Links.Add(selfLink);
if(product.IsAvailable)
{
    var addCart = new Link
    {
      Rel = "add-cart",
      HRef = Url.Route("API Default",
        new
        {
          controller = "Cart"
        })
    };
    product.Links.Add(addCart);
  }           
}
  return Products;           
}

The link “self” for the product was generated from the default route using the controller name ProductCatalog and the product id. The link for adding the product to the cart was also from the default route, but used the controller name Cart instead. As you can see in Figure 9, the link for adding the product to the cart is associated to the response based on product availability (product.IsAvailable). The logic for providing links to the client will pretty much depend on the business rules typically enforced in the controllers.    

Wrapping Up

Hypermedia is a powerful feature that allows clients and servers to evolve independently. By using links or other hypermedia artifacts such as forms offered by the server at different stages, clients can be successfully decoupled from the server business workflow that drives the interaction.