Responsive Images 201: Client Hints

Written by Jason Grigsby on

Last year, I wrote a ten-part series called Responsive Images 101 that introduced the new responsive images syntax. Today, I want to continue that series by talking about Client Hints.

As the title indicates, this is an advanced responsive images topic. However, despite being potentially complex to implement, Client Hints can make the day-to-day task of creating responsive images much easier for designers and developers.

What are Client Hints?

Client Hints define a way for browsers to tell the server what the types of content that the browser prefers. It provides this information via HTTP request header fields.

Every request that a browser makes to a server contains HTTP request headers. For example, the headers that Firefox sends to request our home page look like this:

GET / HTTP/1.1
Host: cloudfour.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: _ga=GA1.2.333160365.1465672418
Connection: keep-alive

You can see HTTP request headers in most developer tools in the network pane.

Chrome developer tools showing the HTTP request headers
Chrome developer tools showing the HTTP request headers for Cloudfour.com

Client Hints add additional HTTP request header fields containing information about the the browser. One of the main uses of these additional headers is to send information about the size of images required for the page.

Client Hints for responsive images

There are three Client Hints that are relevant for responsive images:

  • DPR — Device pixel ratio (e.g., 1x, 2x, etc.)
  • Viewport-Width — The size of the viewport.
  • Width — The width of the image in the page.

Of these three, I find DPR and Width the most useful. As I explained in the Responsive Images 101 series, viewport size can be a poor substitute for the actual size of an image.

Viewport tells us little about the size of an image in the page

What could we do if the browser provided width and DPR information to the server and the server knew what to do with it?

Client Hints can mean much simpler markup

Let’s take a look at a code sample from the Sizes section of the 101 series:

<img src="cat.jpg" alt="cat"
  srcset="cat-160.jpg 160w,
          cat-320.jpg 320w,
          cat-640.jpg 640w,
          cat-1280.jpg 1280w"
  sizes="(max-width: 480px) 100vw,
         (max-width: 900px) 33vw,
         254px">

The reason for the addition of the srcset attribute in the markup is to provide the browser with different image sizes that the browser can request. However, if the server knew the size of the image in the page, it could simply provide an appropriately-sized image to the browser.

That’s what the Width Client Hint provides. It adds the width of the image, calculated using the information in the sizes attribute, to the HTTP headers that are sent to the server.1

GET cat.jpg
Accept: image/webp,image/*,*/*;q=0.8
DPR: 2
Viewport-Width: 1024
Width: 508

Therefore, if the browser and the server both support Client Hints, the markup can be simplified to be:

<img src="cat.jpg" alt="cat"
     sizes="(max-width: 480px) 100vw,
       (max-width: 900px) 33vw,
       254px">

Making this change takes all of the complexity of picking responsive image breakpoints out of the hands of designers and developers and puts the onus on the server.

Support multiple image formats

We’ve already made some big improvements in our markup by using Client Hints, but there is more that can be done if we couple Client Hints with another HTTP request header: Accept.

The Accept header is used by the browser to tell the server what types of media the browser considers acceptable for the current request. For example, when Chrome requests an image, it adds the following Accept header:

Accept: image/webp,image/*,*/*;q=0.8

This tells the server that Chrome prefers WebP image formats, but if WebP isn’t available, then send a file with a mime type that starts with image (image/*). If you don’t have that, then send whatever you’ve got (*/*).

The last little bit, q=0.8, tells the server how much the browser prefers the specified format. You can find more on how the quality preference parameter works in the specification.

Adding the Accept header allows us to simplify responsive images markup used to support multiple image formats. Here is the “before” code:2

<picture>
  <!-- serve WebP to Chrome and Opera -->
  <source
    media="(min-width: 50em)"
    sizes="50vw"
    srcset="/image/thing-200.webp 200w, /image/thing-400.webp 400w,
        /image/thing-800.webp 800w, /image/thing-1200.webp 1200w,
        /image/thing-1600.webp 1600w, /image/thing-2000.webp 2000w"
    type="image/webp">
  <source
    sizes="(min-width: 30em) 100vw"
    srcset="/image/thing-crop-200.webp 200w, /image/thing-crop-400.webp 400w,
        /image/thing-crop-800.webp 800w, /image/thing-crop-1200.webp 1200w,
        /image/thing-crop-1600.webp 1600w, /image/thing-crop-2000.webp 2000w"
    type="image/webp">
  <!-- serve JPEGXR to Edge -->
  <source
    media="(min-width: 50em)"
    sizes="50vw"
    srcset="/image/thing-200.jpgxr 200w, /image/thing-400.jpgxr 400w,
        /image/thing-800.jpgxr 800w, /image/thing-1200.jpgxr 1200w,
        /image/thing-1600.jpgxr 1600w, /image/thing-2000.jpgxr 2000w"
    type="image/vnd.ms-photo">
  <source
    sizes="(min-width: 30em) 100vw"
    srcset="/image/thing-crop-200.jpgxr 200w, /image/thing-crop-400.jpgxr 400w,
        /image/thing-crop-800.jpgxr 800w, /image/thing-crop-1200.jpgxr 1200w,
        /image/thing-crop-1600.jpgxr 1600w, /image/thing-crop-2000.jpgxr 2000w"
    type="image/vnd.ms-photo">
  <!-- serve JPEG to others -->
  <source
    media="(min-width: 50em)"
    sizes="50vw"
    srcset="/image/thing-200.jpg 200w, /image/thing-400.jpg 400w,
        /image/thing-800.jpg 800w, /image/thing-1200.jpg 1200w,
        /image/thing-1600.jpg 1600w, /image/thing-2000.jpg 2000w">
  <source
    sizes="(min-width: 30em) 100vw"
    srcset="/image/thing-crop-200.jpg 200w, /image/thing-crop-400.jpg 400w,
        /image/thing-crop-800.jpg 800w, /image/thing-crop-1200.jpg 1200w,
        /image/thing-crop-1600.jpg 1600w, /image/thing-crop-2000.jpg 2000w">
  <!-- fallback for browsers that don't support picture -->
  <img src="/image/thing.jpg" width="50%">
</picture>

There’s a lot going on in that example—multiple image formats, numerous image sources, and art direction. It is a bit of an extreme example.

With Client Hints and the Accept header combined, it can be simplified to:

<picture>
  <source media="(min-width: 50em)"
          sizes="50vw"
          srcset="/image/thing">
  <img sizes="100vw" src="/image/thing-crop">
</picture>

Amazing, right?

Notice that the paths for the images in the simplified markup do not have a file extension. That’s because the Accept header will tell the server what image formats the browser accepts and what preference it has. It is then up to the server to pick the best image format to send back.

Forget srcset and picture, give me Client Hints!

Some of you may be wondering why you’ve spent so much time learning srcset, picture and the rest of the responsive images syntax if Clients Hints makes it so simple.3 But there are reasons why Client Hints are a Responsive Images 201 topic.

Not all browsers support Client Hints

At the moment, only Chrome and Opera support Client Hints. They are under consideration by the Microsoft Edge and Firefox teams.

You need a server in the mix

HTML can be used without a server. Perhaps you’re building an eBook or something that works primarily offline. In those scenarios, you may need responsive images markup.

Your server has to support Client Hints

The server needs to know what to do with the Client Hints HTTP request headers. We may eventually see servers like Apache and NGINX add support for Client Hints, but it will take some time because it will require them to not only support the HTTP headers, but also add image processing capabilities.

In the short run, the way to utilize Client Hints is by signing up for an image resizing service that supports the specification. I know that Cloudinary, imgix and ScientiaMobile all support Client Hints.

There may be additional services that support Client Hints that I’m unaware of, so I’ve added a column for Client Hints to the spreadsheet of image resizing services so we can track support. If you know of any services that should be updated, please let me know.

Supporting Client Hints on the server isn’t easy

Like many image-related topics, supporting Client Hints on the server sounds easy at first. The browser tells the server the size of the image in the page. The server returns the correctly-sized image.

But doing that would cause all sorts of problems as Eric Portis pointed out on the Cloudinary blog:

To deliver perfectly-scaled resources, our servers must render and store hundreds, maybe thousands of alternate versions of your image, which is both computationally expensive and terrible for your pages’ performance. More, alternate resources means fewer cache hits, and a hit at any level — in the local browser cache, on the CDN, or server-side — will always be significantly faster than a miss.

You probably don’t want small adjustments in the size of the viewport, or a rotation event, to cause the image that has been downloaded to no longer look good. This is one of the side benefits of downloading larger responsive images than needed—resizing the browser doesn’t necessarily mean the image will look bad.

You will also want images to get cached. And you definitely don’t want to pay exorbitant amounts to cache dozens of image sources on your CDN.

This is where the various image resizing services can set themselves apart. If we give them Client Hints, how smart are they about providing the best possible image size and format?

You have to opt-in to Client Hints

Because browser makers don’t want to add excess HTTP headers to every request unnecessarily, you have to explicitly opt in to Client Hints by adding a <meta> tag:

<meta http-equiv="Accept-CH" content="DPR, Viewport-Width, Width">

You should only list the Client Hints that you’re going to use in the comma-separated list to avoid excess HTTP headers.

Wherein Client Hints are both the chicken and the egg

Opting in via the <meta> tag presents a conundrum for your markup. When the browser requests the HTML document from the server, the browser won’t supply any HTTP header information that indicates to the server that the browser supports Client Hints because the browser hasn’t yet seen the <meta> tag declaring browser support.

The server can add the <meta> tag into the HTML document that it returns to the browser, but what does it do with the markup for responsive images within that same HTML document? The server doesn’t know yet if the browser will support the <meta> tag request for Client Hints.

I asked Ilya Grigorik, the Google engineer who has led the charge for Client Hints, if browsers don’t declare their support before the HTML is delivered, do you have any suggestions on how to support browsers that support client hints and those that don’t?

Ilya replied:

In the meantime, I guess you can run a simple UA check (Chrome 46+) if you want to customize the HTML markup — yes, I recognize this is a bit ironic given that CH is all about removing the need for sniffing the UA string :-)

Before you throw your keyboard out the window, there are good reasons why Client Hints work this way.

Using HTTP request headers to define what content the server should send back is commonly referred to as Content Negotiation. Browser vendors have a long-standing aversion to Content Negotiation and for good reasons.

The opt-in nature of Client Hints is one of the ways that Ilya has tried to address these concerns and get buy-in from browser vendors. Instead of adding HTTP headers to every request whether or not the server requires it, the opt-in limits it to servers that are ready to use the information.

How do we solve this conundrum?

If the server doesn’t know if Client Hints are supported before generating the markup, how do we know what markup to generate. Assuming we want to support Client Hints, there are two possible solutions:

  • Server-side user agent detection
  • Supply both Client Hints and responsive images markup

I confirmed that if you supply both the normal responsive images markup and the <meta> tag for Client Hints, that the browser will evaluate the responsive images markup and request the appropriate image source using the Client Hints HTTP headers.

So the simplest solution in the short run may be to use both Client Hints and the responsive images markup.

In reality, whatever implementation makes the most sense is likely going to be dictated by whatever image resizing service you’re using. I’d recommend talking to your service provider about what they provide.

A woman in a superhero t-shirt
Photo by Ashley Rose

You still need responsive images syntax (for now)

Client Hints provide a glimmer of a promising future for responsive images. When Client Hints have broad support, the only responsive images syntax you need will be sizes.4 This is why I consider sizes to be the superhero of responsive images.

Client Hints also remind us that humans shouldn’t be in the business of processing images or picking image breakpoints. As I stated at the end of my Responsive Images 101 series, “The goal for most organizations should be to centralize image resizing and processing and automate as much of their responsive images as possible.”


  1. The first version of this article listed the value of the width header at 254 which is the size of the image in the page in CSS pixels. Johannes Koch pointed out in a comment that the spec says that it should send the physical width of the image, not the CSS pixel width. I confirmed that Johannes is correct and that the spec changed from CSS pixels to the intrinsic width of the image. 
  2. Code examples borrowed from Ilya Grigorik’s great introduction to Client Hints. 
  3. A better question might be why I spent days writing a 10-part series on the topic. 
  4. Unless you’re doing art direction which still requires <picture> because of the manual editing. 
Jason Grigsby

Jason Grigsby is one of the co-founders of Cloud Four, Mobile Portland and Responsive Field Day. He spends far too much time obsessing over mobile and the web. Follow him at @grigs.

Comments

Add a comment

Hi Jason,

thanks for you article. Please note, that according to http://httpwg.org/http-extensions/client-hints.html#width the Width cient hint is in physical pixels, not in CSS pixels (unlike Viewport-Width!). So the HTTP request example in 'Client Hints can mean much simpler markup' should include

Width: 508

instead of

Width: 254

if the DPR is 2.

Awesome post, once again!

I always thought using content negotiation was really nice. I used it in the 90's, mostly for Accept-Language, with great success. But I understand from the WHATWG page you link to that it's not recommended anymore.

Using Cloudinary on my Jekyll personal site, I have to check how to use Client Hints for images, but how do you "use both Client Hints and the responsive images markup"? Duplicate the HTML code and hide the one that's not used?

Replies

I'm not sure how Cloudinary will handle it, but say you had markup like this on a web page that also included the Accept-CH meta tag :

<img src="cat.jpg" alt="cat"
  srcset="cat-160.jpg 160w,
          cat-320.jpg 320w,
          cat-640.jpg 640w,
          cat-1280.jpg 1280w"
  sizes="(max-width: 480px) 100vw,
         (max-width: 900px) 33vw,
         254px">

And the viewport was 1024px wide on a 1x screen. The browser would decide that the size of the image in the page was 254px (via sizes) and would most likely decide to download cat-320.jpg (next largest image available).

When the browser sent of the request for cat-320.jpg, the browser would include the Client Hint request headers that were specified in the Accept-CH meta tag. At that point, Cloudinary (or whatever the service was), could choice to sub out the requested image with an alternative one—maybe one sized closer to 254px or maybe the image is smaller in a different image format.

Whether or not any of the existing services handle it that way is unknown to me. All I was commenting on is the browser behavior if the meta tag and the responsive images markup are both present.

(not really sure if using the "reply" on my own comment will work, but there is none on your reply…)

Ok, thanks for the explanation. I thought you planned to put both syntaxes from your article. It makes more sense now.

Leave a Comment

Please be kind, courteous and constructive. You may use simple HTML or Markdown in your comments. All fields are required.


Let’s discuss your project! Email Us