Ending Responsive Images

Mat Marquis, former Responsive Images Working Group (RICG) chair and amateur boxer, isn’t known for pulling his punches. And yet, Marquis stopped short when he declared The End of Responsive Images. Mat left responsive images on the ropes. I hope to finish them.

Or rather, finish most responsive images. The ones below the fold. Let me explain.

In the beginning of our search for a responsive images solution, the challenge seemed simple: the img element only supported one source file, and we needed multiple sources to support responsive designs. However, if we supply multiple image sources, how will the browser know which source to choose?

Therein lies the rub. Browsers download images before they’ve calculated the layout of the page. And if the browser doesn’t know the page layout, it doesn’t know the size of the image in the page. As I put it back in 2012:

How do we reconcile a pre-parser that wants to know what size image to download ahead of time with an image technique that wants to respond to its environment once the page layout has been calculated?

Solving this problem is how we ended up with srcset and sizes. One tells the browser what source options it can choose from and the other provides the browser with the information it needs to calculate the size of the image in the page based on a measurement the browser always knows—the viewport size.

In the years since, lazy-loading images has become a common practice. We can safely defer loading images outside the initial view (aka, below the fold) in order to ensure that the browser’s resources are focused on the assets that are immediately visible.

So many developers adopted this practice that it is now a web standard. Adding loading="lazy" to any image will tell the browser to delay downloading the image until it will be visible in the viewport.

Because lazy-loaded images won’t download until they are visible, the race condition described earlier no longer exists. When lazy-loaded images are downloaded, the browser knows their size.

Lazy-loaded images now allow us to simplify the markup for sizes as Mat Marquis explained:

A few weeks ago, two patches landed in Gecko and WebKit — championed by Simon Pieters and Yoav Weiss, respectively, two of the RICG’s finest. These patches landed to little fanfare, quietly aligning Gecko and WebKit with Blink in supporting a relatively recent addition to the HTML specification: support for an auto value in sizes attributes. Automatic sizes — the potential sizes of the rendered image, left up to the browser to determine alongside all those other factors. Fully automatic responsive images. Supply the browser with a list of candidates using srcset, bolt on sizes="auto", and let the browser do the rest.

So long as an image has loading="lazy" you can use sizes="auto". Here is the sample Mat provided (with a few cat-related tweaks on my part):

<img 
  loading="lazy"
  src="cat.jpeg" 
  srcset="cat-650.jpeg 650w, cat-960.jpeg 960w, cat-1400.jpeg 1400w"
  sizes="auto, (min-width: 1040px) 650px, calc(94.44vw - 15px)"
  width="650"
  height="400"
  alt="My cat Boyle is super cute.">Code language: Handlebars (handlebars)

You’ll notice that Mat includes fallback sizes values to support older browsers. But with Safari and Firefox now supporting sizes="auto" we likely won’t need the fallback for long.

Caution: Images above the fold shouldn’t be lazy-loaded. In fact, hero images are likely your Largest Content Paint (LCP) image in Web Core Vitals and should be loaded as early as possible. Therefore, images in the initial viewport will likely always require responsive images syntax.

Simplifying sizes is a good start, but I believe we can go further and radically simplify the markup for lazy-loaded images using Responsive Image Client Hints. Responsive Image Client Hints is a draft specification being stewarded by Eric Portis also of RICG fame.

Responsive Image Client Hints builds on the existing Client Hints standard that has been implemented by Blink and thus supported by Chrome, Edge, and other browsers built on the Blink engine. Web page authors can declare support for Client Hints and ask for specific hints like the size of a requested image. The browser then sends this information in HTTP headers.

For example, a server might send the following HTTP headers along with an HTML document:

Accept-CH: Sec-CH-Width
Permissions-Policy: ch-width=(self "https://images.cloudfour.com")Code language: PHP (php)

The first line says that the server will accept client hints and is looking for Secure Client Hints Width (Sec-CH-Width). The permissions policy line helps reduce the potential for passive fingerprinting by restricting access to the specified domains. A similar declaration can be made via a meta tag:

<meta http-equiv="delegate-ch" 
      content="sec-ch-width https://images.cloudfour.com;">Code language: Handlebars (handlebars)

And then when an image on the page is requested by the browser, it will supply the requested Client Hints—in this case, the width of the image in the page:

GET cat.jpg
Accept: image/webp,image/*,*/*;q=0.8
Sec-CH-Width: 650Code language: YAML (yaml)

Sec-CH-Width value will contain the pixel width of the image in the page multipled by the display density. So if the image is 400 pixels wide on a 2x display, the value sent to the server will be Sec-CH Width: 800.

If both the browser and server support client hints, and the image is lazy-loaded, we can greatly simplify our markup:

<img 
  loading="lazy"
  src="cat.jpeg"
  width="650"
  height="400"
  sizes="auto"
  alt="My cat Boyle is super cute.">Code language: Handlebars (handlebars)

We no longer need srcset because the server knows what image sizes are available or can resize the image as needed based on the image size supplied by Client Hints.

Because the image is lazy-loaded, we also know the size of the image in the page which means we shouldn’t need sizes at all. Unfortunately, the Client Hints standard expects the sizes attribute so we still need to supply sizes="auto". Perhaps this can change in the future and we can remove the sizes attribute as well.

If we’re not supplying a list of image sources in srcset to the browser, how will one be selected? For this to work, you need an image resizing service on the server to respond to the image request. If you don’t have one, you can continue to use srcset and sizes.

If this approach takes off, image resizing services may want to change the way they select what image size to return. Instead of working from a list of image sizes that the web page author thinks might be needed, the server can monitor what sizes are requested the most and make sure those sizes are cached. The server can optimize the cached image sizes based on real world usage.

Client Hints aren’t new. Chrome shipped them in v46 back in 2015. I wrote about their possible uses in 2016. But developers haven’t widely adopted Client Hints because Firefox and Safari don’t support them.

Because of this reality, I gave up on Clients Hints for many years. Three developments have caused me to revisit them:

1. The proliferation of lazy-loading images and the standard to support them means we have a large body of images for which the speculative downloader’s race condition doesn’t apply.
2. In my review of ImageEngine, I was impressed with how simple it is to use. Some of this simplicity is due to leveraging Client Hints when they are available.
3. Eric Portis has created a small JavaScript library that provides similar functionality for Cloudinary users in browsers that don’t support Client Hints.

Eric’s script works like this:

• If the browser supports Responsive Image Client Hints, the script does nothing.
• If Client Hints aren’t supported, the script looks for images with the loading="lazy" attribute.
• It also checks for some specific Cloudinary parameters in image’s src URL which tell the script that this image is one that it should modify.
• The script then updates the width value in the Cloudinary URL to send along the actual size of the image in the page.

I’ve modified the sample markup from above to show how it would look using Cloudinary and supporting Eric’s script:

<img 
  loading="lazy"
  src="[…]/upload/c_limit,w_auto:breakpoints:650/v1716911991/catbeans_zbh3iu.jpg"
  width="650"
  height="366"
  sizes="auto"
  alt="My cat Boyle is super cute.">Code language: Handlebars (handlebars)

The key section in the image URL is w_auto:breakpoints:650:

• w_auto — This tells Cloudinary that the image is responsive and should be dynamically resized to fit the size of the image in the page.
• breakpoints — This tells Cloudinary to pick image sizes using a performance budget—a feature they created based on one of my articles. If performance budgets aren’t your speed, you can set a value for how many pixels before a new image is created on the server.
• 650 — this is the fallback width that should be used if the browser doesn’t send additional information via Client Hints.

If the image in the page was actually 900 pixels wide at 2x display density, the script would update the src to be:

  src="[…]/upload/c_limit,w_auto:breakpoints:1800/v1716911991/catbeans_zbh3iu.jpg"Code language: Bash (bash)

For browsers that don’t support Client Hints, the actual size of the image in the page is added to the URL. This technique could be modified to work with other image resizing servers.

There are a few caveats with this script:

• It must be placed in the <head>
• It has to be a blocking script. You can’t async, defer, or use type="module".

Eric explains:

This script needs to see <img>s as soon as they are parsed, so that it can prepare to re-write their URLs as soon as they’re laid out. If the script executes after an <img> has been parsed, it won’t get a chance to do any of that; it’ll do nothing.

Luckily, it’s only ~950 bytes (~600B compressed).

Eric has created a couple of CodePen examples that show the script in action. One uses placeholder images and the other real images from WikiPedia.

I normally wouldn’t advocate for a blocking script, but this seems lightweight and the tradeoff is worth evaluating. Make sure to measure the performance impact on your site.

During the halcyon days of the RICG, not a month went by without someone proposing a way to solve responsive images using JavaScript. We repeatedly and patiently explained why JavaScript couldn’t be used because images are downloaded before JavaScript is processed.

Lazy-loading images outside the initial viewport changes this equation. We now have a small window of opportunity to modify lazy-loaded images so we can add the equivalent of Client Hints to their URLs when their size is known.

Responsive images syntax will always have a role to play. Not every web page will have a image resizing server and images visible in the initial viewport will still face the race condition that gave the RICG fits. Any images in the initial viewport shouldn’t be lazy-loaded.

Fortunately, most images on the web live below the fold and thus can be lazy-loaded. If they can be lazy-loaded, then we no longer face the intractable race condition that forced us to place presentation information in the sizes attribute where it doesn’t belong.

Moving to simpler responsive images will require some work. According to CanIUse.com, 78% of users have access to Client Hints. That’s a great start, but ideally we’d get other browsers on board. Firefox and Safari expressed privacy concerns with client hints which is why they didn’t implement them. I’m hopeful that some combination a mandatory permissions policy (e.g., primary domain and subdomains only would be ideal) and a more limited set of hints (image width and dpr only?), that we might be able to get other browser makers on board.

But even without these changes, it is possible now to remove srcset and simplify sizes using a combination of lazy-loading, Client Hints, and the techniques that Eric Portis has pioneered.

Mat concluded his article by saying:

I won’t miss all those hand-hewn sizes attributes; I never had any love for them to begin with. I will never experience a shred of nostalgia for a thing that I helped make real and inexorably bound to my name. A syntax was never the goal; the goal was always a mechanism. At the time, the web platform lacked a way for browsers to make smarter decisions about what image asset to request and when, and no amount of clever scripting or markup trickery would ever result in an asset request as fast or efficient as one the browser itself could make.

None of us working in the RICG were ever enamored with our solutions. We wanted something simpler, but srcset and sizes was the best we could do with the constraints we were given. Now that lazy-loading has removed one of the constraints, I believe we can do better.

Special thanks to Eric Portis, Mat Marquis, and Scott Vandehey for their feedback and to Tyler Sticka for the grim reaper illustration.