Don’t use <picture> (most of the time)

Written by Jason Grigsby on

Browser support for the picture specification is landing and as Marcos Cáceres said, it is time to “go forth and <picture> all the things!

Except you shouldn’t. You shouldn’t <picture> all the things.

But you should start using picture now for responsive images. No reason to wait.

Confused? You’re not alone.

<picture> vs. picture

Standards are developed in non-linear fashion. Ideas evolve and merge. And often at the end of a lengthy process, you look back and wonder how you got here.

And in this case, where we ended up is with a specification called ‘picture’ that contains much more than the <picture> element. The picture specification includes srcset and sizes and you can use those attributes without using the <picture> element.

Knowing which use case you’re solving tells you what solution you need

One of the earliest pieces of work that the Responsive Images Community Group undertook was defining the use cases for responsive images.

You don’t need to know all of the use cases, but you do need to know the difference between the two most common use cases in order to know which part of the picture specification will solve your problems. The two common use cases are:

  • Resolution switching — In the resolution switching use case, we need to select a different source of the same image because we need a different resolution for any number of reasons. These reasons include the image being used at a different size based on the size of the screen, the pixel density of the screen, or to avoid downloading unnecessarily large images.1

  • Art direction — In the art direction use case, there is some reason why we need to modify the content of an image under certain conditions. Maybe we need to crop the image differently on small screens. Or perhaps we’re working with a hero image that contains text and simply providing a smaller version of the image won’t work because the text will be unreadable.

Basically, if you could resize an image without making any other changes, and have a new source that solves your responsive images needs, then you’re talking about the resolution switching use case. If you need to change anything other than resolution, you’re talking about art direction.2

For most responsive images, you won’t need the <picture> element

Unless you’re solving for art direction, you don’t need to use the <picture> element. In fact, you’re likely doing your users a disservice by using the <picture> element.

The picture specification supports syntax that can be used without the <picture> element. An example syntax, borrow from Yoav Weiss’ excellent article Native Responsive Images, might look like this:

<img src="cat_500px.jpg" srcset="cat_750px.jpg 1.5x, cat_1000px.jpg 2x" alt="lolcat" width="500px" />

Which will provide the browser with different options for display density. Or in a more complex example:

<img src="swing-400.jpg" sizes="(max-width: 30em) 100vw, (max-width: 50em) 50vw, calc(33vw - 100px)" srcset="swing-200.jpg 200w, swing-400.jpg 400w, swing-800.jpg 800w, swing-1600.jpg 1600w" alt="Kettlebell Swing" />

There is a lot to digest in those examples, and unfortunately, I don’t have the space to cover the syntax here. Instead, I recommend reading Yoav’s article or one of the additional resources listed below if you want to understand the details.3

When you use the srcset and sizes attributes on an <img> element, you are providing information that the browser can use to make an informed decision about what image is appropriate for the user based on a bunch of factors that you are unaware of.

As a web designer or developer, you have no way of knowing how much bandwidth the user currently has or if they’ve declared some sort of preference for the density of images they want. If we provide browsers with information via srcset and sizes then browsers can make smarter decisions about the appropriate image source.

In fact, I hope that browser makers compete on how they handle srcset and sizes by developing new user settings or smarter algorithms to help them pick the right images to download.

But none of that is possible when you use the <picture> element and its media attributes:

<picture> <source srcset="large.jpg" media="(min-width: 45em)" /></picture>
<picture> <source srcset="med.jpg" media="(min-width: 32em)" /></picture>
<picture> <img src="small.jpg" alt="The president giving an award." /></picture>

When you specify the media queries for sources, you are providing rules to the browser that it must follow. The browser has no discretion to make smart decisions about what to download based on user preference, network, etc.

You should use the power to dictate what image gets downloaded sparingly. In fact, you should only use it when you’re solving for art direction, not for resolution switching.

For the majority of the images on the web, <picture> is the wrong solution

Last year, Yoav tried to figure out what percentage of responsive images fell under the art direction use case. The answer: 25%.

Responsive design is still early so we may find that the percentage changes, but it is unlikely that we’ll ever reach a point where the number of art directed responsive images out-numbers the number of resolution switching ones.

Therefore, for most responsive images, the <picture> element is the wrong solution. You should be using <img> with srcset and sizes.

The future of the web depends on us getting this right

Getting this right matters a great deal to the future of the web. We’ve seen in the past what happens when web developers create a large installed base of suboptimal web pages. We end up with browsers adopting other browser’s css prefixes or media types such as mobile and TV ignored by all mobile phones and TVs.

If we create thousands of web pages that use the <picture> element for resolution switching, we doom ourselves to having to specify every single image needed instead of letting the computers—the browsers—do what they do best and automatically find the right image based on a multitude of variables.

Worse, we doom our users to a future where they are unable to take advantage of whatever browser advances come because we’ve taken the browser’s discretion away and dictated what image should be downloaded.

Perhaps we need to stop referring to the picture specification

A lot of this confusion comes from the fact that we have a specification that ended up with picture in the title even though the specification covers more than just picture.

For those who have been heavily involved in the development of a solution for responsive images, picture is a nice shorthand. That’s why you see that even it referred to as <picture> in Chrome Status and Modern IE Status.

Using picture as a shorthand it creates confusion when talking to people who are just now looking to implement responsive images. I asked the Responsive Images Community how they’ve been handling this confusion.

Bruce Lawson says:

I tend to refer to “picture and friends” or, generically, “responsive images”

And Odin Hørthe Omdal adds:

I always talk about responsive images, and I think I probably say the respimg specification.

So I’m going to attempt to do the same. I’m going to break myself of the habit of referring to the picture specification and instead refer to responsive images specification even if that isn’t technically the name of the specification. I think people will understand what I mean, and it will help ensure more people understand that it isn’t all about <picture>.

Simple guidelines for using the responsive images specification

To summarize:

  • The picture specification contains more than just the <picture> element. Think of it as the responsive images specification. * For most responsive images, you shouldn’t use the <picture> element. You should use srcset and/or sizes.

  • The way to know when to use the <picture> element is based on the use case you’re trying to solve.

  • If you’re solving for the art direction use case, use the <picture> element. Anything else, you should stick to srcset and sizes.
  • Getting this right early—before we have thousands of pages using <picture> incorrectly—is critical for the future of the web.

And with that, I will amend what Marcos wrote to say, “Go forth and make all your images responsive!”


  1. In the responsive images use case document, resolution switching is described with three different examples: Resolution-based selection, Viewport-based selection and Device-pixel-ratio-based selection.
    And yes, they are all different reasons why you want to select a different resolution source, but they all involve resolution switching which is why I’m lumping them together.
  2. You may find this article I wrote on A Framework for Discussing Responsive Images to be a good high-level overview of the use cases.
  3. Some additional articles to help you understand the full features of the picture specification: HTML5 Rocks: Built-in Browser Support for Responsive Images, Srcset and sizes and Responsive Images Done Right: A Guide To <picture> And srcset by Eric Portis.
Jason Grigsby

Jason Grigsby is one of the co-founders of Cloud Four, Mobile Portland and Responsive Field Day. He spends far too much time obsessing over mobile and the web. Follow him at @grigs.

Comments

I'm not sure I understand why the source element's media queries would be more specific (and less flexible) then the srcset markup - especially considering source elements use the srcset attribute in much the same way the img element does. Maybe you can elaborate a bit?

Replies

Thanks Ben. The difference is that when you have a list of sources, the browser MUST use the first source that has a rule that matches.

With srcset, the browser can pick what it thinks is best. It allows things like downloading one source and then downloading a higher resolution version when the person zooms. The browser can pick the best source instead of having it dictated to it.

I completely forgot about the strict ordering of the source elements. Looks like sizes/srcset is the way to go :)

I think the tag is but a solution looking for a problem. The real issue in my opinion is the lack of understanding web developers have in how to create web images in the first place. None of the articles I have seen on this subject say how much of a file size difference one would get from using this procedure. If the file savings is not significant then creating multiple images and adding the extra coding is not worth the trouble. If I may ask, what are your savings?

Great article, summarised it all really nicely in a really clear way. You've eased my own confusion on the topic after I ended up confused following the respimg email thread!

Super nice write up, Jason! I'm glad to see this being published now rather than later. "Getting this right early—before we have thousands of pages using incorrectly—is critical for the future of the web." You da man!

Totally agree with you, Jason!

We also need to make sure that our automated tools are defaulting to the 75% use case. For example, the pre-release Drupal 8 software has the older picture element implementation. But we are still working on adding the newer srcset+sizes implementation of the responsive images spec. https://www.drupal.org/node/2334387

The docs at http://scottjehl.github.io/picturefill/ are very good, but could probably use a once-over with your note in mind; the docs needs to spell out more clearly that there's more than one way to implement a responsive image on a website.

This is kinda mind-blowing, didn't know srcset and sizes were suggestions (instead of instructions) to UA on how to download image content.

25% for art direction seems low. I can't imagine it'll stay this way for long. The sample size quoted in the original article of just under 30 sites is way too small. Of those sites, many were early adopters with bigger budgets. If you were a smaller site, implementing an in-flux spec with a polyfill just wasn't worth the effort.

Now that things have settled down, I think we're going to start to see the tools get better as well to handle art direction. Especially as more of the world moves to mobile, why serve a badly cropped image to someone on a smaller screen if they're now more than 50% of your visitor base? The art direction use case is going to be a lot more valuable as time goes on if only because users are going to be expect more out of their experiences.

The other piece is the tools are terrible right now for multiple image sizes. Content management systems will start to offer better cropping solutions to help content authors. In WordPress, for example, you can specify multiple images which WP will generate but there's really no good way to preview what all those sizes would look like in the context of your theme so you could adjust the crop as necessary. Until that becomes a core function in a CMS, that's a big problem. Of course, getting the tools—and the spec—out there is one thing, client and developer education is another as to what you can actually do with those tools.

Great article, but many browsers incompatible with `picture` :(

Replies

Thanks for the great article Jason, but from the "usage relative" figures of caniuse.com it doesn't look like there's any hurry to support this, particularly on mobile browsers where the need is more critical because of the bandwidth and browser memory savings. A mobile-first stylesheet (or putting the mobile image in the src attribute) will have mobile browsers load the smaller image first anyway.

It looks like detecting srcset is possible, but the JS fallback will stick around for a long time.

We'll look into adding this to a later release of Sparkle.

Love the article Jason - thanks for sharing!

The one thing that still bothers me is having to maintain a library of x number of images for the multitude of responsive use cases. Imagine if I had a specialized museum site with say 1,000 images, I am asking myself if that could easily turn out to be 4,000 images or more? Between the native image, thumbnail images, and art directed images, the sea of options seems to be very large unless I am thinking about this all wrong which very well may be.

In a perfect world I would hope that I could have a single high-"ish" resolution image and have it scaled by the server and deliver what I need, including art direction crops/clipping, etc. Something like

and then the server would determine what resolution I need on the fly, perhaps using an progressive interlacing approach so there is some low res image there on page load and then it "fills in" the rest once the server has determined the metrics of the device.

Am I dreaming? Is this panacea even possible? All I know is that I would like the option of using a single image so my image library becomes easier to maintain.

Replies

Hi Jim,

Great to hear from you. FYI, your code didn't come through in the comment, but yes, unfortunately, you're dreaming.

First, you can of course get a service that handles cropping, resizing, etc. In fact, it is a core part of what I recommend here:
http://blog.cloudfour.com/8-guidelines-and-1-rule-for-responsive-images/

However, the server does not know:

* The size of the element in the page.
* The pixel density of the device.
* That the viewport has been resized.

In fact, the browser doesn't know the size of the element at the moment it starts downloading images:
http://blog.cloudfour.com/the-real-conflict-behind-picture-and-srcset/

And that lack of knowledge is why even if we had a unicorn image format where the browser only downloaded portions of an image file ala JPEG 2000, we would still need changes to markup to inform the browser when to stop downloading:
http://blog.cloudfour.com/what-a-holy-grail-image-format-would-mean-for-the-browsers-lookahead-pre-parser/

I believe that we'll build tools to automate images and the markup for us. When I started on the web, it was a big pain to simply set up a server. Now I can create an Amazon instance with the press of a button.

Images are difficult now, but they won't be forever.

-Jason

Great comment, Jason!

I remember building forms and pages by hand back in '96 before tools like WordPress came long and let us publish content on the fly. Even those little hit counters that were so popular were resource intensive. These days the average page size is measured in MB. Flash back 20 years that would have taken hours to downed. As they say, this too shall pass just like the screech of the old 14.4k modem.

This is possible now, however it is not the optimal solution because the server needs to do all that processing on the fly so the user gets a slower experience.
So if you're not saving the different image sizes to the server you get this processing overhead every time the page loads.
Not only is it slower for the user bit it takes server resources from other users, meaning you can serve fewer concurrent users.

Storing all the sizes means doesn't mean your users gave to upload multiple images, they can still be generated by the server, it's list better if they are generated once and then stored for future use.

This means you're trading off hard drive space for performance and concurrency. Seeing as hard drive space is so cheap this is a no brainer.

There is an in-between that us often used (in Drupal for example), which is to generate image derivatives the first time a user needs them.

This means that you aren't wasting space on images that are never needed, but that one user has to pay the price to generate it.

Alternatively, you could generate them all when the original image is uploaded, meaning end users always get best performance but you're probably wasting space on some images that will never be seen at a certain size.

This has been a priceless post, creating the confusion that I felt was going to set eventually. But being that the end game is performance, there will always be an area where we'll take a hit - essentially a lose lose.
There's a bit of shot in the dark element to this as the question is: do i have the correct image to serve to the browser. At some point, a resolution will appear and we won't have an image best suited for it.
Or if you go with a very early solution as per Chris Coyier - who was losing some patience with the process (a podcast where he discussed this) - which was to create this middle ground resolution (and I think he quoted something like 150 ppi) and use those all over. But the wrong resolution and size has been proven to be a battery/memory drain on the device, creating it's own negative effect on performance.
Added to that, some will gripe about the additional markup required to get this done properly - but I don't believe that minified and gzipped css will have enough adverse effect to even warrant a convo - but it still is a bigger file on paper.
As an aside, I was wondering why during Apple's WWDC there was no mention of and the devs went straight into why was implemented in Safari 8 (<a href="https://developer.apple.com/videos/wwdc/2014/";there's a 40+ min video called Designing Responsive Web Experiences). Were they on to something? This is the 1st thing that crossed my mind after reading this post. Could be coincidence, or... ;)
Anyhow, awesome post Jason!

The first rule of the picture specification is not to talk about the picture specification. Good.

Hi! I'm sorry, but... Why can't I use just the IMG element even for art direction? Semantics? Sometimes, on a CMS, for example, we can't know if the images uploaded have art direction or not.

Thanks!

Replies

The difference isn't semantics, it is whether or not the browser has discretion or not.

When the use case is art direction, the browser MUST download the specified image. To do anything other than that would cause problems because a different image might be inappropriate for the layout (e.g., the image switches from landscape to portrait on different size screens and if the browser downloads the wrong one, then the image will look terrible).

When the use case is simply resolution switching, giving the browser discretion to decide what image to download is much better. For example, it may be better for a browser to download a slightly smaller image and size it up slightly than it would be to download the next largest image size. The browser can make these judgments based on the device capabilities, screen size, density, bandwidth, etc.

And in this scenario, authors only care that the user gets a good experience. The browser is in the best position to evaluate which image will provide the best experience.

So that's the reason why. It doesn't have much to do with semantics. It is a difference between providing suggestions to the browser versus declarations to the browser. Whenever possible, we're better off providing suggestions and letting the browser make smart decisions.

Thank you so much! You helped me a lot!

Now, I gonna find a way to adapt this to my CMS of choice. Thanks!

This specification is long overdue and it was great to see a clear discussion of use cases so that everyone doesn't go everything and not think through what they are doing. Thanks for posting and sharing this with the community.

It's not just a bad practice. It's not supported!

http://caniuse.com/#search=picture
http://caniuse.com/#search=srcset

Replies

Right. "Browser support for the picture specification is landing". Present progressive tense.

It is currently available in Chrome, Opera and Firefox behind flags for testing purposes. Chrome and Opera have already said they will ship it in their next versions and have started publishing developer documentation. Those browsers will ship soon (days or short weeks) Firefox isn't far behind.

Yoav is working on a Webkit port. And IE is considering it. And of course we can use the PictureFill polyfill in the meantime until the browsers support it.

A few comments:

* Looks like the original blog got the code for fixed with images a little off. Width shouldn't have the "px" there.
* Can we still use the width attribute to make things load slightly faster for desktops? With that we'd still need to use the classic "max-width: 100%" I'd imagine.
* Honesty, what's the best way to test this? I'm trying Chrome's canary build (currently 40) and using the dev tool's emulators and I'm somehow seeing double downloads (or downloading the DPR=2 images on a DPR=1 environment) when I examine the network and just not getting this working right. Am I misunderstanding the canary build and it doesn't have this flipped on already like I thought?

Replies

* Looks like the original blog got the code for fixed with images a little off. Width shouldn’t have the “px” there.

What specifically are you looking at? Are you looking at px in the file names? I think that was just done to explain what the size of the images were. The file names could be whatever.

* Can we still use the width attribute to make things load slightly faster for desktops? With that we’d still need to use the classic “max-width: 100%” I’d imagine.

I think unless you declare width and height attributes, that you don't get any performance benefits. But I'm not certain.

No reason you can't use max-width:100%.

* Honesty, what’s the best way to test this? I’m trying Chrome’s canary build (currently 40) and using the dev tool’s emulators and I’m somehow seeing double downloads (or downloading the DPR=2 images on a DPR=1 environment) when I examine the network and just not getting this working right. Am I misunderstanding the canary build and it doesn’t have this flipped on already like I thought?

Hmm… It works for me. I'm testing this page:
http://scottjehl.github.io/picturefill/examples/demo-01.html

With JavaScript turned off (to make sure that picture isn't doing the work) and it is doing exactly what I would expect. What page are you testing?

In regards to the first point, I was looking at:

width="500px" alt="lolcat"

should be:

width="500" alt="lolcat"

As far as the last point is going, I am trying to update our CMS for our clients, so that they will all automatically get this feature, and I hadn't got it going on my local machine yet. I had some more work to do finish up automatically generating the resized images and the sizes attribute based on the original image. I was just trying to figure out what is the best way to test it natively (i.e. no picturefill).

Thanks nice article! picture or srcset - what bugs me are the breakpoint definitions in the view layer. They belong into the css and nowhere else. There is a elegant solution for that, I wrote a little blog post about this: http://www.andre-abt.com/2013/09/16/responsive-breakpoints-and-images/

Replies

Hi André,

Thanks for the link. I read your article. I like the consolidation of breakpoints. It is something we've written about before in the past as well (http://blog.cloudfour.com/behavioral-breakpoints/).

The tradeoff you're making with the solution you're looking at is that you're eliminating the benefits of the browser lookahead pre-parser. Any solution that relies on JavaScript is taking away the ability of the lookahead pre-parser.

Browser makers will tell you that the lookahead pre-parser is one of the biggest performance improvements made to browsers over the last decade. That's why a native responsive images solution was sought out that would allow the pre-parser to continue to do what it does best.

So yes, if you ignore the pre-parser, then you can get away from having information about image breakpoints in HTML. Ignoring the pre-parser will also mean rolling back all of the speed improvements that come from it.

-Jason

Great article on responsive image. It helps me better understand the difference bettween using the "picture element" and "srcset and sizes attributes on an ".
Thank you.

Very interesting article, but I see a few problems with this method - first, it seems the background-image property doesn't have any equivalent of this srcset method, sites heavily dependent on that could go with media-queries but not the "smart" srcset. Second, I see the 25% art-direction statistic was from 2013, I wonder if parallax-style full-page-images and growth of mobile use would make this an even higher percentage today?

Replies

CSS has image-set:
http://dev.w3.org/csswg/css-images-3/#image-set-notation

I don't know why mobile use would make any difference in art direction versus resolution switching. And parallax is a trend, but it isn't going to be enough to change the equation.

Think about it this way, most user-generated photography and news photography is resolution switching. Allow users to upload images ala Flickr? Resolution switching. Photo of a sporting event? Resolution switching. Photo of a random product on an ecommerce site? Resolution switching.

For big content sites, doing art direction on an image by image basis isn't sustainable. And those big content sites are going to account for the largest percentage of images on the web.

If anything, I suspect that the percentage of images that are resolution switching will go up from Yoav's 2013 study as responsive design spreads to more big sites.

Hey Jason,

What if i would want to include new image formats such as WebP and progressive Jpeg? As far as i know i can't do this properly with srcset and sizes.

In my tests these formats shaved off a lot of the file size of images used on the page.

What do you think is more beneficial is terms of performance. Using the newer and better image formats for browsers which support them or using the srcset/sizes and let the browser decide which image is best.

Wesley Smits

Replies

Yep, if you want to supply different types of images, you need to do so using the picture element.

However, you don't necessarily need to use the media attribute on those sources. You can use picture w/ sources that define srcset and type and skip media so that the browser can still pick the best asset in a resolution switching use case.


Let’s discuss your project! Email Us