Sensor Size and the Total Quantity of Light

IT IS SOMETIMES SURPRISING that even inexpensive cameras can take good quality images in the bright mid-day sun. I've seen many photos from cell-phones and from ridiculously cheap point-and-shoot cameras that have more than adequate image quality. Maybe these images aren't particularly optically sharp, but even low-end cameras can produce images in full sunlight that have a low amount of digital noise. They are “good enough.”

But one of the overarching rules of thumb in photography is that the larger the sensor size (or film size), generally speaking, the better the image quality of the final photograph. A bigger sensor makes it easier to have a lower noise image, a bigger sensor makes it easier to make matching quality optics, a bigger sensor makes it easier (up to a point) to make an ergonomic camera, and so forth. Now, I'm not saying that quality photos can’t be taken with a tiny image sensor, rather, it is easier to take an image with higher technical image quality by using a larger sensor. See the article One Easy Rule for Quality Images for more details.



A comparison of camera sensor sizes. [Source and attribution]

As noted, even low-end digital cameras can produce good images in broad daylight. The problem is that their image quality tends to sharply decline as the light gets dimmer. These cameras, taking photos under dim incandescent lighting, produce images that are a noisy mess, with terrible color rendition and digital grain ruining the sharpness of the image. Now, perhaps a tripod could help, but certainly these kinds of cameras are very disappointing for hand-held images.

Our eyesight doesn’t work as we might naïvely think. One scene, which to our eyes appears to be slightly dimmer than another, might in fact have half of the total amount of light falling on it. Likewise, a scene that appears to be only somewhat brighter than another might in reality be twice as bright.  In particular, where I live, in the mid-lattitudes of the northern hemisphere, we get to enjoy long periods of dusk in mid-summer; the fading daylight seems to last for hours, until we finally notice that it is very dark out. Our eyes valiantly attempt to see in the dimming light, until the laws of physics and biology finally conspire against our vision, and we are plunged into darkness. Our eyes attempt to flatten out the huge range of brightness that we experience.

A hazy day may be objectively half as bright as a sunny day, although it certainly seems to be only slightly dimmer. A cloudy day may be one fourth as bright, while an overcast day may be one eighth as bright. At sunset, it may be one sixteenth as bright as a bright sunny day, and a bright day may be thirty two times as bright as what we find at dusk. On ground covered with snow or white sand, a scene may be twice as bright as what we are accustomed to, and there is a real risk of contracting snow blindness due to the excessive amount of light.

Cameras, like eyes, are designed to work over a large range of brightness. Camera lenses have adjustable apertures to vary the amount of light hitting the sensor, and the shutter speed can be varied over a large ranges of values. The sensors also have varying amounts of sensitivity to light. But a sensor with twice the surface area of another collects twice the total amount of light, and we could assume (all things otherwise being equal) that it can operate similarly in light that is half as bright.

Now I've taken decent photos in dark places with a cheap point-and-shoot camera, but that was only when the camera was sitting on a tripod and its shutter was open for a long time. I certainly could not hand-hold the camera and expect to get anything else except digital noise. However, I can and do often take fairly decent hand-held shots at dusk with my Nikon DSLR. The major difference between these two cameras is simply the size of the sensor: the Nikon lets in a far larger total amount of light.



Same scene taken with a newer cell phone camera on top, and an older DSLR camera on the bottom.

We know that cheap point-and-shoot cameras, selling for less than US$50, and having tiny sensors, can take good images in broad daylight with little digital noise. Let us take this quality as our baseline, and determine what size of a sensor we need if we want to take images of similar quality and with similar camera settings under dimmer lighting. This table shows standard digital sensor sizes, along with the lighting conditions that would be equivalent to typical cell phone cameras in bright daylight:

Sensor size	Use	Sensor area in square millimeters	Lighting condition
1/4”	Cell phones and toy digital cameras.	7.68	Bright daylight
1/3.2”	Premium cell phone cameras.	15.5	Hazy sunlight
1/2.3”	Compact digital cameras.	28	Cloudy bright
1/1.7”	Premium compact cameras.	43	Light overcast
2/3”	Some bridge cameras.	58	Heavy overcast
CX or 1”	Nikon 1 series.	116	Sunset
Micro 4/3rds	Olympus and Panasonic mirrorless interchangeable lens cameras.	225	Dusk
APS-C	Most Nikon, Pentax, and Sony DSLRs; lower-end Canon sensors are slightly smaller at 329 square mm. Also found in some premium rangefinder cameras.	370	Indoor sports, stage shows
35mm, “Full frame”	High-end cameras from Nikon, Pentax, Sony, Canon, and Leica.	864	Bright street lighting at night

A camera sensor that has twice the surface area ought to produce an image with a similar amount of digital noise when the lighting is half as bright, all things else being equal. Certainly there are more factors involved, but sensor size is one of the most significant when it comes to image noise.

Photojournalists tend to use the cameras near the bottom of the list, especially if they need to capture a scene in dim lighting without the use of a flash. Note that the Micro 4/3rds cameras are fairly close in sensor size to the APS-C sized cameras, and their discrete size and noiseless operation make them viable for some work under dim lighting. Manufacturers have recently been putting the larger APS-C and 35mm sensors into compact cameras, which many photographers find highly desirable.

For more information, along with some of the data I used to make the table above, see these Wikipedia articles:

• Image sensor format
• Exposure value

At the Limit of Perception

MANY PHOTOGRAPHERS AIM FOR exceptionally clean images, low in noise, and high in dynamic range. However, extreme sensor sensitivity is rarely needed for most photographs, especially if the photographer sticks to the basic rules of photography, which include the practice of using good lighting.  A good, bright primary source of light, along with perhaps fill-in lights or reflectors, are typically needed to get good photographs.

But consider this photograph of a canoe, taken about 45 minutes after sunset, on a moonless, starless night, illumined by the waning skylight, distant fireworks and lightning, and a lone incandescent lamp a hundred or more yards away:



This was an interesting scene to my eyes, but there isn’t much to see in my image — just a very faint outline of an object.  You might have better luck seeing something if you click the photo twice to see it in Flickr with a dark gray background.

I took this with my camera mounted on a tripod, but because I could hardly focus at all, I set the aperture to f/8 for greater depth of field, and I didn’t use a long exposure time because I didn’t want to spend 2 hours getting my photo — one hour, perhaps, for the exposure, and one hour for dark frame subtraction. Sometimes it is inconvenient or even impossible to get a good exposure, so you have to make do with what you can get. I wanted to see how good of an image I could get at the limit of the camera’s performance.

Three Opportunities to Underexpose

SPRING IS FINALLY here in Saint Louis, and we have spring's floral finery waiting for the camera.

When you photograph brightly colored flowers, be aware that you risk overexposure:



Overexposing bright colors is easy with digital photography — and also when using film for slides. This red color really isn't all that bright compared to pure white, for it is only a bit more luminous than a middle gray. But digital cameras can only record so much red before they become overwhelmed. Blue skies are often overexposed, as we see in the article Three Opportunities for Overexposure.

As we see here, even a slight overexposure can ruin an image; adding contrast or saturation (common settings in a camera that usually make images look better) will make matters worse.  Just by looking at this photo you should be able to tell that it is overexposed: we have a large, nearly featureless areas of bright primary and secondary colors, namely the red tulip as well as section of the yellow flower on the right.

The color of each pixel in this image is determined by three numbers, each of which range in value from 0 to 255, with the numbers determining the amount of red, green, and blue light at each pixel. For an overview of this RGB color system, see the article Color Spaces, Part 1: RGB. Variation of the RGB color numbers across the image gives us detail: but here, much of the red tulip has its red value pegged at 255. So the flower looks flat and largely void of detail.

Overexposure in and of itself is not necessarily bad, and it is often unavoidable, but you likely will want to avoid overexposing even a single color channel on your main subject, like this tulip. If your digital camera offers three separate color histograms, you may want to check it to avoid this problem. Here are the overexposed portions, shown masked in black:



I took another photo nearby, using better exposure:



I took these photos the other day at the Missouri Botanical Garden in Saint Louis. You can see more of my photos taken that day here.

The red channel just barely kisses 255 in the brightest red areas, and so this flower, unlike the one above, is not overexposed. But this image is still disappointing. While we do see detail and texture in the flower, it still looks somewhat flat, especially compared to the adjoining tulips. The yellow parts of those tulips look more interesting and textured.  What is going on here?

Here are the three color channels of this image:



Please note that nearly all of the red tulip is black in the green and blue channels. As we would expect, there is good detail in the red channel. Take a look at the yellow parts of the tulip on the left; it shows good detail in all three channels, and so it has nice texture in the final color image.

We see texture when there is variation of the RGB color channels across an area. If there is little variation around a point, we see little texture at that point. Variation in color shows texture, but we have little to no variation in color with the red tulip. Variation in luminosity or brightness is typically more important when you want to show texture. See my article Luminance is More Important than Color.

A rough model of luminosity is:

Luminosity = 30% red + 59% green + 11% blue

If you want to show lots of texture or detail, then this calculated luminosity has to strongly vary over an area. With our tulip the majority of the green and blue color values are zero — seen as black in the above illustration — and so these color channels do nothing to give us texture. The texture in the red channel is attenuated by more than two thirds, and so the image looks somewhat flat.

Underexposure then is not one thing, but rather is three things, for we need to be concerned with the exposure of all three color channels. Typically we do not expect to see detail in black or white areas of a photograph, but certainly we would like to see texture in mid tones. But as we see here, we cannot count on that. Because two of our channels are underexposed, we lost texture.

---

I might add that the color of the tulip is wrong. It was not that particular shade of red, rather it was more intense, closer to scarlet, closer to the far edge of human color perception.

Just by looking at the RGB color numbers, we see that G and B are zero, and so all color is determined by R alone. This means that the color of the tulip is fixed to the color of the sRGB red primary. The sRGB standard is used by most digital cameras and most computer monitors.

The sRGB color standard can display only about 35% of all possible colors, as illustrated here:



This image originally appeared on Wikipedia. Source and attribution: http://en.wikipedia.org/wiki/File:Cie_Chart_with_sRGB_gamut_by_spigget.png

The horseshoe shape shows the limit of human color vision. The sRGB color gamut is represented by the triangle in this diagram. Our tulip color is somewhat near the lower righthand corner of this chart. Of course, this chart itself is not accurate in color, because it uses the sRGB standard, and so the outlying colors, outside of the triangle, cannot be displayed.

In photography, whenever you get one or two RGB colors at 255, while the other color is zero, then you know that you are out of gamut. We even see this in our first, overexposed photo. Red is largely fixed at 255, while blue and green are zero.

One solution is to capture the image using a color standard that is larger than sRGB. We won't be able to see these extra colors here, because lots of web browsers and computer monitors are limited to the sRGB standard. We can use a wider color gamut when printing, especially if use a higher-end printer than uses more than three color inks.

As it happens, I took this image using Camera RAW, and my Nikon can capture a very broad range of colors. During post-processing on my computer, I can convert the RAW file to sRGB, or I can convert it to the ProPhoto colorspace, which can represent more colors:



Image originally appeared on Wikipedia. Source and attribution: http://en.wikipedia.org/wiki/File:CIExy1931_ProPhoto.png

Notice the much larger triangle here, and also note that the red primary goes all the way to the limit of human vision; undoubtably our tulip can be accurately captured.

Here I edited the photo after converting the RAW image to the ProPhoto colorspace.



The gray here approximately indicates the areas which cannot be represented by sRGB. If we look at all three color channels, we can see that there is good texture in the tulip:



While we cannot show the ProPhoto image on the computer screen in all its glory, it may be printable, depending on the kind of printer we use. The final image ought to have better texture than what we had before.

We could process this image for computer display if we are willing to desaturate the red in the ProPhoto image. This would give us better texture at the expense of color.



Here I used the good green and blue channels from the ProPhoto color gamut to add texture back into the tulip. The flower is no longer intensely red, but I think it is otherwise a slight improvement.

White Balance, Part 1

PLEASE CONSIDER THIS photo of my living room:


(Or rather, this is a photo of the lounge at the Ritz-Carlton in Clayton, Missouri.)

Like very many interior photos — taken without a flash — this has a yellowish color cast. Undoubtably this yellow cast is due to the color of the lighting. Obviously incandescent light has a slightly yellower color of light than daylight. So there is no surprise that incandescent photos appear yellow.

Perhaps you think that the white balance feature on your digital camera is simply a minor adjustment. Perhaps it corrects for the slightly yellower color of incandescent lighting. Certainly I thought so.

Here is a white-balanced version of the photo above. I adjusted the photo so that a white object under this lighting was measurably neutral: that is, the red, green, and blue color values of the white object were equal after adjustment.



It looks a bit better: the yellow color cast has been removed. A small difference, but I think it improves the photo a bit, for it brings out the variety of colors better, and it is closer to what I remember seeing. I recommend always using a neutral white balance unless you have a specific reason not to do so.

Now consider this photo:



This is very yellow. But this so happens to be the scene assuming a daylight white balance. Were I to take a photo of a white object under bright daylight with this balance, it would look white. Incandescent lighting is actually far more yellow/orange in color than is daylight.

You may not be aware that your eye does an automatic white balance: when you look at a scene, your eyes attempt to subtract out the color of the lighting, and your eyes do a pretty good job of this, up to a point. Cameras work the same way; they also attempt to automatically subtract out the color of the light.

Keep in mind that perceptibly slight changes in the color of light may translate to a radically different objectively measurable difference in color.

Here is a photo taken outdoors, with a Daylight white balance:



And the same photo, with the color balance set to incandescent lighting; it is the same white balance I used in the nicely-corrected interior photo #2 above:



We see that the camera's white balance corrects for extreme differences in the color of light, not merely minor differences. How the eye corrects for white balance is a mystery, and the technology behind a camera's automatic white balance feature is ultimately imperfect, which is why I do a manual white balance whenever I'm doing my best work.

White Balance and Noise

Digital cameras have an intrinsic, fixed white balance. The color data captured by the camera sensor is adjusted by the camera's computer according to the white balance setting. Here are our sample pictures again, showing what they look like when using my camera's native white balance:



The images are greenish because each green sensor is more sensitive under most lighting conditions than the others.  For more details, see the article What Does the Camera Really See? For an overview of the color system used by digital cameras, read Color Spaces, Part 1: RGB. Digital cameras generally are rather insensitive to blue light.

To correct for the green snow scene above, the camera must amplify both the red and blue color channels to match the green, and whenever you amplify a signal, you also amplify noise.

For the interior scene, the camera must amplify the red channel a little (since incandescent lighting has plenty of red light), while it has to amplify the blue channel a lot, creating plenty of noise.

Here is an extreme example of this sort of noise amplification:



I took this a number of years ago with an inexpensive point-and-shoot camera. While the red and green channels have a lot of noise, due to the camera being set to ISO 1600, the blue channel at the bottom is particularly bad. The blue channel, due to incandescent white balance, was greatly amplified relative to the other channels, and so it shows plenty of noise.

Generally speaking, if you want low-noise interior photos, you are asking a lot of your camera, and you will likely spend a premium to get this.  Read this for details: One Easy Rule for Quality Images.

White Balance and Exposure

If you don't set your white balance correctly, you risk bad exposure if you shoot JPEG. Please note that I define good exposure by taking all three color channels into consideration; see the article Three Opportunities for Overexposure for details. If even one of your three channels is significantly overexposed, you will get shifts in highlight color.  In an extreme case, here is the daylight photo set with an incandescent white balance:



This is a selection from Nikon's View NX2 software, showing the three color histograms at this white balance. Each graph has the dark pixels to the left, and the bright pixels on the right: we see here that the red channel is underexposed, white the blue channel is overexposed.  Due to poor white balance, we irrevocably lost both highlight and shadow detail.

The histograms seen here are similar to the three color histograms found on many digital cameras. Like many photographers, I usually check the color histograms to make sure that my photos are exposed properly. In order to use as much of the camera's dynamic range as possible, I try to expose the images as much as I can without overexposing any one of the three histograms.  However, were I to use this process with a poor white balance, I would inevitably get a severely underexposed image. In the image above, reducing the exposure to preserve the blue channel highlights would lead to a severely underexposed red channel:



Now you might just want to have this ‘look’ in your photo, and that's fine. But if you instead plan on ‘fixing in Photoshop’, forget it.

Please  note that you will see the same problem if you shoot incandescent lighting with bluer white balances — Daylight, Cloudy, and Shade — except your red channel will likely be greatly overexposed while your blue channel will be underexposed.

As a general rule, you will get the best exposure if you use a neutral white balance. You can expose the image longer with less noise and less chance of clipping highlights if you set your white balance precisely.

But please remember that cameras have fixed intrinsic white balance, as seen above. You get a greater risk of overexposing or underexposing color channels when you shoot JPEG images, because the camera throws away lots of its original sensor data when performing a white balance — and you risk throwing away good useful data if you set your white balance wrong. For this reason, I shoot RAW images (which retain all of the original sensor data), because I can adjust the white balance on my computer after the fact. The risk of bad exposure is lessened — although not eliminated — when you shoot RAW and so you still ought to be careful at the time of shooting.

The trade-off is that RAW files need to be processed on your computer to produce an image usable for either printing or displaying on the web. I find the trade-off acceptable, although I know that many photographers do not.

Some authorities state that since a camera has a fixed intrinsic white balance, then the camera exposure histograms ought to show the RAW color channels. I think this is an excellent idea. Some photographers attempt to do this by forcing the camera to use a white balance that does no color adjustment at all: their histograms in this case are correct. The UniWB method of using the camera's own intrinsic white balance was developed by Iliah Borg and others, and the method is described here. It is not for the faint-hearted, since all your photos will turn out green, and you will have to correct for white balance on your computer. You can however get better exposure. At the very least, this is a good learning tool, if not really practical.

Note to camera manufacturers: please show the RAW histograms when shooting RAW! Also, give the ability to zoom in to the brightest pixels on the histograms, for overexposure in digital photography is worse than underexposure. When you show blinking pixels, be sure that they blink when even one of the channels is overexposed.

When to White Balance and When Not to White Balance

As I mentioned before, I recommend always setting your white balance precisely unless you have a specific need to do otherwise.

Have a look at A Digital Color Wheel. Colors opposite from each other are called opponent colors: a color balance biased towards one edge of the wheel will aways be at the expense of the opposite edge. If your image is too blue you will not get enough yellow, and too much green will mean too little magenta. If the white balance is in the center you will trade off quantity of color for quality of color; a well-white-balanced image will look richer in color content, and you have less risk of having your colors go out of gamut. You can get better results in saturating the colors if your white balance is precisely in the middle.

Sometimes you may want to capture a scene as you remember seeing it; but don't forget that your eye already does strong white balancing. So if you want to capture the warm glow of candlelight, set your white balance to somewhat warmer than neutral. To capture the cold mood of a snowy day, set your white balance to a somewhat cooler balance than neutral. Although your eye does do white balancing, this mechanism doesn't work well under dim lighting, although I am uncertain as to what the actual relationship might be. This is worth further research.

Here is a photo where I did not want the camera to subtract out the color of the light:



A fantastical beast, at the City Museum in Saint Louis. Click the image twice to view it on black.

Sometimes you may want an image with a strong overall color tone, but you may get better results if you first convert the image to black and white, and then add a tone afterwards, not relying on white balance.

Mixed Lighting

Your eyes not only do an automatic white balance, but they adjust this white balance variably across the scene while you are looking at it. When you take a photo of a scene, and reduce that panorama down to a tiny, low-contrast image displayed on a page or on a screen, this automatic white balance hardly operates, which is why you have to get it right in the camera. A severe problem occurs when you have mixed lighting of multiple colors, for example, when you shoot an interior, illuminated with incandescent lighting, while also having windows to the outside appearing within your photos.  Invariably your windows will look fine with the interior too yellow, or the interior looks fine while the scene outdoors is very blue. The photograph just doesn't look as you see it in real life.

With architectural interiors, I will use daylight white balance for the windows and incandescent white balance for the interior, and then composite these versions of the image. The results are good, even though this is a tedious process. Big-budget cinematographers will put large yellow-colored gels over the outside of the windows so that the color of the transmitted daylight matches the lighting used in the interior.

Far more problematic is when fluorescent lighting is used in an interior. Not only do these lights have an odd color — typically they are simultaneously more yellow and more green than daylight — but fluorescent colors are not constant across brands of lamps. They do not provide a continuous full spectrum of light, and even more problematic, they change color as they quickly flicker 50 or 60 times per second. Almost invariably, if you attempt to white balance fluorescent lighting, you will get strong shades of the opponent colors green and magenta throughout your image. These green/magenta color casts will be considerably increased if you also have daylight or incandescent lights in the scene. If at all possible, I will turn the fluorescent lights off.

Also see the article: White Balance, Part 2: The Gray World Assumption and the Retinex Theory.

Over and Under Exposure

GETTING EXPOSURE RIGHT is one of the challenges — and annoyances — of photography. I had long experience with black and white film photography and so I thought I had a pretty good understanding of exposure and how to get a decently-exposed image. When I got into digital photography back in 2001, I was quite disappointed with the results — the automatic exposure was often very wrong among other problems — and I had the bad opinion that it was the camera's job to set itself properly. You can read more of this on my old posting A Camera Diary.

Besides thinking that good photography merely involves choosing the ‘best’ camera, I was quite naïve about the properties of color digital images, and how they differ from black and white film. Exposure is far more critical to color photography relative to black and white.

Please consider the following series of images, taken at ISO 200, f/8, with each exposure time varying from 1/8^th for the darkest, to 8 seconds for the brightest. This Beaux-Arts building was built in 1900 for the Saint Louis Club, later became the headquarters for the Woolworth's company, and now houses the Saint Louis University Museum of Art.



Which image is exposed the best? Certainly exposure is something of a matter of taste, and your particular monitor settings may make one look better than another, and you might change your opinion if you used a different computer or if you printed these. However, too much exposure will give you all white, and too little exposure will give you black, and then you no longer have an image of a building. Objectively speaking, you have to expose within a specific range, which will vary depending on subject matter, your camera, and your post-processing.

If I had to choose between these four images, I'd select either the upper right hand image, or the lower left hand one; although I think that an intermediate exposure between these two would have been better. I took this in the morning, and perhaps I ought to have waited a few minutes for the sky to get brighter, which would have given a better balance of light over the entire image.

OK, I might say that I'll choose whatever image looks best to me; of course you do have to do that. Just because a machine says that one image is better than another doesn't mean that we have to follow that advice, because photographs are intended for humans, not machines. Just because the camera says that the photograph is correctly exposed doesn't mean that it will look best to us. But limiting ourselves to just gut instinct can't be right: “The unexamined life is not worth living for a human being” wrote Plato in the Apology. Instead, we ought to ask some questions. Why does one image look better than another? How can we reliably and predictably make good images?

Just because an image appears to be a bit dark does not mean that it is bad — there is a lot of detail that can be pulled up from the shadows. Generally, overexposure is more of a problem with digital images than underexposure, and so the standard advice is to expose for the highlights and process for the shadows. By the way, this is opposite to the advice used for shooting film negatives, where you generally have to expose to get good shadow detail.

Let's pull up some shadow detail from the upper right hand image:



I think this is an adequate image. Lots of detail is normally lost in the shadows, and it is easy to make this detail visible. This is just a rough brightening, and there are lots of techniques to show good detail in shadows. Were I doing a better job, I'd add more local contrast in the shadows; these shadows look a bit flat.

My intention when taking these photos was to produce a series of images which I would later blend together to make decent single image with lots of highlight and shadow detail with little noise and good color rendition. Before I submit the images to my exposure-blending software, I create hard exposure masks which cut off those parts of the images which are over- and under-exposed; the end result is a nicer looking image with low noise and good color tone. Without these masks, the software produces unpleasant color shifts in the highlights and excessive noise in the shadows. Masking also reduces haloing artifacts generated by my software.

Overexposure

It would be helpful if we define our terms. A pixel in an image is overexposed if any one of the three color channels is at its maximum value; or for eight bit images, if any channel is equal to 255. Now 255 might just happen to be the correct value of a pixel, but that is unlikely, since everything brighter will be equal to 255 also. If any one of the three color channels clips due to overexposure, then you will get color shifts in the final image.

This color shift is rather prominent on the brightest of my sample images.  Here is a close up view; note how the color of the building near the light goes from a nice orange color to yellow, and then white; while the blue sign goes to cyan then white:



In the upper right hand corner, I put in black wherever any one of the three color channels of a pixel is equal to 255. In the lower left hand corner is a mask which shows wherever the RGB luminosity goes to 255; notice how it masks out a smaller area than the full overexposure mask.

RGB luminosity is roughly defined as:

30% Red + 59% Green + 11% Blue

This approximates the sensitivity our eyes have to each primary color.  But this value will often be less than 255, even if one of the channels is overexposed. Some camera histograms will show this value instead of three individual color histograms, which can be less than helpful. Also, some exposure blending and tone-mapping methods use this value as an estimate of brightness, and the final images often show these color shifts.

In the lower right hand corner I superimposed the full overexposure mask on the image. Note that it covers up nearly all of the areas that show an obvious color shift, but not all.  There appears to be some bad color bleeding out from around the edges of the mask.

This image, even though it comes from a Camera RAW file, still has been highly processed by the RAW converter, plus I did some lens distortion correction as well as straightening of the image. My camera uses a matrix of light sensors, and each one is sensitive to only one color.  When the RAW converter makes the final image, it estimates the missing colors at each pixel by examining neighboring pixels. Likewise, when correcting for lens distortion and camera tilt, Photoshop estimates the correct pixel values by also examining neighboring values.  So we are always doing some averaging; but consider this example equation:

Estimated value = (250+240+235+garbage)/4 = garbage

So the effects of overexposure anywhere in an image will spread a bit to neighboring pixels. In practice, when I make a mask like this, I will mask out everything that has a value over 250 or so, which seems to get rid of most if not all of these nominally good, but actually bad pixels, without losing too much highlight detail. Someday I'd like to see software that offers a mask channel associated with images, which will show all pixels which are overexposed, or which are indirectly unreliable due to processing.

Here is the brightest image, with a black overexposure mask superimposed on it:



I extended the mask a little bit, so as to eliminate the color shift which extends a few pixels beyond the measurably overexposed parts. Note that the sky is masked out, because it is completely overexposed in the blue channel. About 1/3^rd of the image is overexposed. There is good detail throughout the rest of the image. Note that there is a slight blue halo around the roofline; this is because this image is not particularly sharp, and so there is a bit of blur along edges which does not get masked out.

Photography has many trade-offs, requiring us to make choices; we neither want to overexpose, nor do we want to underexpose. Ultimately, some detail doesn't matter, and specular highlights and light sources are usually considered unimportant — it is OK to overexpose them most of the time, as we see even in our darkest example photo above. The lights aren't the obvious subject of the photo.

Now if you have large areas of color in your photo, you probably don't want to overexpose them, even if they aren't the subject. Digital cameras will often overexpose blue skies, which I think is objectionable most of the time, even if it is not the subject of the photograph.  This kind of overexposure is particularly objectionable when the sky goes from blue to cyan to white in a single image: that just doesn't look natural. See my article Three Opportunities for Overexposure. Alternatively, it is often best to strongly overexpose a background, turning it a pure color or white, rather than having a muddled partial overexposure with obvious color shifts.

Another problem with overexposure, besides color shift, is that it removes texture from the image. Areas with even one channel overexposed will appear somewhat flat. Now there are techniques which you can use to rescue such overexposed images, by generating plausible detail from the remaining channels. This is difficult to do correctly, and is time-consuming.

Now technique ought to serve the subject matter; the subject does not serve the technique except perhaps when you are creating images for teaching. Just because the blue channel of the sky in an image is overexposed does not mean that you can't end up with a terrific photograph, if the subject is worthy.

Underexposure

Defining overexposure is easy, even if we have to be careful and realize that it isn't quite as simple as we would like. Defining underexposure is far more problematic.

I defined overexposure on any given pixel as the situation where any one channel is at its maximum value, generally equal to 255 with 8 bit images. OK, we can naïvely assume that underexposure is the situation where any color channel equals 0. For example:



Not much of a mask at the bottom. This is next to worthless, just a few black dots here and there, even though there is a lot of black in this image. There are several problems with our effort here, the most significant is that there is a tremendous amount of noise, relatively speaking, that is found in the darkest part of the image, often due to the quantum fluctuation of light. Light is detected in discrete quantities due to a mysterious property of matter and energy on a small scale, and therefore is quite non-uniform.  There are also several sources of noise in the camera itself, and these sources will add to the signal, moving it away from zero. Also consider the indirect problem mentioned with overexposure: image manipulation will ‘infect’ neighboring pixels, and since no pixel value can be less than zero, this averaging will only increase the value found at pixels which ought to be zero. Noise at low levels does not average out to zero, but instead will brighten dark pixels..

Some cameras, as well as RAW converters, will do plenty of image manipulation including noise reduction or black-point cutoff, making our low-value pixels even more unreliable.

Instead I use a working definition of underexposure which masks out those values near zero. Now, should I take into account all three color channels at one time, or each color channel separately? If I choose all three, then I might not mask out a particular poor, noisy channel if the other two are good.

But if I mask out each channel separately, then I might get the situation where a particular pixel is both overexposed and underexposed! I often see this with stained glass. For a particularly brilliant red piece of glass, I may have the Red channel at 255, while the Blue channel is at 0: this indicates that the color is particularly pure and outside of the color gamut of the camera or color space used in Photoshop.

There are several methods I use to mask out dark noise.  The simplest uses the image itself and the Threshold slider; this uses the RGB luminance function shown above. I examine the image while moving Threshold, and stop when a reasonable amount of noise is eliminated.  Using this process on our darkest sample image we see:



The road and the top of the building on the left shows the most noise. I adjusted the Threshold slider until much of that noise is eliminated, as you can see on this detail from the lower right hand corner of the image. You don't want to do too much of this.

I also use the same process, but doing each channel independently. This makes an exceptionally clean final image, but only if we don't have the simultaneous over and under exposure problem mentioned above.



This looks like a pretty good mask. It masks out the most underexposed parts of the image while not showing too much residual noise.

There is another technique which illustrates the problem of dark noise quite dramatically. What I do is take each channel separately, and brighten it with Curves until it no longer appears to be a photograph, but rather a line and charcoal drawing. Instead of a nice apparently continuous series of tones, we get discrete steps. This shows that we don't have enough spacing between brightness levels to produce a good image. Usually this effect is most prominent in the Blue channel because normal in-camera processing greatly amplifies its shadows.



You may want to click on the image to see the full size version. Some image enhancement software will work quite hard to bring out detail in shadows, but this is certainly detail we don't want to emphasize — unless of course we are going for a cartoony look. As it happens in this image, the red and green channels don't quite go to zero much and so we don't get much of the line drawing effect, but the blue channel does; so our mask eliminates considerable amounts of noise.

Creating masks like these can show you how much of your image consists of high quality pixels. I use these also for creating exposure blends. Here are those four images blended together, masking out the big color shifts and dark noise:



It looks pretty good, and there are hardly any color shifts except for the areas which were overexposed on the darkest base image. Notably, the color on the building and the blue sign have uniform color as needed, and we have excellent detail in the shadows. The major artifact here is the sidewalk light to the right of the stairs:  it turned off between the second and third photos and so we get a strange rendering of it here. There is also some roughness along the roofline.  You can click on the image to see the full resolution version.

Conclusion

The phenomenon of color shift — when even one channel is overexposed — severely limits quality color photography.  Of course, solid studio lighting, or supplemental lighting with fill-in reflectors are frequently used by quality photographers. Or, you can blend multiple exposures, but then the problem is finding the right algorithm or software to do this.

On the contrary, this color problem implies that quality black and white photographs ought to be easier to produce. We can introduce severe changes of contrast without worrying about color shifts.

Three Opportunities for Overexposure

EVERY CAMERA — as far as I know — ultimately uses precisely one measure when exposing an image: the Exposure Value. This E.V. is made up from various combinations of:

• lens aperture size,
• the time the shutter is open, and
• the light sensitivity of the sensor or film; 

and although the camera's estimation of E.V. may be adjusted by the photographer, or modified by the evaluative algorithms in the camera's embedded computer — matrix metering — camera exposure is eventually reduced down to a single number.

This one number, if it is not in a narrow range suitable for the subject, will produce a completely unacceptable image. For example, here are two photographs of my cat Shilo:


Portrait of a Cat I (Shilo) [EV=-4.5]


Portrait of a Cat II (Shilo) [EV=21.3]

While these might be interesting exercises in conceptual art, they certainly fail as photographs of my cat. For this subject the valid E.V. would be somewhere around 5. Click here to see how exposure value is calculated.

A suitable exposure value for a particular subject must be within a particular range, and although this range may conceivably be quite large, depending on the intent of the photographer, it is certainly not infinite, as the images above show.

Overexposure gives white, underexposure gives black, and shades of gray and color comes from correct exposure, which is somewhere in-between.  But we have a problem, and that is color. A digital camera does not capture a single image, but rather captures three: one image each for red, green, and blue colored light, which combine together to make full color. Each of these channels has its own range of desirable exposure values, and these ranges will likely not be equal. The camera gives us one E.V., while we really need one for each channel, and so we have three opportunities for overexposure, and not one.

These three color channels can only reduce the acceptable range of exposure values, as we can see here in this particular example:


Practically speaking, this phenomena is the reason why color photography is often benefitted by flat lighting, and also why monochrome images can capture more tonal depth with beautifully deep, detailed shadows.

When a digital camera captures a JPEG format image, every resultant image pixel is made up of three numbers; one for each of the colors of red, green, and blue, and these numbers range from 0 to 255. Disregarding digital noise, if a certain minimum amount of light fails to reach a particular pixel, the resultant value is 0, and if more than a certain maximum amount of light hits it, we get 255. As we increase exposure on a colorful object, one of the channels will eventually reach 255 before the others, and this overexposure will cause a color shift.

Here is an example of a sky blue color, simulating an increase in exposure:



The color noticeably shifts in hue, showing how the acceptable range of exposure values is greatly reduced if we want to capture accurate color.

For a real-world example, see these two photos which have slightly different exposures:



In the first exposure, the blue channel in the sky area is largely at 255, indicating overexposure; the color is obviously wrong since it is an unnatural cyan. The second picture has a good sky color because the blue channel falls below 255 and so retains its proper proportion with the other colors. Note that the green and brown hues of both photos appear to be identical despite different luminance. Generally, if large areas of any one channel is at 255, there is a likelihood of color shifts and loss of detail. Because of this, overexposure ought to be avoided by examining the three color histograms found on many cameras. Reducing exposure like this may make an image relatively dark, but there is typically a lot of detail in the shadows which can be made visible in post-processing.

A photographer needs to determine which areas of a scene contain significant detail, and expose and later post-process the image accordingly. In our example photos above, the blue sky is significant detail and ought to be exposed properly. Usually, out of focus backgrounds and specular highlights can be overexposed without problem, but these often require post processing to remove the strong colors that come from the color shifts described here.

Ultimately, good exposure may not be possible, for the range of detail that must be captured is beyond the range of the camera's sensor; in these cases, supplemental lighting or the High Dynamic Range technique are needed, to either reduce the range of the subject, or to capture the entire range of the subject.

By the way, here is Shilo, well-exposed: