White Balance, Part 1

PLEASE CONSIDER THIS photo of my living room:


(Or rather, this is a photo of the lounge at the Ritz-Carlton in Clayton, Missouri.)

Like very many interior photos — taken without a flash — this has a yellowish color cast. Undoubtably this yellow cast is due to the color of the lighting. Obviously incandescent light has a slightly yellower color of light than daylight. So there is no surprise that incandescent photos appear yellow.

Perhaps you think that the white balance feature on your digital camera is simply a minor adjustment. Perhaps it corrects for the slightly yellower color of incandescent lighting. Certainly I thought so.

Here is a white-balanced version of the photo above. I adjusted the photo so that a white object under this lighting was measurably neutral: that is, the red, green, and blue color values of the white object were equal after adjustment.



It looks a bit better: the yellow color cast has been removed. A small difference, but I think it improves the photo a bit, for it brings out the variety of colors better, and it is closer to what I remember seeing. I recommend always using a neutral white balance unless you have a specific reason not to do so.

Now consider this photo:



This is very yellow. But this so happens to be the scene assuming a daylight white balance. Were I to take a photo of a white object under bright daylight with this balance, it would look white. Incandescent lighting is actually far more yellow/orange in color than is daylight.

You may not be aware that your eye does an automatic white balance: when you look at a scene, your eyes attempt to subtract out the color of the lighting, and your eyes do a pretty good job of this, up to a point. Cameras work the same way; they also attempt to automatically subtract out the color of the light.

Keep in mind that perceptibly slight changes in the color of light may translate to a radically different objectively measurable difference in color.

Here is a photo taken outdoors, with a Daylight white balance:



And the same photo, with the color balance set to incandescent lighting; it is the same white balance I used in the nicely-corrected interior photo #2 above:



We see that the camera's white balance corrects for extreme differences in the color of light, not merely minor differences. How the eye corrects for white balance is a mystery, and the technology behind a camera's automatic white balance feature is ultimately imperfect, which is why I do a manual white balance whenever I'm doing my best work.

White Balance and Noise

Digital cameras have an intrinsic, fixed white balance. The color data captured by the camera sensor is adjusted by the camera's computer according to the white balance setting. Here are our sample pictures again, showing what they look like when using my camera's native white balance:



The images are greenish because each green sensor is more sensitive under most lighting conditions than the others.  For more details, see the article What Does the Camera Really See? For an overview of the color system used by digital cameras, read Color Spaces, Part 1: RGB. Digital cameras generally are rather insensitive to blue light.

To correct for the green snow scene above, the camera must amplify both the red and blue color channels to match the green, and whenever you amplify a signal, you also amplify noise.

For the interior scene, the camera must amplify the red channel a little (since incandescent lighting has plenty of red light), while it has to amplify the blue channel a lot, creating plenty of noise.

Here is an extreme example of this sort of noise amplification:



I took this a number of years ago with an inexpensive point-and-shoot camera. While the red and green channels have a lot of noise, due to the camera being set to ISO 1600, the blue channel at the bottom is particularly bad. The blue channel, due to incandescent white balance, was greatly amplified relative to the other channels, and so it shows plenty of noise.

Generally speaking, if you want low-noise interior photos, you are asking a lot of your camera, and you will likely spend a premium to get this.  Read this for details: One Easy Rule for Quality Images.

White Balance and Exposure

If you don't set your white balance correctly, you risk bad exposure if you shoot JPEG. Please note that I define good exposure by taking all three color channels into consideration; see the article Three Opportunities for Overexposure for details. If even one of your three channels is significantly overexposed, you will get shifts in highlight color.  In an extreme case, here is the daylight photo set with an incandescent white balance:



This is a selection from Nikon's View NX2 software, showing the three color histograms at this white balance. Each graph has the dark pixels to the left, and the bright pixels on the right: we see here that the red channel is underexposed, white the blue channel is overexposed.  Due to poor white balance, we irrevocably lost both highlight and shadow detail.

The histograms seen here are similar to the three color histograms found on many digital cameras. Like many photographers, I usually check the color histograms to make sure that my photos are exposed properly. In order to use as much of the camera's dynamic range as possible, I try to expose the images as much as I can without overexposing any one of the three histograms.  However, were I to use this process with a poor white balance, I would inevitably get a severely underexposed image. In the image above, reducing the exposure to preserve the blue channel highlights would lead to a severely underexposed red channel:



Now you might just want to have this ‘look’ in your photo, and that's fine. But if you instead plan on ‘fixing in Photoshop’, forget it.

Please  note that you will see the same problem if you shoot incandescent lighting with bluer white balances — Daylight, Cloudy, and Shade — except your red channel will likely be greatly overexposed while your blue channel will be underexposed.

As a general rule, you will get the best exposure if you use a neutral white balance. You can expose the image longer with less noise and less chance of clipping highlights if you set your white balance precisely.

But please remember that cameras have fixed intrinsic white balance, as seen above. You get a greater risk of overexposing or underexposing color channels when you shoot JPEG images, because the camera throws away lots of its original sensor data when performing a white balance — and you risk throwing away good useful data if you set your white balance wrong. For this reason, I shoot RAW images (which retain all of the original sensor data), because I can adjust the white balance on my computer after the fact. The risk of bad exposure is lessened — although not eliminated — when you shoot RAW and so you still ought to be careful at the time of shooting.

The trade-off is that RAW files need to be processed on your computer to produce an image usable for either printing or displaying on the web. I find the trade-off acceptable, although I know that many photographers do not.

Some authorities state that since a camera has a fixed intrinsic white balance, then the camera exposure histograms ought to show the RAW color channels. I think this is an excellent idea. Some photographers attempt to do this by forcing the camera to use a white balance that does no color adjustment at all: their histograms in this case are correct. The UniWB method of using the camera's own intrinsic white balance was developed by Iliah Borg and others, and the method is described here. It is not for the faint-hearted, since all your photos will turn out green, and you will have to correct for white balance on your computer. You can however get better exposure. At the very least, this is a good learning tool, if not really practical.

Note to camera manufacturers: please show the RAW histograms when shooting RAW! Also, give the ability to zoom in to the brightest pixels on the histograms, for overexposure in digital photography is worse than underexposure. When you show blinking pixels, be sure that they blink when even one of the channels is overexposed.

When to White Balance and When Not to White Balance

As I mentioned before, I recommend always setting your white balance precisely unless you have a specific need to do otherwise.

Have a look at A Digital Color Wheel. Colors opposite from each other are called opponent colors: a color balance biased towards one edge of the wheel will aways be at the expense of the opposite edge. If your image is too blue you will not get enough yellow, and too much green will mean too little magenta. If the white balance is in the center you will trade off quantity of color for quality of color; a well-white-balanced image will look richer in color content, and you have less risk of having your colors go out of gamut. You can get better results in saturating the colors if your white balance is precisely in the middle.

Sometimes you may want to capture a scene as you remember seeing it; but don't forget that your eye already does strong white balancing. So if you want to capture the warm glow of candlelight, set your white balance to somewhat warmer than neutral. To capture the cold mood of a snowy day, set your white balance to a somewhat cooler balance than neutral. Although your eye does do white balancing, this mechanism doesn't work well under dim lighting, although I am uncertain as to what the actual relationship might be. This is worth further research.

Here is a photo where I did not want the camera to subtract out the color of the light:



A fantastical beast, at the City Museum in Saint Louis. Click the image twice to view it on black.

Sometimes you may want an image with a strong overall color tone, but you may get better results if you first convert the image to black and white, and then add a tone afterwards, not relying on white balance.

Mixed Lighting

Your eyes not only do an automatic white balance, but they adjust this white balance variably across the scene while you are looking at it. When you take a photo of a scene, and reduce that panorama down to a tiny, low-contrast image displayed on a page or on a screen, this automatic white balance hardly operates, which is why you have to get it right in the camera. A severe problem occurs when you have mixed lighting of multiple colors, for example, when you shoot an interior, illuminated with incandescent lighting, while also having windows to the outside appearing within your photos.  Invariably your windows will look fine with the interior too yellow, or the interior looks fine while the scene outdoors is very blue. The photograph just doesn't look as you see it in real life.

With architectural interiors, I will use daylight white balance for the windows and incandescent white balance for the interior, and then composite these versions of the image. The results are good, even though this is a tedious process. Big-budget cinematographers will put large yellow-colored gels over the outside of the windows so that the color of the transmitted daylight matches the lighting used in the interior.

Far more problematic is when fluorescent lighting is used in an interior. Not only do these lights have an odd color — typically they are simultaneously more yellow and more green than daylight — but fluorescent colors are not constant across brands of lamps. They do not provide a continuous full spectrum of light, and even more problematic, they change color as they quickly flicker 50 or 60 times per second. Almost invariably, if you attempt to white balance fluorescent lighting, you will get strong shades of the opponent colors green and magenta throughout your image. These green/magenta color casts will be considerably increased if you also have daylight or incandescent lights in the scene. If at all possible, I will turn the fluorescent lights off.

Also see the article: White Balance, Part 2: The Gray World Assumption and the Retinex Theory.

Challenge: "High Photographic Modernism"

I AM NOW a Challenges host on the dpreview.com website. My first challenge is called High Photographic Modernism, which invites images made in the style of the famous Group f.64.

This is a challenge of straight photography: images ought to have great depth of field, extreme sharpness and detail, little over or under exposure, and generally speaking technically precise with a simple composition — and no special effects. The photos must be black and white, although some toning is acceptable.



To enter this challenge, you need to sign up at dpreview. You may enter up to three images from Tuesday,  January 18th, 2011 through Monday, January 24th, 2011. Voting then occurs the week afterwards. When the challenge is finished, you will get an email with your results. Winners receive fame and glory!

For the following week, the theme will be the New Pictorialism. This is inspired by a style of photography popular a century ago, which used classic techniques from painting to make images that were beautiful with a dreamlike quality.

Focal Length

PHOTOGRAPHY IS NOTORIOUS for its many numbers that a photographer needs to know about. Focal length is one of those numbers.

Most inexpensive compact cameras have an easy-to-use zoom feature, and casual photographers can merely set the zoom to whatever they want without worrying about any confusing numbers. But confusion can occur if they use a camera with interchangeable lenses, for then they need to learn about focal length.

Fortunately for beginners, the kit lens that comes with most inexpensive interchangeable-lens cameras is adequate for most purposes. These cameras may even come with two lenses; for example, 18 to 55 millimeters and 70-200 mm. All you really need to know is that large numbers zoom onto distant objects, while smaller numbers capture ‘more of the scene’.

It's easy. If you want to get the whole scene in your photo, you set your lens to 18 millimeters. If you want to zoom in, you set your lens to 55 mm. But then a friend asks you to take a photo of her, using her camera. You stand about ten feet away, and taking note of the millimeter markings on her lens, you set it to 18 millimeters and then look through the viewfinder — and you are surprised that she appears smaller in the viewfinder than you would expect.  She suggests that you zoom in a bit, using a setting of about 30 millimeters. So 18 mm on your camera is the same as 30 mm on her camera. As it so happens, a nearby photographer is taking a photo of the same scene: his camera is large and he tells you that he is using a 55mm lens — but he too is taking in the whole scene, for 55 millimeters is a wide-angle lens for his camera. You learn that focal length settings are not necessarily commensurate between cameras.

A pinhole lens

Light usually travels in a straight line through air, and so we can construct a very crude, but workable, lens just by making a small hole in an opaque surface. Light will travel in a straight line from an object, through this pinhole, where it reaches its destination, which may be light sensitive film or a digital camera sensor.



The focal length of the pinhole lens is merely the distance from your sensor to the pinhole. To illustrate the angle of view of this pinhole lens, draw a line which is the length of your sensor: say, 35 millimeters wide. Perpendicular and centered on this line, draw a dot, which represents your pinhole. Draw a straight line from the edges of the sensor through the dot: this shows the angle of view of your pinhole lens.  If you bring the pinhole closer, the view gets wider; and draw it farther away, and the angle of view gets narrower. You should see that for any given focal length, a larger sensor will give you a wider angle of view. Using trigonometry, you can calculate the angle of view for any combination of sensor size and focal length.  Suppose you have two cameras, one with a sensor twice as wide as the other: doubling the focal length of the pinhole lens on the larger camera will give you precisely the same angle of view as the smaller camera.

Now take a glass lens, and focus it on some object very, very far away, and note the size of the object projected on your sensor. Then take a pinhole lens, and move it closer or farther from the sensor until its projected image is precisely the same size as the image formed by the glass lens. The distance from the pinhole to the sensor is the effective focal length of the glass lens. An 18 millimeter glass lens projects the same size image as would a pinhole located 18 millimeters from the sensor.

But please note that this equivalence between a glass lens and pinhole lens only works when the distance from lens to the object is much greater than the distance from the lens to the sensor. A regular camera lens, after all, is not a tiny dot like our pinhole lens, but rather is made of multiple thick chunks of glass. If you focus a glass lens upon a subject very close by — like when using a macro lens to focus on a small insect — then its effective focal length will change considerably. Click here for more details.

Also note that this equivalence only works when a glass lens produces a rectilinear image — where straight lines in the scene translate to straight lines on the image. Fisheye lenses are a bit more complicated since they produce so much distortion.

Equivalent focal length

Serious photographers use seriously large cameras. This is for the simple reason that large camera sensors — either digital or film — naturally produce cleaner, sharper, more detailed images. Click here to see why. Now photojournalists also want good picture quality, but they also lug cameras around all day long, and so they need a camera that is a good compromise between weight and image quality. Photojournalists are typically the most commonly-seen type of professional photographer — and amateurs, in imitation, started using similar equipment, which included the 35mm film format. Vast numbers of amateur-grade, interchangeable-lens 35 millimeter film cameras were produced, most notably by the same manufacturers who made the photojournalist cameras.

People became quite used to the sizes of lenses for these cameras.  For example, a 50mm lens produced an image that looked rather normal — not too zoomed in, and not too wide. Lenses in the range of say 105 millimeters or larger were good for portraits, while 30 millimeter or smaller focal lengths were good for architectural interiors. Now, please recall that these focal length sizes are for 35 mm film; a medium-format camera would use longer focal lengths for the same purposes, while an inexpensive consumer camera would use much shorter focal lengths.

Eventually the manufacturers of photojournalist cameras went digital; alas, however, due high cost, the digital sensor size was smaller than the beloved 35 millimeter film. Because people were so familiar with the focal lengths used by 35 millimeter cameras, manufacturers stated equivalent focal lengths. So an 18 mm lens used with the new digital sensor is said to be equivalent to (that is, provides the same angle of view) a 27 mm lens used on a 35 mm camera. A 35mm lens on these digitals is equivalent to a 50 mm lens on a 35 mm camera. Is this helpful, or confusing?

Because the 35mm format was rather standard, digital cameras with sensors smaller than 35 mm are often called cropped-sensor cameras, although this terminology can be rather confusing to beginners. I find that beginners often get hung up on the marketing term ‘crop factor’. A 20 mm lens on a camera with a crop factor of 1.5 will provide the same angle of view as a 20 mm x 1.5 = 30 mm lens on a 35 millimeter film camera. Now this terminology is likely only useful if you are very familiar with the old 35 millimeter cameras and their lenses, and is otherwise confusing.

If you are a beginner, I would suggest you forget all about equivalent focal lengths and crop factors. Instead, find out the size of your sensor, in millimeters. For example, many consumer digital SLR cameras have a sensor that is about 30 millimeters across on the diagonal. A wide-angle lens will have a value that is less than this measurement, while a telephoto lens will be much larger than this value. A normal lens — for this sensor — will be equal to this size or perhaps a bit larger.

A Digital Color Wheel

MOST COLOR WHEELS you find at art stores, or images of wheels you find with Internet searches aren't too helpful for digital photography. While they may illustrate the visual order of the colors, they aren't too helpful if you want to mix colors digitally. They may even be quite misleading. So I created my own color wheel using the primary colors found in the sRGB standard, which is used by digital cameras, computers, and high-definition television.


This color wheel shows the correct relationships between the red, green, and blue colors that are primary in the sRGB color system, as well as their opponent or secondary colors of cyan, magenta, and yellow.

These primary and secondary colors are the brightest and most saturated colors that can be generated from the sRGB color system. The coding in each color circle gives you the formula for generating the color: for example, cyan is GB, which means that Red=0, while Green and Blue = 255. Halfway in between the primaries and secondaries are bright tertiary colors. These tertiaries are coded with lower-case letters indicating half a given color: for example sky blue is coded gB, meaning Red=0, Green=128 and Blue= 255.

Some old color wheels use red, yellow, and blue as primary colors; others use green, purple, and orange as primaries. This is misleading for computer use since they don't give us a good idea of opponent colors.  In this color wheel, if you mix together equal portions of colors opposite to one another, you will get a middle gray color; mixing together blue and yellow gives you a gray where the red, green, and blue values all equal 128.

If your images have a color cast, you can achieve white balance by moving towards the opposite color. An image that is too yellow needs more blue, an image that is too green needs more magenta.

UPDATE: My use of a value of 128 for the tertiary colors is not correct, since 128 is NOT the middle tone. It is for this reason that the wheel does not appear to be visually uniform: the tertiaries appear to be somewhat dark. Updated wheel can be found here.

sRGB Colors Out of Gamut

YOU OWN AN inexpensive desktop color printer. You have a digital camera, and you want to make prints. You print your images, and your final photos are disappointing. Does this sound familiar?

This is the color gamut problem: inexpensive desktop printers — those with four ink colors (cyan, magenta, yellow, and black) — cannot reproduce all of the colors that are produced by a digital camera. The best way around this is to get a printer that has more ink colors — but these can be expensive. And so the best alternative is to process your images to make the most of your printer's limited color gamut.

Here are the three primary colors in the sRGB color system:



In the wide strips, we have one of the pure sRGB primary colors going from a value of 0, which is black, to 255 which is the brightest pure color that can be represented by the sRGB system.

Do you see the blue line at the bottom of each strip? This is the color gamut limit of four-color commercial printing presses and inexpensive desktop printers (this color space is abbreviated CMYK, after the four ink colors used: cyan, magenta, yellow, and black). Everything above the lines cannot be accurately printed — which is most of the image. Note that all the bright primary colors are out of the CMYK gamut. The narrow strips on the right are an approximate representation of the colors you will get from an inexpensive printer. Note that greens and blues are particularly poor and relatively unsaturated.

When we mix colors together in sRGB, we still see the same problem:



Here, red goes from zero on the left to 255 on the right; from the bottom, green goes from zero to 255 on the top. Red and green mix together to make yellow.  The areas surrounded by the blue lines are colors that are within the CMYK color gamut! Reds, oranges, greens, and some leaf green colors cannot be accurately portrayed by CMYK, in fact, most of the colors in this mixture cannot be printed.

Other color mixtures are hardly better:



Red going across, blue going up.  Again, most of the image is not accurately printable.



Green across, blue going up. This is somewhat better, but you still can't print decent primary colors.

Things do get better when we have mixtures of all three colors. Here are mixtures of two colors; in each, the third color is set to 50% of its maximum value:



The top image has dark blue mixed in, the middle dark green, and the bottom a dark red.  The printable color gamut is expanded by the addition of the third color.  If we had a pure grayscale image, then all the gray tones will be printable.

Real-world photos typically don't have too many pure, saturated reds, greens, and blues, and so the out-of-gamut problem may be a bit less prominent that what we see here. But most images will have at least some colors that can't be printed:



In this image of a snowman the out of gamut regions are seen on the right, painted in green. If you were to print this image on a four-color printer, these color regions would look a bit flat and unsaturated. You will lose detail also.

But CMYK gives as well as takes away. Even though we can not print the bright red, green, and blue primary colors, CMYK has its own primary colors: cyan, magenta, and yellow, which are typically brighter and more saturated than what you have with sRGB. You could process your images to take advantage of these colors.

To get an overview of these color systems, you may want to take a look at some of these articles:

Color Spaces, Part 1: RGB
An RGB Quiz
Color Spaces, Part 2: CMYK
Part 2 of "Color Spaces, Part 2: CMYK
A CMYK Quiz

When processing for print, you want to emphasize the colors the printer can print, while toning back the colors that are out of the printer's gamut. Following is a relatively simple process where you can make the most of your images in Photoshop.

Convert your image to a wide-gamut color space. Typically Adobe RGB is used. If you shoot RAW, Photoshop's Adobe Camera RAW (ACR) program can select this color space upon import — this would be useful for best quality. Some cameras can shoot JPEG images in Adobe RGB, but I would suggest not using it unless you really know what you are doing.  Select the menu item Edit, Convert to Profile...

Turn on the Gamut Warning feature in Photoshop:

If your target printer has a color profile installed in Photoshop, go to the Custom... menu and select it instead of CMYK. Following shows an image with the gamut warning on; here I have the warning set to gray, but you can change the color to be more visible on a particular image.

Select the Image, Adjustments, Hue/Saturation... menu item:



Note that the drop-down list has both the RGB and CMYK primary colors. For each of the color classes which are out-of-gamut, adjust the Saturation and Lightness sliders until the Gamut Warning turns off.  You can be as careful or as sloppy as you want here, by adjusting the slider on the bottom. You can also select individual colors with the eyedropper tool. (The middle part of the slider shows the colors that will be fully corrected. You can adjust the outside parts of the slider for good blending.)



Here I brought the red bow tie into the CMYK color gamut by darkening and desaturating the color range; but be aware that there may be more than one way to bring it into gamut, with some being better than others. Were I being more careful, I would have done this on a layer with a mask so as not to also desaturate the snowman's smile. Then I corrected the the blue part of the image to bring it within the CMYK gamut. In other images you may also have to tone down the bright primary green colors.

Next we can enhance the printer's primary colors.  We go through the same process as before, but we work with the cyan, magenta, and yellow color ranges, increasing brightness and saturation. The major improvement we can make here are with the yellow colors, which I was able to saturate and brighten considerably. You can brighten and saturate until the Gamut Warning turns on; then you've gone too far. (However, it is OK if some small parts of your image are out of gamut... you just don't want too much over a broad area, otherwise you will lose detail.)

This is a bit of a leap of faith, since you most likely cannot see the final results of your editing: it is out of your monitor's gamut.  If you have areas of your image that ought to have lots of bright cyan, magenta, or yellow ink, you can place an eyedropper tool on the spot and measure the CMYK values directly. If a spot is supposed to have a very bright yellow component, the brightest you can get, then that spot, after your processing, ought to be rather close to having 100% yellow ink.

Purists may insist that all this manipulation is ‘inauthentic’ but in reality this scene greatly exceeded the color gamut and dynamic range of my digital camera; in fact this is a blend of three separately exposed images. So we are justified in making the colors of the snowman as bright and as saturated as we are able to. Likewise, if we are printing a full-color image to a narrow-gamut CMYK printer, we are justified in printing as much of a full-range of color as we are able.

There are many ways of accomplishing a goal in Photoshop, and this one is particularly straight-forward. The most visually accurate color corrections can be made using the Lab color space. Also of use is the Vibrance tool, layers, masking, and most notably Levels and Curves.

When I am preparing images for commercial press, I eventually manipulate the images directly in the CMYK color space, making the most of that limited range of color. Unfortunately we cannot do the same with desktop printers, since they use different inks than are found in commercial presses, and so they typically require the image to be in RGB format. The Gamut Warning feature is the most powerful tool for this purpose.

A CMYK Quiz

HERE IS A sample image, which shows the four channels of a CMYK image. Use your knowledge of CMYK to determine some facts about this image.

If you haven't read them yet, you may first want to read these articles: Color Spaces, Part 2: CMYK and Part Two of "Color Spaces, Part 2: CMYK".

This sculpture of an acorn is found in Wydown Park, in Clayton, Missouri.

I am convinced that a thorough knowledge of the channel system of digital images is essential for good photography. By looking at a color photograph, you ought to be able to imagine which each color channel ought to look like, and by examining the color channels, you ought to be able to determine what colors are represented.

The CMYK color system represents inks printed on a page, and includes the colors cyan, magenta, yellow, and black. Each channel represents one color of ink. No ink is placed on the page where the channel is white; and where the channel is black, we have 100% ink coverage. For example, a bright cyan-colored object will be black in the cyan channel, and white in the other channels. Where we happen to have roughly equal quantities of cyan, magenta, and yellow ink, the CMYK system will subtract those colors and replace them with black ink. So K (black) will dominate the shadows.

Here is your task:

1. Identify each color channel in the image above.
2. There are two colors of flowers in the image. Identify the colors.  We have taller flowers which can be seen in front of the acorn, and shorter flowers of a different color in the foreground.

Unlike my last quiz, I won't give you any clues. Use your knowledge of nature and the channel structure of CMYK to determine the answers.

The Problem of Resizing Images

IMAGINE YOU HAVE a peculiar boss at work. He wants to make sure that you are at your desk working forty hours per week. So once a day (seven days a week!) at precisely the same time every day (since he is extremely methodical), he peers into your tiny cubical to see if you are at work. You are never there, and he is quite upset. You will hear about this at your next annual review, nine months from now. Sadly, it appears you won't be getting a raise.

Well, he looks into your cubical precisely at midnight every day. Despite graduating with honors in a top M.B.A. program, he really isn't all that bright, and he lacks a life outside of work. As a matter of fact, you do work 8 a.m. to 5 p.m. (with an hour for lunch), Monday through Friday, and you are always at your cubical during those times. But boss marks you down as being absent 100% of the time.

Well, since you actually do your required work, bossman decides to check up on you four times a day. Quite methodically, he appears at your cubical at midnight, 6 a.m., noon, and 6 p.m. As it so happens, you take your lunch at noon, and he just barely misses seeing you every time. You are still absent 100% of the time, in his mind.

Boss is still puzzled. With apparently too much time on his hands, he checks on you eight times a day. Midnight, 3 a.m, 6 a.m, 9 a.m., noon, 3 p.m., 6 p.m., and 9 p.m.  He finally sees you! Since he sees you 2 out of the 8 visits he makes, Monday through Friday, he estimates that you are working at most 2/8 x 24 x 5 = 30 hours per week. He is disappointed, but at least you get to keep your job.

Note that if he visited your cubical three times a day, at midnight, 8 a.m., and 4 p.m., he'd see you twice (the first time just as you got there) and would estimate a working time of 2/3 x 24 x 5 = up to 80 hours per week. But four times per day gives zero. Clearly the frequency of his visits can change the results dramatically.

Your boss's boss likes what he is doing, and asks that he get more data so that he can present an impressive chart at an upcoming meeting. Your boss now checks your cubical 12 times a day. He visits at midnight, 2 a.m., 4 a.m., 6 a.m., 8 a.m., 10 a.m., noon, 2 p.m., 4 p.m., 6 p.m., 8 p.m., and 10 p.m. He sees you working 4 times per day, Monday through Friday, and so he estimates that you work up to 40 hours per week.  If he visited your cubical 24 times a day, or 48 times a day or more, he may (hopefully) notice that his increasing visits didn't give him much more useful data: he would always get a result close to 40 hours per week.

Slightly different

Let's consider a slightly different scenario. Your boss is always on top of the latest scientific theories. He read that the natural sleep-and-wake rhythm of human beings, when not exposed to the cycle of the sun, is 25 hours a day. Always ready to implement these newest findings, and since he apparently never sees any sunlight, your boss now lives out a 25 hour day, although who knows when he actually gets any sleep. Instead of checking your cubical once a day, he checks it once every 25 hours. If he sees you at your desk, he gives you credit for the entire 24 hour day (unfortunately, he has yet to convince his boss to move all the employees to this new scientific schedule). So the first day, he checks for you at midnight, the second day at 1 a.m., the third at 2 a.m., and so forth.

Although he finds you working sometimes four days in a row, he is infuriated that you are (apparently) taking two week (and longer!) vacations at regular intervals. He does estimate that in the long run you are actually working on average 40 hours per week, but he is worried about all the important conference calls you must be missing.

No Common sense

This hypothetical boss, despite being diligent, lacks common sense. This lack of common sense, while being reprehensible in a human being, is quite the norm with digital cameras and with computer software such as Photoshop, although we must credit computer technology with also being diligent. It's hard — no, impossible — to program a computer with common sense, and so we ourselves must make up for what computers lack if we want good results.

Better yet worse

I was delighted when I upgraded from a nice but lowly point-and-shoot camera to a decent, yet inexpensive, DSLR model. Immediately I noticed how sharp my new photos were, as well as having much less noise, even in low light. But there was a problem, and I couldn't quite put my finger on it.  With my old camera, when I was processing images for the Internet, I would simply resize them and add sharpening. Even though there were various re-sizing algorithms available in Photoshop, none seemed to make much of a difference. I did put a lot of effort into using good sharpening algorithms, which made my photos look much crisper without obvious artifacts. But this did not work well with my new camera.

My old process did not work all of the time with my new camera — and the maddening thing was that my results were quite inconsistent — some of my final images looked fine, some were terrible (nature photos were typically the worst). Formerly, when I reduced the size of my images, I had Photoshop set to use the Bicubic Sharper algorithm, which Photoshop says is “best for reduction”, but I found that the new camera's images looked quite rough. So I changed it to use regular Bicubic. This required quite a bit more sharpening than I had used before, and I started using better algorithms that would reduce the bright artifacts I was now seeing, especially around distant leaves on trees and along certain edges. Sometimes I had to manually retouch out some of the sharpness. To me, this is not acceptable, so I started asking around for advice. As it turns out, Photoshop gets it wrong, and does not implement its resizing algorithms correctly.

Hit a brick wall

Digital cameras have ranks and files of pixels, arrayed across their sensor, in precise order, just like the smart-yet-foolish boss in our allegory above. Precisely every x micrometers, a different sensor captures light, just like precisely every y hours the boss would check up on his subordinate.

In the story, you arrive at work at a regular interval, but your nosy boss, if he didn't check up on you frequently enough, would get a wildly inaccurate estimate as to when you actually were present. Only when he checked up on you many times in a day did he get an estimate that was accurate enough.

A similar same thing happens in a digital camera. If there is an underlying, repeating pattern in the scene, the camera may get a wrong estimate of what the scene looks like if you have an inadequate number of pixels to capture enough detail. We see this on initial capture. Here is a section of a photograph, showing textured carpet:



The camera did not have enough resolution to capture the repeating texture of the carpet adequately. So we end up with ugly artifacts, here shown by the odd pattern. There were no curves in the texture of the carpet, but the pixels on the camera, being spaced too far apart, got a strange signal. It was not sampling the texture frequently enough. This pattern is called the Moiré effect or an interference pattern, and is a special case of aliasing.

Not only do we see this on initial capture, but this can be a severe problem when we are downsizing an image. Downsizing in business ruins lives, while downsizing in digital photography ruins images.  If there is a repeating pattern in an image, we can get bizarre patterns upon downsizing if we end up with fewer pixels than a particular pattern requires. Here is a detail of a larger image:



A brick wall. We have a classic, repeating pattern, which will test Photoshop's ability to resize.

First we use Bicubic Sharper, which Photoshop tells us is best for reduction:


Ugh. Bad pattern. Just like the boss who was checking on you at 25 hour intervals (and thinking that you were frequently taking 16 day vacations), we see some bands of the brick wall where the lighter white mortar predominates, and other bands where the dark brick predominates. Also, the rest of the image looks rather rough.

Please note: these sample images are intended to be viewed at 100% resolution. If you are viewing these images on a mobile device, they may be further resized by your device, not giving you an accurate representation.

Now let's try Bicubic:


The repeating pattern is still there, but the rest of the image looks a bit better, if soft. Now normally I'd add sharpening to an image like this, but the pattern on the bricks just looks unprofessional.

Bicubic Softer does not help:


Now lately I've been using the Bilinear algorithm for resizing. The final images, to my eyes, look crisper than Bicubic, yet less rough compared to Bicubic Sharper.  Let's try Bilinear on the brick wall:


Interesting. The pattern changed, and maybe it is somewhat less obvious. But still bad. I do like how the rest of the image turned out though: it would hardly need any sharpening at all.

For the sake of completeness, let's try the Nearest Neighbor resizing. As Photoshop says it is best when we want to ‘preserve hard edges’, and since the building has hard edges which we want to preserve, it should look fine, right?


Nope. Blech. Looks like a zebra.

Note that the big problem we are seeing is due to the regular pattern of the subject, such as the brick wall, coupled with the regular pattern of pixels on the digital image. We do not see moiré patterns with film cameras and prints: the chemical film grains are irregular shapes and sizes.

But fortunately there is a good general theory to help us out. The Nyquist sampling theorem states:

If a function x(t) contains no frequencies equal to or higher than B hertz, it is completely determined by giving its ordinates at a series of points spaced 1/(2B) seconds apart.

Very roughly speaking, if we take a picture of something that has a regular pattern, if we don't allocate more than two pixels for a repeating element, then we will get the moiré pattern. But actually it is slightly more complicated than that, since we have three colors of pixels, at slightly different locations on our sensor. In the photo of the carpet above, there is much more moiré in the red and blue channels compared to green, as we do have twice as many green sensors.

There are also other mathematical effects involved to complicate matters, such as the Nyquist theorem assumes the frequencies are perfect sine waves. A pattern with hard edges, such as the bricks, actually are equivalent to somewhat higher sine wave frequencies.  So some authorities state that for hard-edged repeating patterns such as these bricks, and with a Bayer Array  (where we have separate photo-sites for each color channel), we ought to capture at least three (or maybe up to four) pixels per repeating pattern element to avoid aliasing.

We find the exact same thing when downsizing an image. If the final resampled image does not have more than two pixels capturing each element of a repeating texture in the original image, we will get a moiré pattern. Because of the complications given above, maybe we need a little bit more, like 3 or so pixels just to be safe. So if our bricks, about 10 pixels apart vertically on the original image, are roughly reduced 1/5 or less in size, then they will definitely show a bizarre pattern, since we are allocating two or fewer pixels per brick. This is what we see in the photos above. I didn't get any moiré effect when I downsized the image to either 50% or 33% (5 or 3.3 pixels per brick) — and just started getting moiré at 26%, which is about 2.6 pixels per brick.

This is analogous to what is done in the audio recording industry. Young, healthy human ears can hear frequencies up to about 22 kHz, and audio engineers will sample the audio at more than twice that frequency, 44.1 kHz, to avoid audio artifacts like we see in our aliased images.

Boss tries harder

Your boss still wants to keep track of you, but because he has other duties, he attempts to automate the task. He installs a sensor at your cubical door. Whenever you are in your cubical, the sensor records that fact. At the end of a fixed period of time, the sensor resets itself and sends a signal to your boss's office whether or not you were in your cubical anytime during that period. He sets the sensor to send him data every day at midnight. He successfully finds out that you are in the office every weekday. Under your boss's old system, he knew precisely when you were at your desk at a given moment in time, but the new system, while it is less specific, gives him more useful information. In effect, the sensor blurs the boss's data a bit, but he gets better results. With one sample per day, he gets better results than visiting your cubical four times. Were he to sample the sensor more times per day, he would get a much better idea of your attendance than if he were to visit your cubical the same number of times. Maybe he'll find something better to do with all the time saved.

As it so happens, digital cameras incorporate anti-aliasing filters to combat moiré patterns. This softens the image a bit, but it can lessen the effect that we see in the carpet photo above. Consumer grade compact cameras tend to have heavy anti-alias filters, DSLRs have weaker ones, while medium-format digital camera backs may have none. With the higher-grade cameras, it is up to the photographer to either avoid or correct for these imperfections — although with more pixels, this will be less of a problem.

Blurring is the key

This softening is the key to downsizing images. According to the Nyquist theorem, our samples need to be more than double the frequency of the original signal to avoid artifacts, but when we make an image smaller, we greatly increase the frequency of our patterns. So what we need to do is to blur the image first — before downsizing — so that the Nyquist theorem still holds for our final image. In more technical terms, an image needs to be put through a low-pass filter before being down-sampled — the high-frequency components of the image have to be eliminated first by blurring.

I started getting the ugly artifacts when I reduced the image below 2.6 pixels per brick, and so to eliminate them we need to run the image first through a low-pass filter, which will get rid of any detail 2.6 pixels in size or smaller.

Photoshop does not blur an image prior to downsizing, not even the newest Photoshop CS5. That is why we get these digital artifacts. I would think that this would be fairly easy to implement.

How an image ought to be blurred prior to downsizing is a mathematically complex subject, and certainly the optimal blurring algorithms are not found in Photoshop. But we could experiment with Gaussian Blur, although choosing the Gaussian radius may be a bit problematic.

OK, so we want to be sure that we don't have any frequency components of our bricks being any less than about 2.5 pixels per brick in the final image. I initially choose to apply a Gaussian blur with radius 2.5 before downsizing.  This is a quite naïve start, and so I did blurs in various steps:



Radius = 2.5. Just for fun, I used the Nearest Neighbor resizing algorithm, which gave us the horrendous zebra stripes seen above.  It doesn't look too bad, does it?  I added 50% Photoshop Sharpen to these images to make them look a little better. Better sharpening is called for however.

Here are other Gaussian blur radii:



Radius = 1.  We still have severe aliasing.



Radius = 1.5.  Still some aliasing.



Radius = 2. Some very faint aliasing; otherwise this is a good image.



Radius = 3.  Too soft.

Ok, for sure we can get rid of aliasing when it obviously appears on an image like this one. But this may not be optimal, for the final image appears a bit too soft. One trick I've used is to blend together two copies of an image, reduced using different algorithms. In this case, I'd select the anti-aliased part for the bricks, with a normal downsize for the rest of the image.

However, anti-aliasing may help images even without an obvious pattern such as this. I recall that I often get poor resizing results, particularly with distant leaves against the sky, and along certain edges. Perhaps using even a soft blur will help with these images.

But we really ought to be using better algorithms than Photoshop offers. Very many algorithms are implemented in the free ImageMagick command-line utility, and in-depth discussions are here and here. For downsizing, they recommend the Lanczos algorithm for photographic images. It properly does blurring before reducing, although it does not use the optimal blur algorithm, for the sake of good performance. Using that software, I resized the brick building:



Lanczos still has a bit of moiré, so I'm a bit disappointed. Otherwise it looks pretty good, and is much better than any image found above.

I tweaked the processing a bit and got this:



I blended the above image with a version that I blurred before downsizing. I masked out the building in the unblurred layer, giving us this composite.

Apparently there are some other, better algorithms available, but they are computationally expensive, or difficult to fine-tune optimally. However, whichever resizing algorithm you use, it is important to sharpen the image afterwards to bring back some crispness to the image.