Tuesday, June 29, 2010

Luminance is More Important than Color

A SIMPLE ILLUSTRATION, that luminance in an image is more important than color:

DSC_2905 - color only

This shows the colors from an image, stripped of its luminance.

What is in this picture?

Monday, June 28, 2010

There is Detail in Noise

NOISE IS THE ENEMY in digital photography. Inexpensive compact cameras produce decent enough images in broad daylight, but when light dims, noise increases greatly until the image becomes quite ugly. Understandably, many photographers are interested in computer software that will reduce noise in digital images, but they also ought to also know the limitations of noise reduction.

It is useful to see how digital noise works, so I contrived an image that shows noise in a fairly pure way, unlike what we see in actual digital photographs. Following is an example where I added Gaussian color noise uniformly across the image, without regard to color or luminance. From top to bottom, we have no added noise, then 25%, 50%, 100%, and then 200% noise at the bottom.

There is Detail in Noise 1

Viewing this, it is pretty obvious that noise reduces contrast, and that high contrast objects resist the effects of noise the best. The word ‘THERE’, — black on white or white on black  — is recognizable even at the highest noise level, while blue on black becomes quickly unrecognizable. You can click on the image get a larger version.

But it is clear that there is noticeable detail even in severe noise. This consideration is important when devising methods to reduce noise, and when deciding how much noise to remove from an image.

When we view extremely noisy images, we are at risk of making two types of errors. We may perceive a signal in the noise when in fact nothing but noise actually exists: this is excess credulity and is called a “Type I error”. Likewise, we may not perceive a signal when in fact one does exist: this is excess skepticism and is a Type II error. Noise reduction software makes the same kind of decision: every pixel is judged as being some combination of signal or noise, and the final image produced may exhibit errors of both kinds. An excessively skeptical noise-reduction technique will generate images with very little noise, but with little detail also. An excessively credulous technique will pick out false detail, creating ugly artifacts.

In this sample image, a fluent reader of English would be able to pick out the words in the noise better than someone who is not familiar with English. I included the odd symbols at the bottom so I could include other colors, but it is also clear that familiarity with a symbol helps aid our recognition, and so these symbols seem to suffer particularly poorly under noise.

This example points out a severe limitation of noise reduction technology. We already know what the words are, since we are giving a noise-free example image at the top. Would we be able to determine the words if we are presented with only a noisy image? Our mind certainly ‘fills in’ missing information when we know for certain what that information must be. Furthermore, natural languages have much redundancy in grammar and spelling, and so this helps us to identify noisy unknown sentences if we are familiar with the language.

Noise reduction therefore quickly becomes an ‘AI-complete’ problem — that is, it requires a computer with full human intelligence. If we insist that our noise-reduction algorithm be perfect, then we are asking the computer to have the knowledge of God. Rather, we must be humble and instead accept general-purpose solutions that reduce enough noise to produce an image that is good enough; or, we must accept that our image is too noisy and instead re-shoot the subject using better equipment or technique.

There are special-purpose noise reduction algorithms for character recognition; these take into account the shapes of letters, common dictionary words, and basic grammar.  These algorithms will allow a computer to scan and read a book, but these will be completely unsuitable for general photography.

Fortunately, the image above is quite contrived, and the noise I added does not have the characteristics that we find in a digital photo. I provided that image for the purpose of demonstrating the effects of noise in general, without the complications found in a photograph. However, I would challenge any interested reader to download that image, and try various noise-reduction techniques on it. I failed to significantly reduce the worst noise using various techniques — and ugly digital artifacts or side-effects of these attempts are quite obvious:

There is Detail in Noise 3

From top to bottom are: a section from the original image; the same section processed with Photoshop Reduce Noise; then Noiseware Professional; Surface Blur of chroma channels; and finally the popular Dust and Scratches noise reduction from Photoshop. Note the color shifts and poor performance of all methods. Are any of these results better than then original noisy image? It seems not.

Following is a generated image that has a noise profile closer to what we find in natural photographs. Here, noise is great in the shadows, and is slight in the highlights. There are different levels of overall noise in each channel, similar to what is found with photos taken under incandescent illumination: green has the least noise, red a bit more, and blue has much more noise.

There is Detail in Noise 2

The default settings of Noiseware Professional does a decent job of denoising this image, and it could be easily cleaned up nicely with a bit more effort:

There is Detail in Noise 2b

The limits of noise-reduction technology tell us that we have to be careful to avoid noise when taking a picture, and that we have to use intelligence when applying noise reduction, and not to fully rely on the computer. If we know for certain what the signal ought to be, then we can do manual retouching: in the image above, I would retouch the blue-on-black word NOISE, and do general blurring of the dark background. If our job is to produce a good image, and not necessarily a faithful one, then artistic judgment is needed to create plausible detail with little noise.

Sunday, June 27, 2010

Three Opportunities for Overexposure

EVERY CAMERA — as far as I know — ultimately uses precisely one measure when exposing an image: the Exposure Value. This E.V. is made up from various combinations of:
  • lens aperture size,
  • the time the shutter is open, and
  • the light sensitivity of the sensor or film; 
and although the camera's estimation of E.V. may be adjusted by the photographer, or modified by the evaluative algorithms in the camera's embedded computer — matrix metering — camera exposure is eventually reduced down to a single number.

This one number, if it is not in a narrow range suitable for the subject, will produce a completely unacceptable image. For example, here are two photographs of my cat Shilo:

Portrait of a Cat I (Shilo) [EV=-4.5]
Portrait of a Cat I (Shilo) [EV=-4.5]

Portrait of a Cat II (Shilo) [EV=21.3]
Portrait of a Cat II (Shilo) [EV=21.3]

While these might be interesting exercises in conceptual art, they certainly fail as photographs of my cat. For this subject the valid E.V. would be somewhere around 5. Click here to see how exposure value is calculated.

A suitable exposure value for a particular subject must be within a particular range, and although this range may conceivably be quite large, depending on the intent of the photographer, it is certainly not infinite, as the images above show.

Overexposure gives white, underexposure gives black, and shades of gray and color comes from correct exposure, which is somewhere in-between.  But we have a problem, and that is color. A digital camera does not capture a single image, but rather captures three: one image each for red, green, and blue colored light, which combine together to make full color. Each of these channels has its own range of desirable exposure values, and these ranges will likely not be equal. The camera gives us one E.V., while we really need one for each channel, and so we have three opportunities for overexposure, and not one.

These three color channels can only reduce the acceptable range of exposure values, as we can see here in this particular example:

EV diagram
Practically speaking, this phenomena is the reason why color photography is often benefitted by flat lighting, and also why monochrome images can capture more tonal depth with beautifully deep, detailed shadows.

When a digital camera captures a JPEG format image, every resultant image pixel is made up of three numbers; one for each of the colors of red, green, and blue, and these numbers range from 0 to 255. Disregarding digital noise, if a certain minimum amount of light fails to reach a particular pixel, the resultant value is 0, and if more than a certain maximum amount of light hits it, we get 255. As we increase exposure on a colorful object, one of the channels will eventually reach 255 before the others, and this overexposure will cause a color shift.

Here is an example of a sky blue color, simulating an increase in exposure:

Demonstration of color shift with luminance

The color noticeably shifts in hue, showing how the acceptable range of exposure values is greatly reduced if we want to capture accurate color.

For a real-world example, see these two photos which have slightly different exposures:

Overexposed blue channel

In the first exposure, the blue channel in the sky area is largely at 255, indicating overexposure; the color is obviously wrong since it is an unnatural cyan. The second picture has a good sky color because the blue channel falls below 255 and so retains its proper proportion with the other colors. Note that the green and brown hues of both photos appear to be identical despite different luminance. Generally, if large areas of any one channel is at 255, there is a likelihood of color shifts and loss of detail. Because of this, overexposure ought to be avoided by examining the three color histograms found on many cameras. Reducing exposure like this may make an image relatively dark, but there is typically a lot of detail in the shadows which can be made visible in post-processing.

A photographer needs to determine which areas of a scene contain significant detail, and expose and later post-process the image accordingly. In our example photos above, the blue sky is significant detail and ought to be exposed properly. Usually, out of focus backgrounds and specular highlights can be overexposed without problem, but these often require post processing to remove the strong colors that come from the color shifts described here.

Ultimately, good exposure may not be possible, for the range of detail that must be captured is beyond the range of the camera's sensor; in these cases, supplemental lighting or the High Dynamic Range technique are needed, to either reduce the range of the subject, or to capture the entire range of the subject.

By the way, here is Shilo, well-exposed:

Portrait of a Cat III (Shilo) [EV=5.7]

Saturday, June 26, 2010

The Virtues of Art and Science

THIS IS “A website about the Art and Science of Photography”, so I thought that I would begin by explaining some terms.

To many in the arts establishment, photography was not a legitimate art form until perhaps the 1960s; to me this appears to be wrong, especially in retrospect, considering the many fine photographs produced earlier in history.  In my opinion, photography is and always has been an art, although it has been continuously under refinement (with some hiccups).

I am often amused with the Wikipedia entry on Art because it changes so frequently. As the current edition of that article may or may not show, there are many definitions of art, and most of them are excessively restrictive in my opinion. But I will use a very old, and quite broad definition, in the tradition of Aristotle and Thomas Aquinas:
Art is the virtue of making things well.
What does this mean?

Art isn't just about making paintings or sculptures that are to be viewed in an art gallery; rather anything can be art, if it is artfully done.  For example, cooking isn't normally considered an art, but a master chef is often called an artist, as is only fair.  Some automobiles are so well designed that they are called moving sculptures, and rightfully so.

Art is a property of a person, the artist.  The artist is therefore the subject of art, while the things the artist makes are the objects of art.  A good photograph may or may not be an ‘object of art’ by this definition: does the photographer habitually make good photographs or not?

Art is a virtue, and according to the ancients a virtue is a good habit of a person.  It is habitual, that is, it becomes a part of the person and their ordinary behavior.  According to the philosophers, virtues are obtained through innate ability, knowledge, practice, and inspiration.

The example of a musician may be helpful.  To play a piano well, you need the innate ability to play it: your fingers have to be of a certain minimum size, and you need to have a good ear for tones; some people are tone-deaf for whatever reason, and that is not a good sign for them ever being able to play the piano well. Some people are famed for overcoming obstacles even though they are not well-predisposed to doing something: but this is a shift from having the virtue of art to having some other virtue, like courage or perseverance — so these people become even more admirable.

Getting a good theoretical knowledge of music is very helpful to gaining the virtue of being a musician: although someone may play the piano just through practice, their art would be quite limited.  For example, the inability to read music or transpose keys would perhaps prevent them from being hired to play in an orchestra. Likewise, a theoretical knowledge of photography is important for being a professional: perhaps someone, merely by practice, will eventually learn the relationship between aperture and depth of field, but knowing the theory behind it will allow the photographer to set the camera precisely the first time.

Practice is the most important part of gaining a virtue.  A neophyte musician must constantly practice, no matter how bad the music sounds at first. Feedback from others, especially an established musician, is certainly required. A neophyte photographer should expect to take plenty of bad photos, but with feedback from an experienced photographer, this is extremely helpful and not a waste of time. A very helpful method to improve technique is completing assignments: taking photos of specific subjects with prescribed techniques. The Daily Shoot website can help build this discipline.

Eventually, with ability, knowledge, and practice, the musician gains the virtue of art. They remain a musician even when they are not making music, for music has become a part of them.  Music in the abstract has been instantiated or incarnated in the musician. The Zen advice, sometimes comically seen in the phrase “Be one with the ball!” is not exclusively Eastern but also has a traditional Western philosophical counterpoint.

There are two marks of a virtuous person: they enjoy practicing their virtue, and they make the performance of the virtue look effortless and graceful. Anyone seeing a virtuous artist at work ought to think “that looks easy!”  And it is easy for the artist, but only after much sweat, disappointment, and lots of bad artwork.  We ought to also consider that many people choose professions based on the income or status that the profession confers; they may be good at performing the tasks of that profession, but they hate every minute of it — this is not then the virtue of art; although it may be another virtue, it is not art and the the final objects of art may suffer.

I think a critical part of having the virtue of art is the ability to preconceive an object of art, and then be able to reliably make it.  A photographer should be able to know ahead of time what he or she wants as a final image, and then be able to reliably and repeatably do what it takes to get the desired results. Now I often use the shotgun approach towards photography, and certainly some of that is required for journalists, sports photographers, and anyone who uses a live model, but artistic virtue would tend to greatly minimize excess ill-advised shutter clicks.  It is a delightful problem having to choose only one photo from one hundred good ones, rather having to salvage one mediocre image from a hundred bad ones.

Artists can also be inspired.  The subject of inspiration is problematical, and is rather outside the scope of this website.  Click here for some of my thoughts on the matter.  However, we can take heart in the theory put forth by Plato in the Ion: by imitating inspired works, that inspiration flows through to us. Only when you don't care about creativity can you be truly creative, for mere novelty does not always lead to goodness in objects of art.

So from all this we can be confident that photography is an art, and has been an art since the very beginning.

Often it is said that the arts deals with aesthetics, a meaning that is not found in my definition above.  A problem then becomes defining aesthetics, of which there seems to be no good definition.  But consider its opposite, anesthetic; a person under anesthesia has little or even no feeling whatsoever, and may even be unconscious. This makes it clear that everything has an aesthetic dimension, and that it may be impossible to cut off aesthetics from the rest of experienced reality.  Certainly the practical split between aesthetics and utility found in our world today is quite ugly, as I've mentioned elsewhere.  This problem of the scope and meaning of aesthetics is an unresolved problem in my mind.

Let's consider science.  The ancient definition is:
Science is the virtue of conforming the intellect to reality.
This again is a very broad definition. Science is not a virtue for making or doing anything, but rather is a virtue of changing yourself.  A virtuous scientist conforms his mind to what is real, not to what he feels, or what he wants to do. Now of course feelings are real and must always be taken into account; being cold and heartless is definitely not a virtue. The ancients valued reason over emotion — but reason includes the heart, which has its own reasons, as Pascal noted.

By the ancient definition, science is not technology, nor is it a profession, nor is it a method, nor is it a set of results; rather it is a habitual state of mind.  A good scientist habitually asks questions such as: what actually exists? how do certain things work? why do certain things happen? when and where are certain things likely to occur? who knows more about this subject than I do? A scientific photographer would not be content with merely following formulae or memorizing rules of thumb about photography, and certainly not rely on the “shotgun approach” of taking photos, but rather would ask why certain things work the way they do. Ansel Adams, who was no doubt a photographic artist, was also certainly a photographic scientist, as was demonstrated by his Zone System.

According to this ancient model, art and science are not opposed to each other as they are in the more modern philosophies of Romanticism and Enlightenment Rationalism, but rather both virtues are closely related and harmonize with each other. Ideally, a good photographer would have both virtues, but no one is perfect: however, this should not be an excuse, but rather should be a challenge to improve yourself.

So, the art of photography is found in an artist, who reliably and repeatedly makes good photographs according to plan. Likewise, a photographic scientist is a person who distinguishes what is true about photography, and what is merely opinion.

The ancients distinguished between the intellectuals virtues of art and science and the moral virtues of justice, prudence, courage, and temperance.  The moral virtues are what make you a good person, while the intellectual virtues merely make good things.  While being a good artist or scientist is laudable, this does not make you a good in the moral sense.


WELCOME TO The Refracted Light, a website about the art and science of photography.

The name ‘Refracted Light’ refers to the process where waves, when hitting a boundary at an angle, will change velocity, and so will also change direction.  This principle is what makes optics and cameras possible: light waves, when hitting the glass of a lens at an angle, will be bent.  Lenses are precisely constructed to focus these light waves at a point, where an image can be captured. This name perhaps is evocative of photography.

But the name also comes from the poem Mythopoeia, by J.R.R Tolkien, who is better known for his epic novel of high fantasy, The Lord of the Rings. Quoting the poem:
Man, Sub-creator, the refracted light
through whom is splintered from a single White
to many hues, and endlessly combined
in living shapes that move from mind to mind.
Tolkien wrote this in defense of creative myth-making, rebutting his friend “who said that myths were lies and therefore worthless, even though ‘breathed through silver’”. Tolkien's imagery can also apply to photography: a thousand photographers can each photograph the same scene with the same camera, yet a thousand different images result. Photography may be our most objective art form, but the single white light of Truth gets refracted through each person, producing many hues.

This is a website about photography, and not of photography.  I do not plan to showcase my photography here, other than as useful illustrations.  You can find my photography on my other website Rome of the West, which is:
A web log about Catholicism in Saint Louis, which was once called the “Rome of the West”. Topics of interest are the historical Catholic patrimony of our City, the restoration of Catholic culture, manners, and morals, increasing public and private piety, and fostering interest in the liturgical arts.
It is a common rule of publishing that if you write about subjects A and B, you will only get readers who like both A and B, and not readers who like one or the other.  This is why I created this new website. Taste in photography also varies, and as I hope what appears on this page is useful, I'd rather not turn away those readers who do not share my taste in photos.

So I intend to write about the more certain, objective factors in photography, things that are less a matter of personal taste and legitimate varying opinion. The laws of physics are objective, the operation of specific cameras is in principle knowable, but even some subjective factors can be known objectively to a good degree, such as the properties of vision that are fairly uniform among persons. I also intend to delve into Photoshop and other image processing software as it relates to photography.

It do not intend this to be a gear blog.  The debates between Nikon and Canon or Sony and Olympus partisans may be exciting to some but they annoy me; that is not to say that I don't have my own opinions, but I would like to keep the level of discussion here at a somewhat higher level.  I don't have the time or money to do comprehensive tests on equipment. Now if any manufacturer wishes to send me some gear for a review, I would most likely write something complimentary about their product: after all, it is polite to exchange courtesy for courtesy, and if you can't say something nice about someone, then don't say anything at all.  A truer test of any piece of gear is whether or not I'm still using it six months later, and I will most certainly let you know what I am using.

I think there is a problem with the ‘Consumer Reports’ model of purchasing gear, where the consumer pours over the various camera ratings before making the ‘correct’ purchase.  A major problem is that the people who make the ratings do not know you as an individual, there is no give-and-take or dialogue between you and them, which could help you make a better purchase for you. You might decide what is important to you, but how did you come up with that? Psychologically, this turns consumption into into a mainly passive activity: you feel wise about a purchase, but your own choices end with the purchase. But do you do your part after the purchase? Do you make a strong effort to use your new tool effectively?  Or do you blame the equipment if the results are inadequate? When I first went digital, I bought the ‘best’ but my results were bad, and I blamed the equipment; I did not make an attempt to do my part to produce good photography. An opposite, but equally bad attitude would be insisting that your photographs are the best because your camera is the best. I wrote about my early and bad attempts at digital photography the article A Camera Diary over at Rome of the West. My attitude of blaming the equipment is actually quite common these days, and have deep psychological and philosophical reasons behind them: for more details, I recommend the excellent, although somewhat dated book by Walker Percy, Lost in the Cosmos.

Dear readers, I hope that you won't get offended if I have very high expectations for you. The scientific aspects of photography can get very complex and mathematical — and even the artistic aspects can be very difficult, as with the excellent work of Ansel Adams and his Zone System. Please forgive me for not dumbing-down my writing for you. But I hope that I won't make things obscure either, which is all too common in modern art criticism.

My name is Mark Scott Abeln, I have been blogging for about five and a half years, and have been taking photography again seriously for a bit less amount of time.  Although I was long enamored of film photography, having many fine cameras and a darkroom, I made the transition to digital photography rather poorly.  Circumstance and motivation led me to relearn photography from scratch, and this website is part of my own continuing education in the subject.  My college degree is in physics from Caltech, and I worked for many years in the computer and industrial engineering fields, so this ought to give you some idea of where I am coming from.

I am Catholic and this has a profound effect on my preferred subject matter and approach to photography; however, if you want to know more about this — as well as higher approaches to art theory — please see Rome of the West.  I intend this to be a secular website, although there may be a Catholic flavoring at times.

My major book of photography is Catholic St. Louis: A Pictorial History, with text by prominent historian Fr. William Barnaby Faherty, S.J.; my photography can also be found in Saint Louis University: A Concise History, and on the covers of the books Tower Grove and Caring for Victor: A U.S. Army Nurse and Saddam Hussein.  Other books of my photography are planned for the future.

My writing and photography can be found on very many websites, and on posters, bulletins, and magazines.


Catholic St. Louis: A Pictorial History can be purchased at all major online booksellers, or you can click here to purchase an autographed copy: