Tuesday, May 3, 2011

Zillions and Jillions

PLEASE CONSIDER the absolutely most minimalist camera possible. This camera will have precisely one pixel or light sensor location, and it will generate a photograph that has a bit depth of exactly one binary digit: black or white. This camera can produce precisely two images:


This maximally minimalistic digital camera is actually useful. These are incorporated into proximity switches; devices that answer the question is something there? You might find them on conveyor lines in factories, or on automatic door openers.

Instead of just one pixel, let's consider a camera with four. Here are all possible images taken with this sort of camera:


With just a 2 by 2 pixel array, at one bit depth, we are able to take 16 different photographs.

We can easily calculate the total number of individual photographs that can be possibly taken by a digital camera capable of displaying only black or white:
  • 1 bit camera = 2 photos
  • 2 bit camera = 4 photos
  • 3 bit camera = 8 photos
  • 4 bit camera = 16 photos
We add one bit and we double the number of possible photos we can take with our camera. This number gets very big very fast. Suppose we take a very low end digital camera, 500x500 pixels = 250,000 total pixels. Even if we limit this camera to using only black and white pixels, we end up with a huge total number of possible images:

Total images possible =
  • 2 x 2 x 2 x ..... (multiply a total of 249,999 times)
  • = 2250000
  • =  3 followed by 75,257 zeros, plus a bit more.
By comparison, the total number of elementary particles (electrons, protons, etc.) in the entire universe is estimated to be a 1 followed by 80 zeros, or 1080.

Suppose you have an entire universe of particles, and then you give every particle its own universe of the same size, and in each of those universes, you assign a similarly sized universe for each elementary particle inside them, and repeat the process nearly a thousand times. That's how many unique photographs you can take with your cruddy 500x500 pixel 1-bit depth digital camera.

We can get large numbers when we examine matter, but information is another class of being in itself, vastly larger than mere matter. It is for this reason that philosophers have posited that information resides in a realm above, beyond, or outside that of mere matter.

The following ought to convince you that an image 500 pixels on a side with 1 bit pixels can be quite rich:

Union Station in Saint Louis - 1 bit depth image

My fellow Saint Louisians ought to recognize this scene. 1-bit images are generally quite useful, and have been used for decades in copying machines.

Even a 1 bit depth digital camera can produce an astounding number of different images. But suppose we add color — the number of possible images becomes even more staggering. Take my lowly Nikon D40 camera; it has about 6 million sensors, and if I shoot JPEG, I get 256 possible values per sensor. The camera has about 3 million green sensors, and 1.5 million sensors each of red and blue; full color is mathematically estimated for each pixel location.

So for each pixel location, we multiply by 256. The total number of images = 2566016000 which is approximately equal to a 1 followed by 14,487,972 zeroes. This is a mere pittance compared to the Seitz D3 digital scan back, which can capture 500 megapixel images with 48 bit color per pixel:
  • Total = (248)500000000 = 1 followed by 7,224,719,896 zeros.
I think it is safe to assume that like snowflakes, no two photographs are alike.

But this vast potentiality of photography ought not get us puffed up with pride in our creativity. Alas, the vast, overwhelming majority of these theoretically unique photographs look like this:

Random colored bits

Uniform random noise. A trillion monkeys, each generating a trillion random images per second for a trillion years, will likely never once produce anything that looks like a photograph. You could call this an ‘image’, but only in the most general terms. At best, images like this are only an exercise in conceptual art, and then only for the first photographer who does it. And I just did it.

[Note: to generate a random noise image such as this in Photoshop, be sure to start with a 50% gray image. If your starting point is a white image, then half of your final pixels will be white, and if you start with a black image, then half your pixels will still be black. Using the Filter->Noise->Add Noise... function, add 50% Uniform noise. Be sure to turn off ‘Monochromatic’. I find I get better results if I do this for each color channel independently.]

We call an image like this ‘random’ because it doesn't look like anything in particular. Perhaps a supremely intelligent being can look at this and be able to immediately discern the specific algorithm used by Photoshop to generate this. But we can't; we are finite creatures, and so the perception of noise is very much a human phenomenon. Perhaps we can imagine some figure in this noise, but that is tenuous at best. Actually, the entire concept of randomness is problematic: see my article here.

But we are good at perceiving some image if the noise isn't too great, especially if we are familiar with the subject:

Mystery image in noise

That is an image with a 1:10 signal/noise ratio. Can you guess the subject? What does your gut say?

I've never before attempted to estimate signal to noise ratios for digital images. Here, I've created a series of images with varying relative amounts of noise:

Signal to noise

Clearly, high ISO images, and heavily manipulated images, can have very high relative amounts of noise, much more than I suspected.

A 1:4 signal-to-noise ratio is roughly the limit I'm able to handle, while surprisingly a 1:1 ratio isn't horribly bad for some purposes. But please note that noise found in actual digital images is not uniform; rather it lurks most of all in the shadows, or rather, the signal-to-noise ratio is smallest there, for highlights have a large absolute amount of noise, even though it is small relative to the signal.

Suppose we are willing to accept a 4:1 signal-to-noise ratio, which means that 1/5 of our bits are just noise. For this image, which is 500 pixels on an edge, with 48 bits of color, that means that about 10 bits are noise.  We can calculate:
  • Number of acceptably noisy images equivalent to a reference image = (210)250000 = a 1 followed by 752,576 zeros.
2% total noise is virtually undetectable by the human eye, unless you look very closely. Also, slight adjustments or rotations of images are hard to see:

Four variations

For our 500 x 500 pixel full-color image, I estimate that there are approximately 4 trillion largely undetectable variations for every reference image. For larger images, the number goes up to values never heard of even in government economics.

OK, so I assert that of all theoretically possible images, the relative number which would be recognized as photographs is vanishingly small. Fortunately, since we are dealing with almost unimaginably large numbers, this isn't an issue. Roughly estimating the number of photographic images possible ought to be doable.

Patterns make an image. Either there are significantly large adjacent patches of pixels that are very similar, or there are similarities between pixels even though they are widely separated. Or, we can find a pattern that repeats on various scales.

Look at this image:


It is pretty obvious that we have symmetry of the left and right halves, as well as the top and bottom halves. Recall that the total number of possible variations of a 500x500 pixel, 1 pixel depth image is represented by a 3 followed by 75,257 zeros.  Because we have mirror symmetry, we basically repeat a 250x250 pixel image four times:

Total number of images:
  • = 2250 x 250 = 2 62500
  • = about 1 followed by 18184 zeros
This is an enormous figure, but is only 1/(1056433) fraction of the total number possible. We will be able to see the mirror reflection in nearly all the images produced: we would not perceive the symmetry if the image is all black or all white, or if the pattern is symmetric to begin with, but this is a very small proportion of the total.

With this 1 bit depth image, I find that I can only detect symmetry when I have greater than a 1:1 signal to noise ratio. We also should be able to detect other variations, such as changing the location and orientation of the axes of symmetry — all we have to do is ensure that our axis of symmetry is not too close to the boundary of the image.

The eye can detect global patterns such as seen above, and also local patterns, where there is some correlation between adjacent pixels:


Within a 500 x 500 pixel image, we can produce a total of 400 x 400 = 160,000 different images of a black 100x100 pixel square; our numbers go up if we can accept slight variations in the color and size of the square and its orientation. But if we are willing to accept some noise, as we see here, we can get zillons of possible images.

No two photographs are alike, and even if you take multiple shots with an ordinary camera under controlled conditions, there is no chance in your life that your resulting images will be exactly the same. This can be helpful: if you take multiple shots, you can blend them together to greatly reduce the amount of noise visible in the final image. Super-resolution techniques can also use multiple images to construct a higher-resolution final image, and can even remove diffraction artifacts.

The sharpness of lenses is a major limiting factor for producing unique photographs. The blurring found in some optics means that adjacent pixels are often strongly correlated, even if they are supposed to capture a high-contrast edge. Chromatic aberration and the mysterious purple fringing also lead to greater correlation between pixels, reducing the originality of photographs. Same goes with noise reduction software. Extreme, high quality bokeh, found in excellent portrait lenses, can reduce the out-of-focus background detail to a nearly uniform blur; this lack of detail leads us to focus our attention on the subject of the photograph. With extreme blurring and the use of the Photoshop Threshold tool, we can even reduce our images to match the 1-bit camera demonstrated at the top of this article.

Despite, or rather because of, the huge variation of possible photographs, it is relatively easy to detect unauthorized duplicates of images. Forensic analysts can detect duplicates with overwhelmingly high certainty, even if the original image was severely altered. Likewise, the use of the Clone tool in Photoshop — this copies one part of an image to another part — is easily detected if a large enough area is cloned, even if it is visually blended well, because these kinds of correlations within an image can not be practically attributed to chance. Never in a jillon years can we expect something like that to happen on its own.

If two photographers with two different cameras each take a photograph of the same scene, they will be different from each other in a huge number of minor (or even major) ways, so much so that we can be absolutely certain that the two images are in fact different. On the contrary, with a reasonably complex scene and good image quality, it seems that we ought to be highly confident that these two photographs were in fact photographed at the same time, and we also ought to be able to detect if the scene was artificially recreated at a later time, or was adjusted in Photoshop.

I think this discussion of seemingly impossibly large numbers tells us that photography, and digital art in general, is an incredibly rich and humanly inexhaustible medium.

[Click here for a discussion of names for long numbers. As it so happens, practically the only time these names for long numbers are actually used are in lists of names of long numbers. The word ‘zillion’, even though it is almost meaningless, is a good enough name for the quantities we are discussing here.]

No comments: