Colorizing Black-and-White Photos: What AI Can and Can't Do

Seeing a great-grandparent’s portrait in color can be startling. Suddenly the photo feels less like history and more like a person. AI colorization tools have made this possible in seconds, and the results can be beautiful.

They can also be quietly wrong. Before you colorize a family archive or share a colorized historical photo, it helps to understand what the technology actually does, where it’s reliable, and where it’s guessing.

How AI colorization works

A black-and-white photo records brightness only. Each pixel has a single value from black to white. The colors that were in front of the camera are gone.

AI colorization models are trained on huge numbers of color photos that have been converted to grayscale. During training, the model learns to predict the original colors from the gray version. Over time it picks up strong associations: skies tend to be blue, grass tends to be green, skin falls within a range of tones, tree bark is brown.

When you give it your photo, it doesn’t recover the colors. It predicts plausible ones based on what similar-looking things usually are in its training data.

That distinction matters more than anything else in this article.

The core problem: many colors, one shade of gray

Very different colors can produce the same gray. A medium red, a medium green and a medium blue can all look nearly identical in a black-and-white photo if their brightness is similar.

That means:

• A dress could have been red, green, navy or brown. The photo can’t tell you.
• A car, a front door, a uniform’s trim or a flower bed are all genuine unknowns.
• The model will choose something, usually the most statistically common option, and present it with the same confidence as the sky.

There’s a further twist with older photos. Many early photographic materials were orthochromatic: sensitive to blue and green light but not to red. On those materials, red things recorded much darker than they looked, so red lips and ruddy cheeks can appear very dark, and a pale red object might look almost black. Panchromatic film, sensitive to all visible colors, became common in the early twentieth century, but many studio photos and plates before then were orthochromatic. A model that doesn’t account for this can get skin tones and reds badly wrong.

What AI colorization does well

• Skies, water, foliage and landscapes, where the likely colors are fairly predictable.
• Skin tones in general terms, especially with good lighting and a well-exposed print.
• Making a photo emotionally accessible, for a family gathering, a memorial or a social post.
• Speed. A manual colorization by a skilled artist can take many hours; AI does a first pass instantly.

Where it commonly goes wrong

• Clothing and fabrics: often colored brown, beige or muted blue by default, because those are safe statistical guesses.
• Uniforms, flags, insignia and vehicles: these have specific, documented colors that a general model may not know.
• Skin tone: models can lighten or darken skin based on biased training data or the photo’s exposure, which is both inaccurate and potentially offensive.
• Color bleeding: color spilling across edges, such as a blue sky tinting a person’s hair.
• Washed-out, sepia-like results: when uncertain, many models drift toward a brownish average.
• Damaged areas: dust, scratches and stains confuse the model and pick up strange colors.

That last point is easy to fix: clean and repair the photo before you colorize it. Remove dust, scratches and stains first, as covered in our restoration guide. A clean source gives a much cleaner colorization.

How to colorize more accurately

Gather evidence first

Look for real clues about the original colors:

• Family memory: ask relatives about eye color, hair color, a favorite coat, the color of the house.
• Written notes on the back of prints or in letters and diaries.
• Color photos from the same period of the same people or places.
• Reference material for uniforms, vehicles, buildings and products, which often had documented colors.

Use AI as a first draft

Let the AI produce a base, then correct specific regions by hand in an editor. Paint corrected color on a layer set to a Color blend mode, which changes hue and saturation while keeping the original brightness. This keeps every detail of the original photo intact.

Keep saturation modest

Old color photographs and film stocks rarely had the punchy saturation of modern phone cameras. Lower saturation often looks more believable and makes errors less glaring.

Fix skin tones carefully

Sample tones from color photos of the same family where possible. Check under different lighting conditions in the image (shadowed side versus lit side) so the face doesn’t look painted on.

The ethics of colorizing

Colorizing is an interpretation, not a restoration. That’s fine, as long as you’re honest about it.

Label colorized images

When you share a colorized photo, say so: “Colorized from a black-and-white original.” For historical and public images, also say that colors are approximate. Viewers, especially years later, may otherwise assume the colors are documentary.

Always keep the original

Store the untouched black-and-white scan alongside the colorized version, with clear file names:

1931_farm_family_scan.tif
1931_farm_family_colorized.jpg

The black-and-white file is the historical record. The colorized one is a derivative.

Be careful with sensitive history

Photos of war, atrocities, enslaved people, or communities affected by colonialism carry particular weight. Colorizing them can bring viewers closer to the people pictured, but it can also distort skin tones, clothing and context, or turn suffering into spectacle. Researchers and archives often prefer to present such images as they were made. If you colorize them, do so with research, care and clear labeling.

Consider the people pictured

For family photos, ask relatives how they feel about it. Some people love seeing a grandparent in color; others find it unsettling or prefer the original.

Should you colorize at all?

Colorizing is worth doing when:

• it’s for personal enjoyment or a family occasion, and everyone understands it’s an interpretation;
• you’ve researched the details that matter and corrected the AI’s guesses;
• you’ll keep and share the original too.

It’s worth skipping, or at least treating with caution, when the photo is evidence of something, will be used in research or education without context, or when the colors you’d invent could mislead.

Where DirtAI stands

DirtAI is focused first on cleanup and repair: removing dust, scratches, stains and unwanted objects so the photo you already have looks its best. That’s the step that makes any later colorization better, and it doesn’t involve guessing at colors that were never recorded. Whatever we offer beyond that, we’ll be clear about which parts of an image are reconstructed.

If you’re preparing a family archive, start with a good scan, clean it up, and keep the original black-and-white master safe. Then colorize, thoughtfully, if you want to.