EX: Restore color to a 120 year old photograph from the Prokudin-Gorskii collection

The collection features color photographic surveys of the Russian Empire made between ca. 1905 and 1915.

Prokudin-Gorskii's unusual triple-frame black-and-white negatives consist of three exposures made through blue, green, and red filters to produce photographs that could be printed or projected in color, usually for magic lantern slide shows. All 1,902 triple-frame glass negatives in the collection have been digitized, including about 150 that did not appear as prints in Prokudin-Gorskii's albums. All have been reproduced to show the full three frames. Single-frame views (usually the green-filtered center section) are also provided for ready enlargement of details.

The three images correspond to the blue, green, and red filters, respectively. Your program needs to extract the three color channel images, place them on top of each other, and align them so that they form a single color image.

Lab Overview (30 Points)

Resources

Start with the following notebook:

Colab Notebook Link

Tasks

  1. Exhaustive Search Alignment (10 Points)
  2. Pyramid Search (10 Points)
  3. Edge Detection - Implement using Canny or Sobel on the image of the Emir (10 Points)
  4. Extra Credit - Use SIFT or ORB to align channels on a landscape or building (5 Points)

Background

In the early 1900s, photographer Sergei Mikhailovich Prokudin-Gorskii created a method for recording color by taking three separate photos of a scene using red, green, and blue filters. These images can be aligned and combined to form a full-color image. In this lab, you will use computational techniques to align and stack these historical photos into full-color images.

Objective

The goal of this lab is to digitally render and align the grayscale images from Prokudin-Gorskii's glass plate collection to create a full-color image. You will perform this task using Python in Jupyter Notebooks hosted on Google Colab, utilizing various image processing techniques.

Key Steps

  1. Load the triptych grayscale images into Python.
  2. Stack the images into RGB channels.
  3. Optimize pixel offsets to align the color channels.
  4. Use similarity metrics to evaluate alignment.
  5. Produce the final aligned color images.

Lab Parts

Part 1: Image Loading and Stacking

Objective

Load images and stack them into color channels.

Equipment

  • Python 3
  • Numpy
  • Matplotlib
  • Prokudin-Gorskii triptych images

Procedure

  1. Load a single triptych image with three grayscale photos.
  2. Split the image into three equal vertical sections.
  3. Stack the sections into RGB channels to form an output image.
  4. Save the color-stacked image.

Part 2: Channel Alignment

Objective

  1. Align color channels by finding optimal offsets.

Equipment

  • Python 3
  • Numpy
  • Scipy
  • Matplotlib
  • Stacked color images from Part 1

Procedure

  1. Set one channel as the reference and fix it in place.
  2. Search for pixel offsets for the other channels within a range of -15 to 15 pixels in both the x and y directions.
  3. Evaluate alignment using a similarity metric, such as:
    1. Dot product
    2. Normalized cross-correlation (NCC)
  4. Apply the best alignment offsets.
  5. Save the final aligned color image.

Materials

  • Access to Google Colab with Python and Jupyter Notebooks
  • Prokudin-Gorskii's grayscale image collection from The Library of Congress
  • Python's OpenCV and NumPy libraries

Lab Procedure

  1. Open a Jupyter Notebook on Google Colab and load essential Python libraries such as OpenCV and NumPy.
  2. Browse the Prokudin-Gorskii collection from the provided URL and select an image composed of grayscale shots taken with Blue, Green, and Red filters.
  3. Load these grayscale images into the notebook as separate color channel images (Blue, Green, and Red).
  4. Implement edge detection on these images using the Canny edge detector and discuss the results.
  5. Write a Python script to align the color channel images, starting with aligning the Green channel to Blue, followed by aligning Red to Blue, using spatial displacement vectors (x, y).
  6. Implement a scoring metric, like L2 norm (SSD) or Normalized Cross-Correlation (NCC), to measure the accuracy of the alignment.
  7. For efficient handling of high-resolution images, employ a faster search procedure such as an image pyramid approach.
  8. Combine these components to create a function that outputs an aligned, full-color image from a three-channel input.
  9. Apply automatic white balance to the images and adjust contrast using OpenCV’s CLAHE (Contrast Limited Adaptive Histogram Equalization).
  10. Extra Credit: Experiment creatively with professional-level adjustments like cropping colored borders, adjusting color levels, and removing blemishes.

Your results should look like this.

Student examples

https://vimeo.com/manage/videos/1038702804