Camera Processing Pipeline VIS+NIR

 Version history:

  • V1.20: 21.08.2012: Dominic Rüfenacht:
    – Corrected problem with highlights which sometimes appeared pink;
    – Rearranged some parts in the code.
  • V1.12: 19.07.2012: Dominic Rüfenacht:
    – Added compiled binaries of DCRaw for both Windows and Mac;
    – Added option ‘None’ as white-balancing “mode”;
    – Added a helper function which is not part of all Matlab distributions.
  • V1.11: 28.06.2012: Dominic Rüfenacht:
    – Made compilation of Amaze.cpp automatical;
    – Removed unnecessary command line outputs;
    – Generalized file path for loading images.
  • V1.1: 25.06.2012: Dominic Rüfenacht:
    – Added premultipliers in function “addNewCameraGUI.m”;
    – Recomputed cameras.mat to take the aforementioned change into account;
    – Excluded black/whitepoint correction for the “Colorimetric reproduction” tone mapping case.
  • V1.0: 18.05.2012: Dominic Rüfenacht, “Initial release of the camera processing pipeline”. Please report any bugs you find.

Purpose:
This tool allows developing RAW files that contain visible and/or NIR information. The main advantages of this tool over commercial RAW development software are:

  • Takes into account the measured camera sensitivities;
  • Is cabaple of handling both visible and/or NIR images;
  • New features can easily be added into the workflow.

The figure below shows the building blocks and the workflow of the camera pipeline. Green arrows indicate the workflow for visibl images, and red arrows for NIR images, where the main difference is that NIR images are not converted to the sRGB color space.

Figure 1


Installation:
You can get the latest version of the tool here. Unzip it in the directory of your preference.

This tool was written in Matlab, so you need a recent version of Matlab. The only tool you need to install is DCRaw. The zip-file above contains the latest binaries for both Windows and Mac, so everything should work out of the box. If it doesn’t work, you find the official site of the tool here. Maybe you also want to try the latest version of DCRaw and directly download compiled binaries for Windows and Mac.

The tool comes with measured camera sensitivities for the two Canon EOS 5D Mark II we have in our lab (one original and one with hot mirror removed), with different filters. You find them in the folder ./txt/. If you are using another camera, you will first have to measure the camera sensitivities as described on this site.

If you find any error or have a request for an additional feature, please either correct/add it yourself and post a new entry in the version history, or contact the author.


Tutorial:
This tutorial is organized in two main parts. The first part explains how the it can be used, and the second one shows how one can add extensions/features.

Part 1: Processing RAW files
Start the tool by launching the file workflow_gui.m. The following window appears:

Input related parameters:
Start by choosing the input type of the pipeline. The actual version supports you to use either single RAW (visible or NIR) or two RAW (visible and NIR) input. Single RAW of a visible and NIR sensor will be implemented as soon as such a sensor is ready.

Next you need to specify the camera and filter type used when the RAW images were captured. This loads the measured camera sensitivities. If your camera/filter setting is not in the list, you can add it by clicking on the button Add/Remove Camera. This will open a new window called AddNewCameraGUI. This menu is explained in more detail in the section Adding a new camera. Don’t forget to update the database by clicking on the button Update list.

The next step is to select the images you want to convert. You can do this by clicking on Select Image(s) button. You can add multiple images. If you have two RAW input, the visible and NIR images need to have the same filenames. You can rename the images by clicking on the button Rename image(s). This tool is described in more detail in the section Renaming images.

Output related parameters:
This part allows you to control the different blocks of the camera processing pipeline. You can choose the processing order of white balancing and demosaicing, and then choose the demosaicing method (Amaze is recommended), white-balancing and tone mapping method.

You can also apply a global gamma correction as well as black/whitepoint correction. These global parameters can either be set at the beginning for all images by selecting the option Same for all images, or for any image individually with live-preview by selecting Variable.

The last part let’s you choose the directory where you want the images to be saved, as well as the output format.

If you plan to batch-process big collections of RAW images, you might want to consider to uncheck the option Show results, which will prevent the tool from showing intermediate results. This can significantly speed up the whole process, as Matlab can be quite slow at displaying images, especially for large images.

A click on start processing starts developing the RAW files according to your parameters.

Adding a new camera
This tool allows you to add a new camera to the database that is used in the camera processing pipeline.In particular, it computes the cRGB2XYZ converison matrix, which is a 3×3 matrix which converts from the camera color space to XYZ.

This window shows you all the camera/filter pairs that are actually in the database. You can remove entries from the database by selecting the respective entry and then clicking on the button Delete selected entry. Note that you cannot undo this action.

If you want to add a camera, you need to enter the camera model, make and the filter type used. You then need to locate the text-file which is the output from the camSPECS tool and which contains the spectral measurements of the camera/filter combination. Details about how to measure the camera sensitivities can be found here.
If you took several measurements, you have to put them all in the same directory, and rename them according to the following scheme: your_filename_X.txt, where X = 1, 2, 3…
You then need to specify the CFA arrangement, which you usually find in the header of the RAW file.
The optimization method has two choices, least squares (LS) or white-point preserving least squares (WPPLS). It is recommended that you use WPPLS. Details on that method can be found in the paper:

Graham D. Finlayson and Mark S. Drew, White-Point Preserving Color Correction, 1997.

You have the choice to plot the full spectrum from 380nm-1100nm, or only plotting the visible (380nm-750nm) or the NIR spectrum (750nm-1100nm). Note that this has no influence on the cRGB2XYZ matrix.

Renaming images
This tool renames images according to the expected scheme used by the camera processing pipeline.

All you need to do is locate the folders where you have the visible and NIR RAW images, and select the file extension. All images in that folder will then be renamed and put into the specified output directory output_dir, according to the following scheme:

Visible images: output_dir/visible/filename_X.extension
NIR images:      output_dir/nir/filename_X.extension

Part 2: Adding new functionalities
This tool was conceived in a way that should make it easy and intuitive to add new functionalities, such as (better) demosaicing, white-balancing and tone mapping methods. All you need to do is to open the respective function, called apply_demosaicing.m, apply_white_balancing.m, and apply_tonemapping.m, and add a new case statement to this file where you then code you algorithm. Once this is done, you only need to add the same string that you used as case statement and add it in the main file workflow_gui.m  to the respective modes that are defined around at line 50. Look at how it is done for the existing methods and it should not be a problem.

Important: If you add a new functionality, please share it with the rest of the lab by uploading the new function, and adding the new functionality to the Version history, along with your name and date of last modification.