Calibration files

1 Proposal

1.1 Location of calibration files

Store calibration files in a subdirectory (axiombeta-calibration), located in a way similar to dcraw's badpixels file.

dcraw badpixels (from man page):

./.badpixels, ../.badpixels, ../../.badpixels, ...

beta calibration files

./axiombeta-calibration ../axiombeta-calibration ../../axiombeta-calibration

or maybe also ~/axiombeta-calibration ?

1.2 Types of calibration

We will use two types of calibration: sensor profiling (e.g. dark calibration, response curves, gain corrections, all that's needed to get a linear radiometric response) and color profiling (with argyll, for example), so we'll add two subdirectories:

 axiombeta-calibration/sensor-profiles
 axiombeta-calibration/color-profiles

In theory, these two things (sensor profiling and color profiling) should be independent (if they are not, we probably did a poor job with calibrating the sensor).

1.3 Sensor profiles

1.3.1 Directory structure

Inside that subdirectory, we can have a generic profile (one size fits all) and various profiles for different setting combinations (that can cover either just one particular situation, or can also be generic to some degree):

 axiombeta-calibration/sensor-profiles/generic/
 axiombeta-calibration/sensor-profiles/gain1_expo1-50/
 axiombeta-calibration/sensor-profiles/gain1_expo100-1000/
 axiombeta-calibration/sensor-profiles/blacksun8192_expo100-1000/
 axiombeta-calibration/sensor-profiles/gain1_expo100-1000_plr32-10ms-24-1ms/

That means, from all the settings that might affect the calibration, we can build an unique name from the settings that differ from the generic profile. A calibration might cover files at some particular setting value, or some range of values (detected from the supplied input files during calibration).

When selecting a profile, we may want to choose the one with the narrowest setting range that fits the settings of the file being rendered (that one should probably give the lowest calibration errors). If no match is found, the generic profile will be used.

Or, we may want to test the profile against some validation data, store the results, and pick the profile likely to minimize the calibration errors (some solid math skills required for this one).

1.3.2 Sensor calibration files

Inside one profile, we can have the following calibration files (all optional; if missing, that calibration step will be skipped):

For Black Calibration:

 beforegain.pgm : static offsets whose source is before the gain amplifiers
                 (that means, will be multiplied by gain, then subtracted)
 aftergain.pgm  : static offsets whose source is after the gain amplifiers
 darkcurrent.pgm: dynamic offsets dependent on many variables (exposure, temperature, gain),
                  whose scaling factor can be measured from hot pixels
                  (idea from [http://www.photonics.com/Article.aspx?AID=44298])

These 3 files allow computing a dark frame for every gain/expo/temperature combination (at least in theory), and should generalize fairly well to other settings (to be tested).

The computed dark frame would be:

beforegain * gain + aftergain + darkcurrent * darkcurrent_scaling

where darkcurrent_scaling can be measured from hot pixels.

Therefore, a simple dark frame (without any computation) can be saved in aftergain.pgm. If that's confusing, should we add darkframe.pgm as a synonym?

Other calibration files (e.g. Response Curves, Pattern Noise corrections):

 gainframe.pgm   : per-pixel gain variations
 clipframe.pgm   : clipping offsets
 responseXXXX.pgm: per-pixel response curve data point,
                   (raw sensor outputs that should give the corrected value XXXX)
                   (these might replace gain/clip frames completely, need to experiment)
 lut.spi1d       : overall or per-channel response curve (shaper)

1.4 Color profiles

Color profiles are stored under this subdirectory:

axiombeta-calibration/color-profiles/

These depend on other things, like ambient light and maybe lens used, so we could think about something like:

 axiombeta-calibration/color-profiles/D50/
 axiombeta-calibration/color-profiles/Tungsten/
 axiombeta-calibration/color-profiles/OsramDulux/                 # model of the light bulbs
 axiombeta-calibration/color-profiles/InTheCave/                  # or the location where the calibration was performed
 axiombeta-calibration/color-profiles/D50/EF50mm_f/1.8_II/        # from lens metadata

Those variables will be entered manually by the user (or chosen from a pickbox from a GUI), except maybe lens metadata, where it can be obtained from the lens itself (when the lens communication will be functional).

Color profile formats:

- ICC (as saved by argyll)
- config.ocio files?
- ...

2 Wish list

A way to identify which sensor or camera is used (by serial numbers) could be very helpful for those who own more than one camera, or more than one sensor.

In this case, the directory structure would be be:

 ~/axiombeta-calibration/cam1234/sensor5678/sensor-profiles
 ~/axiombeta-calibration/color-profiles

That's because sensor profiling is expected to account for the differences between sensors (that means, consistent output), so color profiles should be interchangeable between cameras/sensors (and should only account for lighting/lens differences).

Also, with a serial number, we no longer have to use tricks like having to search for the calibration subdirectory up 1/2/3/n levels from the current location.

Proposal from danieel:

~/.config/axiom/beta/calib/

also: "if the data is of low entropy, then just pack the multi-file structure into one .tgz"