Difference between revisions of "Calibration files"
| (One intermediate revision by one other user not shown) | |||
| Line 18: | Line 18: | ||
We will use two types of calibration: '''sensor profiling''' (e.g. dark calibration, response curves, gain corrections, all that's needed to get a linear radiometric response) and '''color profiling''' (with argyll, for example), so we'll add two subdirectories: | We will use two types of calibration: '''sensor profiling''' (e.g. dark calibration, response curves, gain corrections, all that's needed to get a linear radiometric response) and '''color profiling''' (with argyll, for example), so we'll add two subdirectories: | ||
<pre style="white-space: pre-wrap"> | |||
axiombeta-calibration/sensor-profiles | axiombeta-calibration/sensor-profiles | ||
axiombeta-calibration/color-profiles | axiombeta-calibration/color-profiles | ||
</pre> | |||
In theory, these two things (sensor profiling and color profiling) should be independent (if they are not, we probably did a poor job with calibrating the sensor). | In theory, these two things (sensor profiling and color profiling) should be independent (if they are not, we probably did a poor job with calibrating the sensor). | ||
| Line 29: | Line 29: | ||
Inside that subdirectory, we can have a '''generic profile''' (one size fits all) and various '''profiles for different setting combinations''' (that can cover either just one particular situation, or can also be generic to some degree): | Inside that subdirectory, we can have a '''generic profile''' (one size fits all) and various '''profiles for different setting combinations''' (that can cover either just one particular situation, or can also be generic to some degree): | ||
<pre style="white-space: pre-wrap"> | |||
axiombeta-calibration/sensor-profiles/generic/ | axiombeta-calibration/sensor-profiles/generic/ | ||
axiombeta-calibration/sensor-profiles/gain1_expo1-50/ | axiombeta-calibration/sensor-profiles/gain1_expo1-50/ | ||
| Line 35: | Line 35: | ||
axiombeta-calibration/sensor-profiles/blacksun8192_expo100-1000/ | axiombeta-calibration/sensor-profiles/blacksun8192_expo100-1000/ | ||
axiombeta-calibration/sensor-profiles/gain1_expo100-1000_plr32-10ms-24-1ms/ | axiombeta-calibration/sensor-profiles/gain1_expo100-1000_plr32-10ms-24-1ms/ | ||
</pre> | |||
That means, from all the settings that might affect the calibration, we can build an unique name from the settings that differ from the generic profile. A calibration might cover files at some particular setting value, or some range of values (detected from the supplied input files during calibration). | That means, from all the settings that might affect the calibration, we can build an unique name from the settings that differ from the generic profile. A calibration might cover files at some particular setting value, or some range of values (detected from the supplied input files during calibration). | ||
| Line 47: | Line 47: | ||
For [[Black Calibration]]: | For [[Black Calibration]]: | ||
<pre style="white-space: pre-wrap"> | |||
beforegain.pgm : static offsets whose source is before the gain amplifiers | beforegain.pgm : static offsets whose source is before the gain amplifiers | ||
(that means, will be multiplied by gain, then subtracted) | (that means, will be multiplied by gain, then subtracted) | ||
| Line 54: | Line 54: | ||
whose scaling factor can be measured from hot pixels | whose scaling factor can be measured from hot pixels | ||
(idea from [http://www.photonics.com/Article.aspx?AID=44298]) | (idea from [http://www.photonics.com/Article.aspx?AID=44298]) | ||
</pre> | |||
These 3 files allow computing a dark frame for every gain/expo/temperature combination (at least in theory), and should generalize fairly well to other settings (to be tested). | These 3 files allow computing a dark frame for every gain/expo/temperature combination (at least in theory), and should generalize fairly well to other settings (to be tested). | ||
| Line 65: | Line 65: | ||
Other calibration files (e.g. [[Response Curves]], [[Pattern Noise]] corrections): | Other calibration files (e.g. [[Response Curves]], [[Pattern Noise]] corrections): | ||
<pre style="white-space: pre-wrap"> | |||
gainframe.pgm : per-pixel gain variations | gainframe.pgm : per-pixel gain variations | ||
clipframe.pgm : clipping offsets | clipframe.pgm : clipping offsets | ||
| Line 71: | Line 72: | ||
(these might replace gain/clip frames completely, need to experiment) | (these might replace gain/clip frames completely, need to experiment) | ||
lut.spi1d : overall or per-channel response curve (shaper) | lut.spi1d : overall or per-channel response curve (shaper) | ||
</pre> | |||
==Color profiles== | ==Color profiles== | ||
| Line 79: | Line 80: | ||
These depend on other things, like ambient light and maybe lens used, so we could think about something like: | These depend on other things, like ambient light and maybe lens used, so we could think about something like: | ||
<pre style="white-space: pre-wrap"> | |||
axiombeta-calibration/color-profiles/D50/ | axiombeta-calibration/color-profiles/D50/ | ||
axiombeta-calibration/color-profiles/Tungsten/ | axiombeta-calibration/color-profiles/Tungsten/ | ||
| Line 85: | Line 86: | ||
axiombeta-calibration/color-profiles/InTheCave/ # or the location where the calibration was performed | axiombeta-calibration/color-profiles/InTheCave/ # or the location where the calibration was performed | ||
axiombeta-calibration/color-profiles/D50/EF50mm_f/1.8_II/ # from lens metadata | axiombeta-calibration/color-profiles/D50/EF50mm_f/1.8_II/ # from lens metadata | ||
</pre> | |||
Those variables will be entered manually by the user (or chosen from a pickbox from a GUI), except maybe lens metadata, where it can be obtained from the lens itself (when the lens communication will be functional). | Those variables will be entered manually by the user (or chosen from a pickbox from a GUI), except maybe lens metadata, where it can be obtained from the lens itself (when the lens communication will be functional). | ||
| Line 99: | Line 100: | ||
In this case, the directory structure would be be: | In this case, the directory structure would be be: | ||
<pre style="white-space: pre-wrap"> | |||
~/axiombeta-calibration/cam1234/sensor5678/sensor-profiles | ~/axiombeta-calibration/cam1234/sensor5678/sensor-profiles | ||
~/axiombeta-calibration/color-profiles | ~/axiombeta-calibration/color-profiles | ||
</pre> | |||
That's because sensor profiling is expected to account for the differences between sensors (that means, consistent output), so color profiles should be interchangeable between cameras/sensors (and should only account for lighting/lens differences). | That's because sensor profiling is expected to account for the differences between sensors (that means, consistent output), so color profiles should be interchangeable between cameras/sensors (and should only account for lighting/lens differences). | ||
Also, with a serial number, we no longer have to use tricks like having to search for the calibration subdirectory up 1/2/3/n levels from the current location. | Also, with a serial number, we no longer have to use tricks like having to search for the calibration subdirectory up 1/2/3/n levels from the current location. | ||
Proposal from danieel: | |||
~/.config/axiom/beta/calib/ | |||
also: "if the data is of low entropy, then just pack the multi-file structure into one .tgz" | |||
Latest revision as of 20:05, 2 March 2018
1 Proposal
1.1 Location of calibration files
Store calibration files in a subdirectory (axiombeta-calibration), located in a way similar to dcraw's badpixels file.
dcraw badpixels (from man page):
./.badpixels, ../.badpixels, ../../.badpixels, ...
beta calibration files
./axiombeta-calibration ../axiombeta-calibration ../../axiombeta-calibration
or maybe also ~/axiombeta-calibration ?
1.2 Types of calibration
We will use two types of calibration: sensor profiling (e.g. dark calibration, response curves, gain corrections, all that's needed to get a linear radiometric response) and color profiling (with argyll, for example), so we'll add two subdirectories:
axiombeta-calibration/sensor-profiles axiombeta-calibration/color-profiles
In theory, these two things (sensor profiling and color profiling) should be independent (if they are not, we probably did a poor job with calibrating the sensor).
1.3 Sensor profiles
1.3.1 Directory structure
Inside that subdirectory, we can have a generic profile (one size fits all) and various profiles for different setting combinations (that can cover either just one particular situation, or can also be generic to some degree):
axiombeta-calibration/sensor-profiles/generic/ axiombeta-calibration/sensor-profiles/gain1_expo1-50/ axiombeta-calibration/sensor-profiles/gain1_expo100-1000/ axiombeta-calibration/sensor-profiles/blacksun8192_expo100-1000/ axiombeta-calibration/sensor-profiles/gain1_expo100-1000_plr32-10ms-24-1ms/
That means, from all the settings that might affect the calibration, we can build an unique name from the settings that differ from the generic profile. A calibration might cover files at some particular setting value, or some range of values (detected from the supplied input files during calibration).
When selecting a profile, we may want to choose the one with the narrowest setting range that fits the settings of the file being rendered (that one should probably give the lowest calibration errors). If no match is found, the generic profile will be used.
Or, we may want to test the profile against some validation data, store the results, and pick the profile likely to minimize the calibration errors (some solid math skills required for this one).
1.3.2 Sensor calibration files
Inside one profile, we can have the following calibration files (all optional; if missing, that calibration step will be skipped):
For Black Calibration:
beforegain.pgm : static offsets whose source is before the gain amplifiers
(that means, will be multiplied by gain, then subtracted)
aftergain.pgm : static offsets whose source is after the gain amplifiers
darkcurrent.pgm: dynamic offsets dependent on many variables (exposure, temperature, gain),
whose scaling factor can be measured from hot pixels
(idea from [http://www.photonics.com/Article.aspx?AID=44298])
These 3 files allow computing a dark frame for every gain/expo/temperature combination (at least in theory), and should generalize fairly well to other settings (to be tested).
The computed dark frame would be:
beforegain * gain + aftergain + darkcurrent * darkcurrent_scaling
where darkcurrent_scaling can be measured from hot pixels.
Therefore, a simple dark frame (without any computation) can be saved in aftergain.pgm. If that's confusing, should we add darkframe.pgm as a synonym?
Other calibration files (e.g. Response Curves, Pattern Noise corrections):
gainframe.pgm : per-pixel gain variations
clipframe.pgm : clipping offsets
responseXXXX.pgm: per-pixel response curve data point,
(raw sensor outputs that should give the corrected value XXXX)
(these might replace gain/clip frames completely, need to experiment)
lut.spi1d : overall or per-channel response curve (shaper)
1.4 Color profiles
Color profiles are stored under this subdirectory:
axiombeta-calibration/color-profiles/
These depend on other things, like ambient light and maybe lens used, so we could think about something like:
axiombeta-calibration/color-profiles/D50/ axiombeta-calibration/color-profiles/Tungsten/ axiombeta-calibration/color-profiles/OsramDulux/ # model of the light bulbs axiombeta-calibration/color-profiles/InTheCave/ # or the location where the calibration was performed axiombeta-calibration/color-profiles/D50/EF50mm_f/1.8_II/ # from lens metadata
Those variables will be entered manually by the user (or chosen from a pickbox from a GUI), except maybe lens metadata, where it can be obtained from the lens itself (when the lens communication will be functional).
Color profile formats:
- ICC (as saved by argyll) - config.ocio files? - ...
2 Wish list
A way to identify which sensor or camera is used (by serial numbers) could be very helpful for those who own more than one camera, or more than one sensor.
In this case, the directory structure would be be:
~/axiombeta-calibration/cam1234/sensor5678/sensor-profiles ~/axiombeta-calibration/color-profiles
That's because sensor profiling is expected to account for the differences between sensors (that means, consistent output), so color profiles should be interchangeable between cameras/sensors (and should only account for lighting/lens differences).
Also, with a serial number, we no longer have to use tricks like having to search for the calibration subdirectory up 1/2/3/n levels from the current location.
Proposal from danieel:
~/.config/axiom/beta/calib/
also: "if the data is of low entropy, then just pack the multi-file structure into one .tgz"