Difference between revisions of "Raw2dng"
| (8 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
raw2dng is a commandline tool to convert raw12 format images to digital negatives in the open standard DNG format | raw2dng is a commandline tool to convert raw12 format images to digital negatives in the open standard DNG format whilst simultaneously applying several corrections/compensations in the process. | ||
==Installation== | ==Installation== | ||
| Line 22: | Line 22: | ||
==Usage== | ==Usage== | ||
raw2dng image.raw12 | raw2dng image.raw12 | ||
==Reference== | |||
<pre style="white-space: pre-wrap"> | |||
DNG converter for Apertus .raw12 files | |||
Usage: | |||
raw2dng input.raw12 [input2.raw12] [options] | |||
cat input.raw12 | raw2dng output.dng [options] | |||
Flat field correction: | |||
- for each gain (N=1,2,3,4), you may use the following reference images: | |||
- darkframe-xN.pgm will be subtracted (data is x8 + 1024) | |||
- dcnuframe-xN.pgm will be multiplied by exposure and subtracted (x8192 + 8192) | |||
- gainframe-xN.pgm will be multiplied (1.0 = 16384) | |||
- clipframe-xN.pgm will be subtracted from highlights (x8) | |||
- reference images are 16-bit PGM, in the current directory | |||
- they are optional, but gain/clip frames require a dark frame | |||
- black ref columns will also be subtracted if you use a dark frame. | |||
Creating reference images: | |||
- dark frames: average as many as practical, for each gain setting, | |||
with exposures ranging from around 1ms to 50ms: | |||
raw2dng --calc-darkframe *-gainx1-*.raw12 | |||
- DCNU (dark current nonuniformity) frames: similar to dark frames, | |||
just take a lot more images to get a good fit (use 256 as a starting point): | |||
raw2dng --calc-dcnuframe *-gainx1-*.raw12 | |||
(note: the above will compute BOTH a dark frame and a dark current frame) | |||
- gain frames: average as many as practical, for each gain setting, | |||
with a normally exposed blank OOF wall as target, or without lens | |||
(currently used for pattern noise reduction only): | |||
raw2dng --calc-gainframe *-gainx1-*.raw12 | |||
- clip frames: average as many as practical, for each gain setting, | |||
with a REALLY overexposed blank out-of-focus wall as target: | |||
raw2dng --calc-clipframe *-gainx1-*.raw12 | |||
- Always compute these frames in the order listed here | |||
(dark/dcnu frames, then gain frames (optional), then clip frames (optional). | |||
General options: | |||
--black=%d : Set black level (default: 128) | |||
- negative values allowed | |||
--white=%d : Set white level (default: 4095) | |||
- if too high, you may get pink highlights | |||
- if too low, useful highlights may clip to white | |||
--width=%d : Set image width (default: 4096) | |||
--height=%d : Set image height | |||
- default: autodetect from file size | |||
- if input is stdin, default is 3072 | |||
--swap-lines : Swap lines in the raw data | |||
- workaround for an old Beta bug | |||
--hdmi : Assume the input is a memory dump | |||
used for HDMI recording experiments | |||
--pgm : Expect 16-bit PGM input from stdin | |||
--lut : Use a 1D LUT (lut-xN.spi1d, N=gain, OCIO-like) | |||
--totally-raw : Copy the raw data without any manipulation | |||
- metadata and pixel reordering are allowed. | |||
Pattern noise correction: | |||
--rnfilter=1 : FIR filter for row noise correction from black columns | |||
--rnfilter=2 : FIR filter for row noise correction from black columns | |||
and per-row median differences in green channels | |||
--fixrn : Fix row noise by image filtering (slow, guesswork) | |||
--fixpn : Fix row and column noise (SLOW, guesswork) | |||
--fixrnt : Temporal row noise fix (use with static backgrounds; recommended) | |||
--fixpnt : Temporal row/column noise fix (use with static backgrounds) | |||
--no-blackcol-rn : Disable row noise correction from black columns | |||
(they are still used to correct static offsets) | |||
--no-blackcol-ff : Disable fixed frequency correction in black columns | |||
Flat field correction: | |||
--dchp : Measure hot pixels to scale dark current frame | |||
--no-darkframe : Disable dark frame (if darkframe-xN.pgm is present) | |||
--no-dcnuframe : Disable dark current frame (if dcnuframe-xN.pgm is present) | |||
--no-gainframe : Disable gain frame (if gainframe-xN.pgm is present) | |||
--no-clipframe : Disable clip frame (if clipframe-xN.pgm is present) | |||
--no-blackcol : Disable black reference column subtraction | |||
- enabled by default if a dark frame is used | |||
- reduces row noise and black level variations | |||
--calc-darkframe : Average a dark frame from all input files | |||
--calc-dcnuframe : Fit a dark frame (constant offset) and a dark current frame | |||
(exposure-dependent offset) from files with different exposures | |||
(starting point: 256 frames with exposures from 1 to 50 ms) | |||
--calc-gainframe : Average a gain frame (aka flat field frame) | |||
--calc-clipframe : Average a clip (overexposed) frame | |||
--check-darkframe : Check image quality indicators on a dark frame | |||
Debug options: | |||
--dump-regs : Dump sensor registers from metadata block (no output DNG) | |||
--fixpn-dbg-denoised: Pattern noise: show denoised image | |||
--fixpn-dbg-noise : Pattern noise: show noise image (original - denoised) | |||
--fixpn-dbg-mask : Pattern noise: show masked areas (edges and highlights) | |||
--fixpn-dbg-col : Pattern noise: debug columns (default: rows) | |||
--export-rownoise : Export row noise data to octave (rownoise_data.m) | |||
</pre> | |||
==Further Notes== | ==Further Notes== | ||
| Line 28: | Line 123: | ||
[[File:Screen_Shot_2018-06-02_at_11.47.43.png | 300px]] | [[File:Screen_Shot_2018-06-02_at_11.47.43.png | 300px]] | ||
It is recommended to install/compile the latest version of [https://github.com/apertus-open-source-cinema/beta- | It is recommended to install/compile the latest version of [https://github.com/apertus-open-source-cinema/axiom-beta-firmware/tree/master/software/sensor_tools/snap cmv_snap3] on your AXIOM Beta to fix the issue for future captured images. | ||
To fix raw12 images already recorded with even/odd lines swapped use the --swap-lines parameter with raw2dng: | To fix raw12 images already recorded with even/odd lines swapped use the --swap-lines parameter with raw2dng: | ||
Latest revision as of 15:06, 14 May 2020
raw2dng is a commandline tool to convert raw12 format images to digital negatives in the open standard DNG format whilst simultaneously applying several corrections/compensations in the process.
1 Installation
Compiling raw2dng on a 64bit system requires the gcc-multilib package.
Ubuntu:
sudo apt-get install gcc-multilib
Then acquire the source from https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng and run "make"
openSUSE:
sudo zypper install gcc-32bit libgomp1-32bit
Then acquire the source from https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng and run "make"
AXIOM Beta:
- acquire the source from https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng
- copy files to AXIOM Beta
- remove "-m32" from Makefile
- run "make" inside camera
2 Usage
raw2dng image.raw12
3 Reference
DNG converter for Apertus .raw12 files
Usage:
raw2dng input.raw12 [input2.raw12] [options]
cat input.raw12 | raw2dng output.dng [options]
Flat field correction:
- for each gain (N=1,2,3,4), you may use the following reference images:
- darkframe-xN.pgm will be subtracted (data is x8 + 1024)
- dcnuframe-xN.pgm will be multiplied by exposure and subtracted (x8192 + 8192)
- gainframe-xN.pgm will be multiplied (1.0 = 16384)
- clipframe-xN.pgm will be subtracted from highlights (x8)
- reference images are 16-bit PGM, in the current directory
- they are optional, but gain/clip frames require a dark frame
- black ref columns will also be subtracted if you use a dark frame.
Creating reference images:
- dark frames: average as many as practical, for each gain setting,
with exposures ranging from around 1ms to 50ms:
raw2dng --calc-darkframe *-gainx1-*.raw12
- DCNU (dark current nonuniformity) frames: similar to dark frames,
just take a lot more images to get a good fit (use 256 as a starting point):
raw2dng --calc-dcnuframe *-gainx1-*.raw12
(note: the above will compute BOTH a dark frame and a dark current frame)
- gain frames: average as many as practical, for each gain setting,
with a normally exposed blank OOF wall as target, or without lens
(currently used for pattern noise reduction only):
raw2dng --calc-gainframe *-gainx1-*.raw12
- clip frames: average as many as practical, for each gain setting,
with a REALLY overexposed blank out-of-focus wall as target:
raw2dng --calc-clipframe *-gainx1-*.raw12
- Always compute these frames in the order listed here
(dark/dcnu frames, then gain frames (optional), then clip frames (optional).
General options:
--black=%d : Set black level (default: 128)
- negative values allowed
--white=%d : Set white level (default: 4095)
- if too high, you may get pink highlights
- if too low, useful highlights may clip to white
--width=%d : Set image width (default: 4096)
--height=%d : Set image height
- default: autodetect from file size
- if input is stdin, default is 3072
--swap-lines : Swap lines in the raw data
- workaround for an old Beta bug
--hdmi : Assume the input is a memory dump
used for HDMI recording experiments
--pgm : Expect 16-bit PGM input from stdin
--lut : Use a 1D LUT (lut-xN.spi1d, N=gain, OCIO-like)
--totally-raw : Copy the raw data without any manipulation
- metadata and pixel reordering are allowed.
Pattern noise correction:
--rnfilter=1 : FIR filter for row noise correction from black columns
--rnfilter=2 : FIR filter for row noise correction from black columns
and per-row median differences in green channels
--fixrn : Fix row noise by image filtering (slow, guesswork)
--fixpn : Fix row and column noise (SLOW, guesswork)
--fixrnt : Temporal row noise fix (use with static backgrounds; recommended)
--fixpnt : Temporal row/column noise fix (use with static backgrounds)
--no-blackcol-rn : Disable row noise correction from black columns
(they are still used to correct static offsets)
--no-blackcol-ff : Disable fixed frequency correction in black columns
Flat field correction:
--dchp : Measure hot pixels to scale dark current frame
--no-darkframe : Disable dark frame (if darkframe-xN.pgm is present)
--no-dcnuframe : Disable dark current frame (if dcnuframe-xN.pgm is present)
--no-gainframe : Disable gain frame (if gainframe-xN.pgm is present)
--no-clipframe : Disable clip frame (if clipframe-xN.pgm is present)
--no-blackcol : Disable black reference column subtraction
- enabled by default if a dark frame is used
- reduces row noise and black level variations
--calc-darkframe : Average a dark frame from all input files
--calc-dcnuframe : Fit a dark frame (constant offset) and a dark current frame
(exposure-dependent offset) from files with different exposures
(starting point: 256 frames with exposures from 1 to 50 ms)
--calc-gainframe : Average a gain frame (aka flat field frame)
--calc-clipframe : Average a clip (overexposed) frame
--check-darkframe : Check image quality indicators on a dark frame
Debug options:
--dump-regs : Dump sensor registers from metadata block (no output DNG)
--fixpn-dbg-denoised: Pattern noise: show denoised image
--fixpn-dbg-noise : Pattern noise: show noise image (original - denoised)
--fixpn-dbg-mask : Pattern noise: show masked areas (edges and highlights)
--fixpn-dbg-col : Pattern noise: debug columns (default: rows)
--export-rownoise : Export row noise data to octave (rownoise_data.m)
4 Further Notes
Older versions of cmv_snap3 (current AXIOM Beta firmware still contains this old version) had a bug and swapped the order of even and odd lines in written raw12 files. As a result converting these raw12 images to DNG created wrong colors (all pink) as seen in the following picture:
It is recommended to install/compile the latest version of cmv_snap3 on your AXIOM Beta to fix the issue for future captured images.
To fix raw12 images already recorded with even/odd lines swapped use the --swap-lines parameter with raw2dng:
raw2dng image.raw12 --swap-lines
Details about the RAW12 format.