(Further Notes)
(Examples)
 
(8 intermediate revisions by 3 users not shown)
Line 1: Line 1:
raw2dng is a commandline tool to convert raw12 format images to digital negatives in the open standard DNG format and applies several corrections/compensations in the process.
+
raw2dng is a commandline tool to convert raw12 format images to digital negatives in the open standard DNG format whilst simultaneously applying several corrections/compensations in the process.
  
 
==Installation==
 
==Installation==
Line 22: Line 22:
 
==Usage==
 
==Usage==
 
  raw2dng image.raw12
 
  raw2dng image.raw12
 +
 +
==Reference==
 +
<pre style="white-space: pre-wrap">
 +
DNG converter for Apertus .raw12 files
 +
 +
Usage:
 +
  raw2dng input.raw12 [input2.raw12] [options]
 +
  cat input.raw12 | raw2dng output.dng [options]
 +
 +
Flat field correction:
 +
- for each gain (N=1,2,3,4), you may use the following reference images:
 +
- darkframe-xN.pgm will be subtracted (data is x8 + 1024)
 +
- dcnuframe-xN.pgm will be multiplied by exposure and subtracted (x8192 + 8192)
 +
- gainframe-xN.pgm will be multiplied (1.0 = 16384)
 +
- clipframe-xN.pgm will be subtracted from highlights (x8)
 +
- reference images are 16-bit PGM, in the current directory
 +
- they are optional, but gain/clip frames require a dark frame
 +
- black ref columns will also be subtracted if you use a dark frame.
 +
 +
Creating reference images:
 +
- dark frames: average as many as practical, for each gain setting,
 +
  with exposures ranging from around 1ms to 50ms:
 +
        raw2dng --calc-darkframe *-gainx1-*.raw12
 +
- DCNU (dark current nonuniformity) frames: similar to dark frames,
 +
  just take a lot more images to get a good fit (use 256 as a starting point):
 +
        raw2dng --calc-dcnuframe *-gainx1-*.raw12
 +
  (note: the above will compute BOTH a dark frame and a dark current frame)
 +
- gain frames: average as many as practical, for each gain setting,
 +
  with a normally exposed blank OOF wall as target, or without lens
 +
  (currently used for pattern noise reduction only):
 +
        raw2dng --calc-gainframe *-gainx1-*.raw12
 +
- clip frames: average as many as practical, for each gain setting,
 +
  with a REALLY overexposed blank out-of-focus wall as target:
 +
        raw2dng --calc-clipframe *-gainx1-*.raw12
 +
- Always compute these frames in the order listed here
 +
  (dark/dcnu frames, then gain frames (optional), then clip frames (optional).
 +
 +
General options:
 +
--black=%d          : Set black level (default: 128)
 +
                      - negative values allowed
 +
--white=%d          : Set white level (default: 4095)
 +
                      - if too high, you may get pink highlights
 +
                      - if too low, useful highlights may clip to white
 +
--width=%d          : Set image width (default: 4096)
 +
--height=%d        : Set image height
 +
                      - default: autodetect from file size
 +
                      - if input is stdin, default is 3072
 +
--swap-lines        : Swap lines in the raw data
 +
                      - workaround for an old Beta bug
 +
--hdmi              : Assume the input is a memory dump
 +
                      used for HDMI recording experiments
 +
--pgm              : Expect 16-bit PGM input from stdin
 +
 +
--lut              : Use a 1D LUT (lut-xN.spi1d, N=gain, OCIO-like)
 +
 +
--totally-raw      : Copy the raw data without any manipulation
 +
                      - metadata and pixel reordering are allowed.
 +
 +
Pattern noise correction:
 +
--rnfilter=1        : FIR filter for row noise correction from black columns
 +
--rnfilter=2        : FIR filter for row noise correction from black columns
 +
                      and per-row median differences in green channels
 +
--fixrn            : Fix row noise by image filtering (slow, guesswork)
 +
--fixpn            : Fix row and column noise (SLOW, guesswork)
 +
--fixrnt            : Temporal row noise fix (use with static backgrounds; recommended)
 +
--fixpnt            : Temporal row/column noise fix (use with static backgrounds)
 +
--no-blackcol-rn    : Disable row noise correction from black columns
 +
                      (they are still used to correct static offsets)
 +
--no-blackcol-ff    : Disable fixed frequency correction in black columns
 +
 +
Flat field correction:
 +
--dchp              : Measure hot pixels to scale dark current frame
 +
--no-darkframe      : Disable dark frame (if darkframe-xN.pgm is present)
 +
--no-dcnuframe      : Disable dark current frame (if dcnuframe-xN.pgm is present)
 +
--no-gainframe      : Disable gain frame (if gainframe-xN.pgm is present)
 +
--no-clipframe      : Disable clip frame (if clipframe-xN.pgm is present)
 +
--no-blackcol      : Disable black reference column subtraction
 +
                      - enabled by default if a dark frame is used
 +
                      - reduces row noise and black level variations
 +
--calc-darkframe    : Average a dark frame from all input files
 +
--calc-dcnuframe    : Fit a dark frame (constant offset) and a dark current frame
 +
                      (exposure-dependent offset) from files with different exposures
 +
                      (starting point: 256 frames with exposures from 1 to 50 ms)
 +
--calc-gainframe    : Average a gain frame (aka flat field frame)
 +
--calc-clipframe    : Average a clip (overexposed) frame
 +
--check-darkframe  : Check image quality indicators on a dark frame
 +
 +
Debug options:
 +
--dump-regs        : Dump sensor registers from metadata block (no output DNG)
 +
--fixpn-dbg-denoised: Pattern noise: show denoised image
 +
--fixpn-dbg-noise  : Pattern noise: show noise image (original - denoised)
 +
--fixpn-dbg-mask    : Pattern noise: show masked areas (edges and highlights)
 +
--fixpn-dbg-col    : Pattern noise: debug columns (default: rows)
 +
--export-rownoise  : Export row noise data to octave (rownoise_data.m)
 +
</pre>
  
 
==Further Notes==
 
==Further Notes==
Line 28: Line 123:
 
[[File:Screen_Shot_2018-06-02_at_11.47.43.png | 300px]]
 
[[File:Screen_Shot_2018-06-02_at_11.47.43.png | 300px]]
  
It is recommended to install/compile the latest version of [https://github.com/apertus-open-source-cinema/beta-software/tree/master/software/cmv_tools/cmv_snap3 cmv_snap3] on your AXIOM Beta to fix the issue for future captured images.
+
It is recommended to install/compile the latest version of [https://github.com/apertus-open-source-cinema/axiom-beta-firmware/tree/master/software/sensor_tools/snap cmv_snap3] on your AXIOM Beta to fix the issue for future captured images.
  
 
To fix raw12 images already recorded with even/odd lines swapped use the --swap-lines parameter with raw2dng:
 
To fix raw12 images already recorded with even/odd lines swapped use the --swap-lines parameter with raw2dng:

Latest revision as of 15:06, 14 May 2020

raw2dng is a commandline tool to convert raw12 format images to digital negatives in the open standard DNG format whilst simultaneously applying several corrections/compensations in the process.

1 Installation

Compiling raw2dng on a 64bit system requires the gcc-multilib package.

Ubuntu:

sudo apt-get install gcc-multilib

Then acquire the source from https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng and run "make"

openSUSE:

sudo zypper install gcc-32bit libgomp1-32bit

Then acquire the source from https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng and run "make"

AXIOM Beta:

  1. acquire the source from https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng
  2. copy files to AXIOM Beta
  3. remove "-m32" from Makefile
  4. run "make" inside camera

2 Usage

raw2dng image.raw12

3 Reference

DNG converter for Apertus .raw12 files

Usage:
  raw2dng input.raw12 [input2.raw12] [options]
  cat input.raw12 | raw2dng output.dng [options]

Flat field correction:
 - for each gain (N=1,2,3,4), you may use the following reference images:
 - darkframe-xN.pgm will be subtracted (data is x8 + 1024)
 - dcnuframe-xN.pgm will be multiplied by exposure and subtracted (x8192 + 8192)
 - gainframe-xN.pgm will be multiplied (1.0 = 16384)
 - clipframe-xN.pgm will be subtracted from highlights (x8)
 - reference images are 16-bit PGM, in the current directory
 - they are optional, but gain/clip frames require a dark frame
 - black ref columns will also be subtracted if you use a dark frame.

Creating reference images:
 - dark frames: average as many as practical, for each gain setting,
   with exposures ranging from around 1ms to 50ms:
        raw2dng --calc-darkframe *-gainx1-*.raw12
 - DCNU (dark current nonuniformity) frames: similar to dark frames,
   just take a lot more images to get a good fit (use 256 as a starting point):
        raw2dng --calc-dcnuframe *-gainx1-*.raw12
   (note: the above will compute BOTH a dark frame and a dark current frame)
 - gain frames: average as many as practical, for each gain setting,
   with a normally exposed blank OOF wall as target, or without lens
   (currently used for pattern noise reduction only):
        raw2dng --calc-gainframe *-gainx1-*.raw12
 - clip frames: average as many as practical, for each gain setting,
   with a REALLY overexposed blank out-of-focus wall as target:
        raw2dng --calc-clipframe *-gainx1-*.raw12
 - Always compute these frames in the order listed here
   (dark/dcnu frames, then gain frames (optional), then clip frames (optional).

General options:
--black=%d          : Set black level (default: 128)
                      - negative values allowed
--white=%d          : Set white level (default: 4095)
                      - if too high, you may get pink highlights
                      - if too low, useful highlights may clip to white
--width=%d          : Set image width (default: 4096)
--height=%d         : Set image height
                      - default: autodetect from file size
                      - if input is stdin, default is 3072
--swap-lines        : Swap lines in the raw data
                      - workaround for an old Beta bug
--hdmi              : Assume the input is a memory dump
                      used for HDMI recording experiments
--pgm               : Expect 16-bit PGM input from stdin

--lut               : Use a 1D LUT (lut-xN.spi1d, N=gain, OCIO-like)

--totally-raw       : Copy the raw data without any manipulation
                      - metadata and pixel reordering are allowed.

Pattern noise correction:
--rnfilter=1        : FIR filter for row noise correction from black columns
--rnfilter=2        : FIR filter for row noise correction from black columns
                      and per-row median differences in green channels
--fixrn             : Fix row noise by image filtering (slow, guesswork)
--fixpn             : Fix row and column noise (SLOW, guesswork)
--fixrnt            : Temporal row noise fix (use with static backgrounds; recommended)
--fixpnt            : Temporal row/column noise fix (use with static backgrounds)
--no-blackcol-rn    : Disable row noise correction from black columns
                      (they are still used to correct static offsets)
--no-blackcol-ff    : Disable fixed frequency correction in black columns

Flat field correction:
--dchp              : Measure hot pixels to scale dark current frame
--no-darkframe      : Disable dark frame (if darkframe-xN.pgm is present)
--no-dcnuframe      : Disable dark current frame (if dcnuframe-xN.pgm is present)
--no-gainframe      : Disable gain frame (if gainframe-xN.pgm is present)
--no-clipframe      : Disable clip frame (if clipframe-xN.pgm is present)
--no-blackcol       : Disable black reference column subtraction
                      - enabled by default if a dark frame is used
                      - reduces row noise and black level variations
--calc-darkframe    : Average a dark frame from all input files
--calc-dcnuframe    : Fit a dark frame (constant offset) and a dark current frame
                      (exposure-dependent offset) from files with different exposures
                      (starting point: 256 frames with exposures from 1 to 50 ms)
--calc-gainframe    : Average a gain frame (aka flat field frame)
--calc-clipframe    : Average a clip (overexposed) frame
--check-darkframe   : Check image quality indicators on a dark frame

Debug options:
--dump-regs         : Dump sensor registers from metadata block (no output DNG)
--fixpn-dbg-denoised: Pattern noise: show denoised image
--fixpn-dbg-noise   : Pattern noise: show noise image (original - denoised)
--fixpn-dbg-mask    : Pattern noise: show masked areas (edges and highlights)
--fixpn-dbg-col     : Pattern noise: debug columns (default: rows)
--export-rownoise   : Export row noise data to octave (rownoise_data.m)

4 Further Notes

Older versions of cmv_snap3 (current AXIOM Beta firmware still contains this old version) had a bug and swapped the order of even and odd lines in written raw12 files. As a result converting these raw12 images to DNG created wrong colors (all pink) as seen in the following picture:

Screen Shot 2018-06-02 at 11.47.43.png

It is recommended to install/compile the latest version of cmv_snap3 on your AXIOM Beta to fix the issue for future captured images.

To fix raw12 images already recorded with even/odd lines swapped use the --swap-lines parameter with raw2dng:

raw2dng image.raw12 --swap-lines

Details about the RAW12 format.