Difference between revisions of "Factory Calibration"
Line 151: | Line 151: | ||
Make sure you have the required octave function file in place: | Make sure you have the required octave function file in place: | ||
wget https://github.com/apertus-open-source-cinema/misc-tools-utilities/blob/master/darkframes/read_raw.m | wget https://github.com/apertus-open-source-cinema/misc-tools-utilities/blob/master/darkframes/read_raw.m | ||
Also you need to install the octave "signal" package from: http://octave.sourceforge.net/packages.php | |||
then run to install: | |||
pkg install -forge package_name | |||
To check whether the entire row noise is dynamic, load the two raw images in octave and check the autocorrelation between the two row noise samples: | To check whether the entire row noise is dynamic, load the two raw images in octave and check the autocorrelation between the two row noise samples: |
Revision as of 14:22, 30 October 2016
1 Hint:
Create a variable containing your Betas IP for easy access.
export BETA=192.168.1.101
2 Preparations
Install on your AXIOM Beta:
pacman -S python-numpy
Install the following packages on your PC:
dcraw octave
For Ubuntu this would look like:
sudo apt-get install dcraw octave
2.1 Step 1: Check range of the input signal
On the Beta set gain to x1 by running:
./set_gain.sh 1
Download this Octave file to your PC into your current work directory:
wget https://github.com/apertus-open-source-cinema/misc-tools-utilities/blob/master/darkframes/read_raw.m
Capture an overexposed image with the Beta and check the levels:
ssh root@$BETA "./cmv_snap3 -2 -b -r -e 100ms" > snap.raw12 ./raw2dng snap.raw12 --totally-raw octave octave:1> a = read_raw('snap.DNG') octave:2> prctile(a(:), [0.1 1 50 99 99.9])
If everything worked you will get a wall of numbers now. TODO: We should extract the essential pieces of information here... (min/max maybe)?
Lower numbers should be around 50...300 (certainly not zero). Higher numbers should be around 4000, but not 4095.
Repeat for gains 2, 3, 4.
Put this in startup script (ie: kick_manual.sh) :
./set_gain.sh 1
2.2 Step 2: RCN calibration
Make sure you have these scripts already in your Betas /root/ directly: https://github.com/apertus-open-source-cinema/beta-software/tree/master/beta-scripts
Clear the old RCN values:
ssh root@$BETA "./rcn_clear.py"
Now you need to make sure that your Beta is not capturing any light (really not a single photon should hit the sensor :) ):
- close the lens aperture as far as possible
- attach lens cap
- put black lens bag over Beta
- turn off all lights in the room - do this at night or in a completely dark room
Take 64 dark frames at 10ms, gain x1.
./set_gain.sh 1 for i in `seq 1 64`; do ssh root@$BETA "./cmv_snap3 -2 -b -r -e 10ms" > dark-x1-10ms-$i.raw12 done
Compute a temporary dark frame for RCN calibration:
raw2dng --swap-lines --no-blackcol --calc-darkframe dark-x1-10ms-*.raw12
This should process quite quickly and output something like the following at the end:
Averaged 64 frames exposed from 12.00 to 12.00 ms. Could not compute dark current. Please use different exposures, e.g. from 1 to 50 ms. Dark offset : 0.00 Writing darkframe-x1.pgm... Done.
Rename and upload darkframe to your Beta:
mv darkframe-x1.pgm darkframe-rcn.pgm scp darkframe-rcn.pgm root@$BETA:/root/
Set the RCN values:
ssh root@$BETA "./rcn_darkframe.py darkframe-rcn.pgm"
Put this in startup script (ie : kick_manual.sh) :
./rcn_darkframe.py darkframe-rcn.pgm
If you get an error report like this:
Traceback (most recent call last): File "rcn_darkframe.py", line 17, in <module> import png ImportError: No module named 'png'
Make sure the Beta is connected to the Internet via Ethernet and run:
pip install pypng
and then run the python script again
2.2.1 Validation
2.2.1.1 Method 1:
Put a lens cap on the camera and check the image on a HDMI monitor.
In the camera set the matrix gains to:
./mat4_conf.sh 20 0 0 0 0 10 10 0 0 10 10 0 0 0 0 10 0 0 0 0
run:
./rcn_clear.py
The static noise profile should be visible.
run:
./rcn_darkframe.py darkframe-rcn.pgm
The static noise profile should be gone. You will still see dynamic row noise (horizontal lines flickering) - thats expected.
2.2.1.2 Method 2:
This Method is now entirely automated with running one script: https://github.com/apertus-open-source-cinema/beta-software/blob/master/beta-scripts/rcn_validation.sh
Capture one darkframe without compensations:
ssh root@$BETA "./rcn_clear.py" ssh root@$BETA "./cmv_snap3 -2 -b -r -e 10ms" > dark-check-1.raw12
Capture one darkframe with compensations:
ssh root@$BETA "./rcn_darkframe.py darkframe-rcn.pgm" ssh root@$BETA "./cmv_snap3 -2 -b -r -e 10ms" > dark-check-2.raw12
Then use raw2dng to analyze the differences:
raw2dng --no-darkframe --check-darkframe dark-check-1.raw12 raw2dng --no-darkframe --check-darkframe dark-check-2.raw12
With the compensated snapshot the column noise should disappear, and only row noise left should be dynamic (not static). Visual inspection: the dark frame should have only horizontal lines, not vertical ones.
Sample output:
Average : 127.36 # about 128, OK Pixel noise : 5.44 # this one is a bit high because we only corrected row and column offsets (it's OK) Row noise : 2.30 (42.2%) # this one should be only dynamic row noise - see Method 3 below. Col noise : 0.20 (3.8%) # this one is very small, that's what we need to check here
2.2.1.3 Method 3:
Capture 2 frames:
ssh root@$BETA "./cmv_snap3 -2 -b -r -e 10ms" > dark-check-1.raw12 ssh root@$BETA "./cmv_snap3 -2 -b -r -e 10ms" > dark-check-2.raw12
Convert the two darkframes with raw2dng:
raw2dng dark-check-*
Make sure you have the required octave function file in place:
wget https://github.com/apertus-open-source-cinema/misc-tools-utilities/blob/master/darkframes/read_raw.m
Also you need to install the octave "signal" package from: http://octave.sourceforge.net/packages.php then run to install:
pkg install -forge package_name
To check whether the entire row noise is dynamic, load the two raw images in octave and check the autocorrelation between the two row noise samples:
pkg load signal a = read_raw('dark-check-1.DNG'); b = read_raw('dark-check-2.DNG'); ra = mean(a'); ra = ra - mean(ra); rb = mean(b'); rb = rb - mean(rb); xcov(ra, rb, 0, 'coeff')
Result should be very small (about 0.1 or lower). When running this check on two uncalibrated dark frames, you will get around 0.8 - 0.9.
2.3 Step 3: Dark frame calibration
Take dark frames at various exposure times and gains.
for i in 1 2 3 4; do for e in `seq 1 100`; do for g in 1 2 3 4; do ssh root@$BETA "./set_gain.sh $g" ssh root@$BETA "./cmv_snap3 -2 -b -r -e ${e}ms" > dark-x${g}-${e}ms-$i.raw12 done done done
Compute dark frames for each gain:
raw2dng --swap-lines --calc-dcnuframe dark-x1-*.raw12 raw2dng --swap-lines --calc-dcnuframe dark-x2-*.raw12 raw2dng --swap-lines --calc-dcnuframe dark-x3-*.raw12 raw2dng --swap-lines --calc-dcnuframe dark-x4-*.raw12
Save the following files (N=1..4):
darkframe-xN.pgm dcnuframe-xN.pgm
These files should be used in postprocessing. Place them in the directory where you capture raw12 files, so raw2dng will use them.
2.3.1 Validation
On the same dark frames, or - even better - on a new set of dark frames, run:
raw2dng --swap-lines --check-darkframe dark*.raw12 > dark-check.log
Upload the log.
2.4 Step 4: Color profiling
Set gain x1.
ssh root@$BETA "./set_gain.sh 1"
Take a picture of the IT8 chart, correctly exposed.
Edit the coordinates and the raw file name in calib_argyll.sh.
ssh root@$BETA "./cmv_snap3 -2 -b -r -e 10ms" > it8chart.raw12 ./calib_argyll.sh IT8
Save the following files:
- ICC profile (*.icc)
- OCIO configuration (copy/paste from terminal) + LUT file (*.spi1d)
2.4.1 Validation
Render the IT8 chart in Blender, using the OCIO configuration.
Same with the ICC profile (Adobe? RawTherapee? What apps support ICC?)
(todo: detailed steps)
2.5 Step 5: HDMI dark frames
Record a 1-minute clip with lens cap on.
Average odd and even frames.
(todo: polish and upload the averaging script)
(todo: check if the HDMI dark frames can be computed from regular dark frames)
Results: darkframe-hdmi-A.ppm and darkframe-hdmi-B.ppm.
2.6 Step 6: HDMI filters for raw recovery
This calibration is for to the recorder, not for the camera. It's for recovering the original raw data from the HDMI, so it has nothing to do with sensor profiling and such.
Record some scene with high detail AND rich colors.
Take a raw12 snapshot in the middle of recording. The HDMI stream will pause for a few seconds.
Upload two frames from the paused clip, together with the raw12 file. This calibration will be hardcoded in hdmi4k.
The two frames must be in the native format of your video recorder (not DNG). You should be able to cut the video with ffmpeg -vcodec copy.
2.7 TODO
- batch script to copy all the utilities for the workflow
- implement this: https://wiki.apertus.org/index.php/Calibration_files
- automate HDMI calibration
- remind Herbert to fix the line swapping bug