Difference between revisions of "AXIOM Alpha Software"

From apertus wiki
Jump to: navigation, search
 
(69 intermediate revisions by 10 users not shown)
Line 1: Line 1:
==Image Pipeline==
Image Acquisition Pipeline:
[[File:Image04.png]]
HDMI Image Processing Pipeline:
[[File:Image01.png]]
----
==Register Memory Space==
==Register Memory Space==
[[CMV12000 Register Blocks]]
[[CMV12000 to HDMI registers]]


Image sensor related registers: 0x60xxxxxx
Image sensor related registers: 0x60xxxxxx
Line 15: Line 34:
  V = register block
  V = register block
  WWWWW =  addresses inside a register block
  WWWWW =  addresses inside a register block


Register Addresses HDMI:
Register Addresses HDMI:
Line 22: Line 42:
  0x803xxxxx = 4 channel 12bit LUT
  0x803xxxxx = 4 channel 12bit LUT
  0x804xxxxx = HDMI PLL dynamic reconfig
  0x804xxxxx = HDMI PLL dynamic reconfig


Register Addresses Image Sensor:
Register Addresses Image Sensor:
Line 32: Line 53:
Examples:  
Examples:  
  0x60000000 = sensor, register block 0, address 0 (CMV register 0)
  0x60000000 = sensor, register block 0, address 0 (CMV register 0)
----
== Connecting to the camera ==
Connect the camera trough a micro usb cable.
Zedboard provides a serial to usb bridge
On linux : dmesg | cat tty
Find the correct tty /dev
Use putty and connect at 115200 bauds, standard settings.
./cmv_train3 -a
this trains the lvds lanes and needs to be done every time the camera reboots
please run it and see if it has proper output
sometimes I saw that due to some loose contact or some other problem all the training values were 0
if this is the case wiggling a bit with the sensor connector or just waiting did the trick every time
./setup.sh
prepares the HDMI output
it also takes other parameters like “1080p60” or “1080p50” or “SWIT”
from what we saw each monitor requires this to be a bit different
check the details by reading the contents of the setup.sh script
or just try each line and see if the monitor does something (should stay black but register signal input)
test the HDMI output by loading a single image:
./mimg -w newsmoke_n.rgb16
this does not work if you initiate live image acquisition already with ./fil_reg…. as it will overwrite the framebufer 25 times a second already so all you might end up seeing is a short flash of the above newsmoke image
./fil_reg 11 …
enables live video acquisition
it is a function in cmv.func so you need to run
. ./cmv.func beforehand
./chroot_init
starts the raspian chroot environment and webserver
you can go to the cameras IP over ethernet then and see the webgui (dont forget to configure ifconfig IP firsthand) Alpha will register DHCP automatically if used with a router
----


==Reading and Writing Sensor Register==
==Reading and Writing Sensor Register==
Line 47: Line 123:
  0x00000C31
  0x00000C31


===statically linked busybox===
 
----
 
 
==Enable/Disable LEDs==
 
Disable LEDs on the back:
fil_reg 15 0x01FF01FF
 
disable LEDs on the front:
fil_reg 14 0xFFFF0000
 
Enable all LEDs again:
 
Front:
fil_reg 14 0x0
Back:
fil_reg 15 0x0
 
 
----
 
 
==Row/Coloumn Noise Correction==
To run this calibration an evenly lit image is essential. Be sure to use a lens with little vignetting effects and set the aperture to a range that suffers from least vignetting effects. Diffusion gel and turning focus to infinite can help smooth out light differences.
It is advised to disable automatic image acquisition before running the RCN profiler:
./cmv_rcn3 -zsacr -N 5
 
Pixels going out of range will show up as blue dots. To prevent this enable clipping by setting register 11 bits: 21:20 to 1:
fil_reg 11 0xFC31F000 # clipping on
 
for reference (dont apply this setting in the FPN process:
fil_reg 11 0xFC01F000 # clipping off
 
Save FPN profile into a file (all 4 channels):
pmem -B 0x60000000 -m R0x60300000+0x8000/W > /tmp/rcn.save
 
Load FPN profile from a file (all 4 channels):
pmem -B 0x60000000 -m W0x60300000+0x8000/W </tmp/rcn.save
 
Clear FPN profile:
./cmv_rcn3 -z
 
If only a particular channel should be saved/loaded the addresses and lengths are different, eg:
pmem -B 0x60000000 -m R0x60304000+0x2000/W > /tmp/rcn_lut1.save
 
 
With the real time histogram you can evaluate how evenly lit your image is and can also judge the calibration results. The spikes in the histogram should for optimal results be in the 33% gray range and all line up in one place. With a good RCN profile they should get higher and less wide in the slope.
 
==statically linked busybox==
http://vserver.13thfloor.at/Stuff/AXIOM/FAKE/
http://vserver.13thfloor.at/Stuff/AXIOM/FAKE/


Line 58: Line 184:


/bin/sh and /sbin/devmem both link to busybox on the axiom alpha filesystem so both can be tested with this executeable
/bin/sh and /sbin/devmem both link to busybox on the axiom alpha filesystem so both can be tested with this executeable
----


==Capturing an Image==
==Capturing an Image==
Preparations:
Preparations:
  ssh root@*alpha-IP* "./cmv_train2
  ssh root@*alpha-IP* "./cmv_train3 -a"


Capture the image:
Capture the image:
  ssh root@*alpha-IP* "./cmv_snap2 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-
  ssh root@*alpha-IP* "./cmv_snap3 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-


So we dont need to type the password every time we snap an image:
So we dont need to type the password every time we snap an image:
Line 72: Line 202:


Use:
Use:
  sshpass -p '1234' ssh root@*alpha-IP* "./cmv_snap2 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-
  sshpass -p '1234' ssh root@*alpha-IP* "./cmv_snap3 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-
 
 
----
 
 
==Create Custom Image==
Using imagemagick:
convert originalimage.png -depth 16 -size 1920x1080 rgb:output.raw16
 
 
----
 
 
==Display Custom Image==
 
./mimg -h
This is ./mimg V1.4
options are:
-h        print this help message
-w        use word sized data
-D <val>  image color depth
-W <val>  image width
-H <val>  image height
-T <val>  load test pattern
-B <val>  memory mapping base
-S <val>  memory mapping size
-A <val>  memory mapping address
 
You can load custom images into the HDMI image pipeline. Be sure to disable automatic image acquisition beforehand or you will only be able to see the custom image flashing for a single frame in the video stream:
 
./mimg -w newsmoke_n.rgb16
 
 
----
 
 
==cmv_perf3==
 
displays image stats and performance data, the longer it runs the more accurate the stats will be.
 
./cmv_perf3 -h
This is ./cmv_perf3 V1.0
options are:
-h        print this help message
-f        measure frame rates
-m        measure memory bandwidth
-D <val>  delay between updates
-T <val>  run for a given time
 
Will get more accurate with ntpd running
ntpd -p 86.59.80.170 -p 86.59.118.153


==cmv_snap2==
Typical output:
  ./cmv_perf3
mapped 0x60000000+0x00400000 to 0x60000000.
mapped 0x80000000+0x00400000 to 0x80000000.
hdmi: 29.97 FPS  cseq: 29.97 FPS
 
 
----
 
 
==cmv_snap3==
capture one still image and display it over HDMI immediately. If the -z parameter is not used this tool will provide the image data via stdout.
<pre>
<pre>
./cmv_snap2 -h
./cmv_snap3 -h
This is ./cmv_snap2 V1.5
This is ./cmv_snap3 V1.9
options are:
options are:
-h        print this help message
-h        print this help message
-8        output 8 bit per pixel
-8        output 8 bit per pixel
-2        output 12 bit per pixel
-b        enable black columns
-b        enable black columns
-r        dump sensor registers
-r        dump sensor registers
-t        enable cmv test pattern
-t        enable cmv test pattern
-e <exp>  exposure time
-z        produce no data output
-m <valcapture mask and value
-e <exp>  exposure times
-n <num>  number of frames
-v <expexposure voltages
-s <num>  shift values by <num>
-s <num>  shift values by <num>
-B <val>  register mapping base
-S <val>  writer byte strobe
-S <val>  register mapping size
-M <val>  buffer memory base
-Z <val>  buffer memory size
-R <fil>  load sensor registers
-R <fil>  load sensor registers
-P <pat>  idle pattern
</pre>
</pre>


"./cmv_snap2 -r" without an -e parameter outputs only registers
./cmv_snap3 -r # without an -e parameter outputs only sensor registers
./cmv_snap3 -e 20ms > image.raw16 # capture an image and write the data into a file
 
Can also be used for direct capture to remote host:
ssh root@*cameraip* "./cmv_snap3 -e 20ms" > image.raw16 # capture an image and write the data into a file
 
 
To save all current sensor registers into a file
./cmv_snap3 -r > settings.reg
 
To load all current sensor registers from a file
./cmv_snap3 -R settings.reg
 
 
----
 
 
==cmv_hist3==


==cmv_hist2==
Outputs RAW histogram data of the last captured image. Values are tab separated in 4 columns (Order: GRBG[image flip on], RGGB[image flip off]) with 4096 values (when operating in 12 bit mode) each.
Outputs RAW histogram data of the last captured image. Values are tab separated in 4 columns (Order: GRBG[image flip on], RGGB[image flip off]) with 4096 values (when operating in 12 bit mode) each.


Download: http://vserver.13thfloor.at/Stuff/AXIOM/cmv_io2/cmv_hist2
Download: http://vserver.13thfloor.at/Stuff/AXIOM/cmv_hdmi2/cmv_hist3


<pre>
<pre>
Line 116: Line 321:


[[Real Time Histogram Plot]]
[[Real Time Histogram Plot]]
----


===Examples===
===Examples===
  ./cmv_hist2 -s
  ./cmv_hist2 -s
Acquire new snapshot and output histogram
Acquire new snapshot and output histogram
Line 124: Line 334:
Acquire histogram from last captured snapshot in memory
Acquire histogram from last captured snapshot in memory


==cmv_train2==
ssh root@192.168.2.88 "./cmv_hist3 -s" > capt.hist && gnuplot hist.gplot
 
 
----
 
 
==cmv_train3==


Executes the LVDS training routines to align LVDS reading timing. Execute this script when the Axiom Alpha prototype is booted up before you capture any images:
Executes the LVDS training routines to align LVDS reading timing. Execute this script when the Axiom Alpha prototype is booted up before you capture any images:
  ./cmv_train2
  ./cmv_train3 -a


<pre>
<pre>
./cmv_train2 -h
./cmv_train3 -h
This is ./cmv_train2 V1.0
This is ./cmv_train3 V1.3
options are:
options are:
-h        print this help message
-h        print this help message
Line 140: Line 356:
-P <val>  training pattern
-P <val>  training pattern
</pre>
</pre>
----
==Linearization==
This happens as very first step in the image pipeline, before the Row/Column Noise compensation.
./linear_conf.sh *factor* *offset*
Examples:
./linear_conf.sh 1.2 -0.1 # multiple by factor 1.2 and shift offset 10% into the darker area


==Post Processing images==
==Post Processing images==
Line 147: Line 376:
  convert -size 4096x3072 -depth 16 -crop +0-1 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch2.raw
  convert -size 4096x3072 -depth 16 -crop +0-1 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch2.raw
  convert -size 4096x3072 -depth 16 -crop -1-1 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch3.raw
  convert -size 4096x3072 -depth 16 -crop -1-1 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch3.raw
----


===Simple debayer with ImageMagick:===
===Simple debayer with ImageMagick:===
For flipped images:
For flipped images:
  convert \( -size 4096x3072 -depth 16 gray:colors_500ms.raw16 \) \
  convert \( -size 4096x3072 -depth 16 gray:colors_500ms.raw16 \)
  \( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \) \
  \( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \)
  -sample 2048x1536 \( -clone 2,3 -average \) -delete 2,3 -swap 0,1 +swap -combine colors_500ms.png
  -sample 2048x1536 \( -clone 2,3 -average \) -delete 2,3 -swap 0,1 +swap -combine colors_500ms.png


For unflipped images:
For unflipped images:
  convert \( -size 4096x3072 -depth 16 gray:IT8_incand.raw16 \) \
  convert \( -size 4096x3072 -depth 16 gray:IT8_incand.raw16 \)  
  \( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \) \
  \( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \)  
  -sample 2048x1536 \( -clone 0,1 -average \) -delete 0,1 +swap -combine IT8_incand.png
  -sample 2048x1536 \( -clone 0,1 -average \) -delete 0,1 +swap -combine IT8_incand.png
----


=== Plot Histogram of Raw Image ===
=== Plot Histogram of Raw Image ===
Line 185: Line 422:
execute the script with:
execute the script with:
  gnuplot thescript > output.svg
  gnuplot thescript > output.svg
----


==chroot environment on Axiom Alpha==
==chroot environment on Axiom Alpha==
Full raspian image is available on the alpha prototype in a changeroot environment to keep the core system free from too much clutter.
Full raspian image is available on the alpha prototype in a changeroot environment to keep the core system free from too much clutter.


Line 201: Line 443:
Changeroot:
Changeroot:
  chroot /opt/raspian/ /bin/bash
  chroot /opt/raspian/ /bin/bash
----


==Live HDMI video output==
==Live HDMI video output==


Init:
Init:
cd root
  ./cmv_train3 -a
  ./cmv_train3 -a
  ./setup.sh
  ./setup.sh




Run Video Acquisition:
Start Video Acquisition:
  time ./cmv_snap3 -N 0 -z -e 25ms
  . ./cmv.func
fil_reg 15 0x01000100
 
Stop Video Acquisition:
fil_reg 15 0x0
 
Half HDMI Frequency (1080p60 -> 1080p30 or 1080p50 -> 1080p25)
pll_reg 22 0x2106
 
 
Full HDMI Frequency (1080p60)
pll_reg 22 0x2083
 
 
----
 
 
===HDMI Output Parameters===
 
. ./i2c.func
 
i2c_map 0x39 0x55 5 6 0x0 #RGB output
i2c_map 0x39 0x55 5 6 0x1 #YCbCr 422 output
i2c_map 0x39 0x55 5 6 0x2 #YCbCr 444 output
 
 
----


Alternative (slower):
while true; do ./cmv_snap3 -z -e 32ms 2>/dev/null; done


==Update Firmware==
==Update Firmware==
u-boot access:
u-boot access:
  mount /dev/mmcblk0p1 /mnt/
  mount /dev/mmcblk0p1 /mnt/
----


==Real Time FPN correction==
==Real Time FPN correction==
FPN correction supports a 9bit signed value for each column and row.
FPN correction supports a 9bit signed value for each column and row.


Line 246: Line 523:


Row/Column 0 is in the top left corner.
Row/Column 0 is in the top left corner.
----




===Balance (FPN profiling tool)===
===Balance (FPN profiling tool)===
Download: http://vserver.13thfloor.at/Stuff/AXIOM/balance
Download: http://vserver.13thfloor.at/Stuff/AXIOM/balance


Line 272: Line 553:
One double is 8 bytes (64 bits)
One double is 8 bytes (64 bits)


==Matrix Color Conversation==
 
----
 
 
===In-camera FPN profiling===
 
Point the camera at a totally evenly lit non-clipping same color surface like a white wall (out of focus helps to even out minor details) and run:
 
./cmv_rcn3 -azc
 
To help visualize how evenly lit a surface is this LUT can help:
./lut_conf.sh -M 0x100000 -N 4096
 
 
----
 
 
==Matrix Color Conversion==
 
Each color channel is going through 4 stages that each look like this:
Each color channel is going through 4 stages that each look like this:
(I+A)*E+O
(I+A)*E+O
Line 313: Line 612:


Examples:
Examples:
  ./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1    0 0 0 0 # unity matrix but not optimal as both green channels are processed separately
  ./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1    0 0 0 0   # unity matrix but not optimal as both green channels are processed separately
  ./mat4_conf.sh  1 0 0 0  0 0.5 0.5 0  0 0.5 0.5 0  0 0 0 1    0 0 0 0 # the two green channels inside each 2x2 pixel block are averaged and output on both green pixels
  ./mat4_conf.sh  1 0 0 0  0 0.5 0.5 0  0 0.5 0.5 0  0 0 0 1    0 0 0 0   # the two green channels inside each 2x2 pixel block are averaged and output on both green pixels
  ./mat4_conf.sh  0 0 0 1  0 1 0 0  0 0 1 0  1 0 0 0    0 0 0 0  # red and blue are swapped
  ./mat4_conf.sh  0 0 0 1  0 1 0 0  0 0 1 0  1 0 0 0    0 0 0 0  # red and blue are swapped
  ./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1    0.5 0 0 0    # red 50% brigther
  ./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1    0.5 0 0 0    # red 50% brigther
./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1.5    0 0 0 0    # blue multiplied with factor 1.5
./mat4_conf.sh  .25 .25 .25 .25  .25 .25 .25 .25  .25 .25 .25 .25  .25 .25 .25 .25    0 0 0 0    # black/white
./mat4_conf.sh  -1 0 0 0  0 -0.5 -0.5 0  0 -0.5 -0.5 0  0 0 0 -1    1 1 1 1    # negative
An RGB 3x3 matrix can easily be extended to a 4x4 matrix by using the coefficient for green and offsets twice or better to half both of the green coefficients and use the same value for both greens twice.
[[File:Eqn3.png]]
[[File:Eqn4.png]]
----
==LUTs==
The result from the matrix conversation goes straight into 4x12 bit Look Up Tables (LUT).
The output is 16bit wide.
These 4096x16bit values can be set at the register address space: 0x8030xxxx [0x80300000 (ch0 - R), 0x80304000 (ch1 - G1), 0x80308000 (ch2 - G2), 0x8030C000 (ch3 - B)]
For creating classic gamma curves for the LUTs there is a tool called: lut_conf3
./lut_conf3 -h
This is ./lut_conf3 V1.5
options are:
-h        print this help message
-d        dump current values
-r        read index/value pairs
-w        write index/value pairs
-z        wrap on limits
-s <val>  scale input value
-m <val>  minimum output value
-N <val>  number of lut entries
-M <val>  maximum output value
-C <val>  sigmoid/sine center
-F <val>  output value factor
-O <val>  output value offset
-B <val>  memory mapping base
-S <val>  memory mapping size
-A <val>  memory mapping address
-G <val>  gamma value
Examples:
./lut_conf3 -N 4096 -M 65535 -G 0.6 -B 0x80300000 # create LUT of gamma curve for gamma value 0.6 and set it for red channel (ch0)
-N = number of entries
-M = maximum value of the curve
-G = gamma value
-B = channel address
-F = multiplication factor
-O = offset
Another tool that simplifies setting gamma curves is:
./gamma_conf.sh 0.6
Save LUT into a file (all 4 channels):
pmem -B 0x80000000 -m R0x80300000+0x10000/W > /tmp/lut.save
Load LUT from a file (all 4 channels):
pmem -B 0x80000000 -m W0x80300000+0x10000/W </tmp/lut.save
Write a LUT pair table into a file:
./lut_conf3 -d > gamma-table.save
Read a LUT pair table file back in:
./lut_conf3 -r < gamma-table.save
----
[[Category:AXIOM Alpha]]
[[Category:LUTs]]

Latest revision as of 16:49, 1 April 2018

1 Image Pipeline

Image Acquisition Pipeline:

Image04.png


HDMI Image Processing Pipeline:

Image01.png




2 Register Memory Space

CMV12000 Register Blocks

CMV12000 to HDMI registers

Image sensor related registers: 0x60xxxxxx

cmv.func contains the functions to deal with this address space


HDMI related registers: 0x80xxxxxx

hdmi.func contains the functions to deal with this address space

0xU0VWWWWW
U = image sensor or HDMI
V = register block
WWWWW =  addresses inside a register block


Register Addresses HDMI:

0x800xxxxx = scan generator
0x801xxxxx = address generator
0x802xxxxx = component matrix 4x4
0x803xxxxx = 4 channel 12bit LUT
0x804xxxxx = HDMI PLL dynamic reconfig


Register Addresses Image Sensor:

0x600xxxxx = CMV SPI registers
0x601xxxxx = address generator/capture control
0x602xxxxx = LVDS input delay registers
0x603xxxxx = column/row noise LUTs


Examples:

0x60000000 = sensor, register block 0, address 0 (CMV register 0)




3 Connecting to the camera

Connect the camera trough a micro usb cable.

Zedboard provides a serial to usb bridge

On linux : dmesg | cat tty

Find the correct tty /dev

Use putty and connect at 115200 bauds, standard settings.


./cmv_train3 -a

this trains the lvds lanes and needs to be done every time the camera reboots please run it and see if it has proper output sometimes I saw that due to some loose contact or some other problem all the training values were 0 if this is the case wiggling a bit with the sensor connector or just waiting did the trick every time

./setup.sh

prepares the HDMI output it also takes other parameters like “1080p60” or “1080p50” or “SWIT” from what we saw each monitor requires this to be a bit different check the details by reading the contents of the setup.sh script or just try each line and see if the monitor does something (should stay black but register signal input)

test the HDMI output by loading a single image: ./mimg -w newsmoke_n.rgb16

this does not work if you initiate live image acquisition already with ./fil_reg…. as it will overwrite the framebufer 25 times a second already so all you might end up seeing is a short flash of the above newsmoke image


./fil_reg 11 …

enables live video acquisition it is a function in cmv.func so you need to run . ./cmv.func beforehand


./chroot_init

starts the raspian chroot environment and webserver

you can go to the cameras IP over ethernet then and see the webgui (dont forget to configure ifconfig IP firsthand) Alpha will register DHCP automatically if used with a router




4 Reading and Writing Sensor Register

Execute the script that contains the sensor interface functions before we can use them:

. ./cmv.func

Set Sensor register 82 to the decimal value "3122"

cmv_reg  82   3122

Read Register 82:

 cmv_reg  82

will output:

0x00000C31




5 Enable/Disable LEDs

Disable LEDs on the back:

fil_reg 15 0x01FF01FF

disable LEDs on the front:

fil_reg 14 0xFFFF0000

Enable all LEDs again:

Front:

fil_reg 14 0x0

Back:

fil_reg 15 0x0




6 Row/Coloumn Noise Correction

To run this calibration an evenly lit image is essential. Be sure to use a lens with little vignetting effects and set the aperture to a range that suffers from least vignetting effects. Diffusion gel and turning focus to infinite can help smooth out light differences.

It is advised to disable automatic image acquisition before running the RCN profiler:

./cmv_rcn3 -zsacr -N 5

Pixels going out of range will show up as blue dots. To prevent this enable clipping by setting register 11 bits: 21:20 to 1:

fil_reg 11 0xFC31F000 # clipping on

for reference (dont apply this setting in the FPN process:

fil_reg 11 0xFC01F000 # clipping off

Save FPN profile into a file (all 4 channels):

pmem -B 0x60000000 -m R0x60300000+0x8000/W > /tmp/rcn.save

Load FPN profile from a file (all 4 channels):

pmem -B 0x60000000 -m W0x60300000+0x8000/W </tmp/rcn.save

Clear FPN profile:

./cmv_rcn3 -z

If only a particular channel should be saved/loaded the addresses and lengths are different, eg:

pmem -B 0x60000000 -m R0x60304000+0x2000/W > /tmp/rcn_lut1.save


With the real time histogram you can evaluate how evenly lit your image is and can also judge the calibration results. The spikes in the histogram should for optimal results be in the 33% gray range and all line up in one place. With a good RCN profile they should get higher and less wide in the slope.

7 statically linked busybox

http://vserver.13thfloor.at/Stuff/AXIOM/FAKE/

builtin fake devmem

all you need to get it to work is the following:

dd if=/dev/zero of=/tmp/mem bs=1k seek=4M count=1

this will create a sparse 4GB file /tmp/mem, which will be used by the fake devmem values written can be read back, non existing values return 0

/bin/sh and /sbin/devmem both link to busybox on the axiom alpha filesystem so both can be tested with this executeable




8 Capturing an Image

Preparations:

ssh root@*alpha-IP* "./cmv_train3 -a"

Capture the image:

ssh root@*alpha-IP* "./cmv_snap3 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-

So we dont need to type the password every time we snap an image:

Install:

sudo apt-get install sshpass

Use:

sshpass -p '1234' ssh root@*alpha-IP* "./cmv_snap3 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-




9 Create Custom Image

Using imagemagick:

convert originalimage.png -depth 16 -size 1920x1080 rgb:output.raw16




10 Display Custom Image

./mimg -h
This is ./mimg V1.4
options are:
-h        print this help message
-w        use word sized data
-D <val>  image color depth
-W <val>  image width
-H <val>  image height
-T <val>  load test pattern
-B <val>  memory mapping base
-S <val>  memory mapping size
-A <val>  memory mapping address

You can load custom images into the HDMI image pipeline. Be sure to disable automatic image acquisition beforehand or you will only be able to see the custom image flashing for a single frame in the video stream:

./mimg -w newsmoke_n.rgb16




11 cmv_perf3

displays image stats and performance data, the longer it runs the more accurate the stats will be.

./cmv_perf3 -h
This is ./cmv_perf3 V1.0
options are:
-h        print this help message
-f        measure frame rates
-m        measure memory bandwidth
-D <val>  delay between updates
-T <val>  run for a given time

Will get more accurate with ntpd running

ntpd -p 86.59.80.170 -p 86.59.118.153

Typical output:

 ./cmv_perf3
mapped 0x60000000+0x00400000 to 0x60000000.
mapped 0x80000000+0x00400000 to 0x80000000.
hdmi: 29.97 FPS  cseq: 29.97 FPS




12 cmv_snap3

capture one still image and display it over HDMI immediately. If the -z parameter is not used this tool will provide the image data via stdout.

./cmv_snap3 -h
This is ./cmv_snap3 V1.9
options are:
-h        print this help message
-8        output 8 bit per pixel
-2        output 12 bit per pixel
-b        enable black columns
-r        dump sensor registers
-t        enable cmv test pattern
-z        produce no data output
-e <exp>  exposure times
-v <exp>  exposure voltages
-s <num>  shift values by <num>
-S <val>  writer byte strobe
-R <fil>  load sensor registers
./cmv_snap3 -r # without an -e parameter outputs only sensor registers
./cmv_snap3 -e 20ms > image.raw16 # capture an image and write the data into a file

Can also be used for direct capture to remote host:

ssh root@*cameraip* "./cmv_snap3 -e 20ms" > image.raw16 # capture an image and write the data into a file


To save all current sensor registers into a file

./cmv_snap3 -r > settings.reg 

To load all current sensor registers from a file

./cmv_snap3 -R settings.reg




13 cmv_hist3

Outputs RAW histogram data of the last captured image. Values are tab separated in 4 columns (Order: GRBG[image flip on], RGGB[image flip off]) with 4096 values (when operating in 12 bit mode) each.

Download: http://vserver.13thfloor.at/Stuff/AXIOM/cmv_hdmi2/cmv_hist3

./cmv_hist2
This is ./cmv_hist2 V1.0
options are:
-h        print this help message
-s        acquire snapshot
-B <val>  register mapping base
-S <val>  register mapping size
-M <val>  buffer memory base
-Z <val>  buffer memory size

Real Time Histogram Plot




13.1 Examples

./cmv_hist2 -s

Acquire new snapshot and output histogram

./cmv_hist2

Acquire histogram from last captured snapshot in memory

ssh root@192.168.2.88 "./cmv_hist3 -s" > capt.hist && gnuplot hist.gplot




14 cmv_train3

Executes the LVDS training routines to align LVDS reading timing. Execute this script when the Axiom Alpha prototype is booted up before you capture any images:

./cmv_train3 -a
 ./cmv_train3 -h
This is ./cmv_train3 V1.3
options are:
-h        print this help message
-a        test all bit pattern
-B <val>  memory mapping base
-S <val>  memory mapping size
-A <val>  memory mapping address
-P <val>  training pattern




15 Linearization

This happens as very first step in the image pipeline, before the Row/Column Noise compensation.

./linear_conf.sh *factor* *offset*

Examples:

./linear_conf.sh 1.2 -0.1 # multiple by factor 1.2 and shift offset 10% into the darker area

16 Post Processing images

16.1 Create RGGB separated color images from raw file:

convert -size 4096x3072 -depth 16 -crop +0+0 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch0.raw
convert -size 4096x3072 -depth 16 -crop -1+0 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch1.raw
convert -size 4096x3072 -depth 16 -crop +0-1 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch2.raw
convert -size 4096x3072 -depth 16 -crop -1-1 -sample 2048x1536 gray:colors_500ms.raw16 gray:colors_500ms_ch3.raw




16.2 Simple debayer with ImageMagick:

For flipped images:

convert \( -size 4096x3072 -depth 16 gray:colors_500ms.raw16 \)
\( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \)
-sample 2048x1536 \( -clone 2,3 -average \) -delete 2,3 -swap 0,1 +swap -combine colors_500ms.png

For unflipped images:

convert \( -size 4096x3072 -depth 16 gray:IT8_incand.raw16 \) 
\( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \) 
-sample 2048x1536 \( -clone 0,1 -average \) -delete 0,1 +swap -combine IT8_incand.png



16.3 Plot Histogram of Raw Image

Create histogram values file with imagemagick:

convert -size 4096x3072 -depth 16 gray:image.raw16 -format "%c" histogram:info: > histogram.hist

Reformat the file:

gawk -F, '(NF>3) { printf "%d\t%d\n", $2/16, $1 }' 

Or all in one command:

convert -size 4096x3072 -depth 16 gray:image.raw16 -format %c histogram:info:- | gawk -F, '(NF>3) { printf "%d\t%d\n", $2/16, $1 }' > histogram.info

Then draw it with this gnuplot script:

set notitle
set term svg size 1024, 512
set xlabel "Values"
set ylabel "Number of Pixels"
set multiplot
set obj 1 rectangle behind from screen 0,0 to screen 1,1
set obj 1 fillstyle solid 1.0 fillcolor rgbcolor "white"
set xrange[0:4096]
set yrange[0:8000]
plot 	'histogram.info' using 1:2	with lines lc rgb "#FF0000"	title "name your curve", \
       'histogram2.info' using 1:2	with lines lc rgb "#FF0000"	title "name your curve"
unset multiplot

execute the script with:

gnuplot thescript > output.svg




17 chroot environment on Axiom Alpha

Full raspian image is available on the alpha prototype in a changeroot environment to keep the core system free from too much clutter.

Download:

http://vserver.13thfloor.at/Stuff/AXIOM/raspian.tar.xz

Preferably download from the alpha prototype directly and upack into /opt/raspian, moving files to the prototype using Filezilla has produced some problems.

Map required folders:

mount -o bind /proc /opt/raspian/proc
mount -o bind /tmp /opt/raspian/tmp

Changeroot:

chroot /opt/raspian/ /bin/bash




18 Live HDMI video output

Init:

cd root
./cmv_train3 -a
./setup.sh


Start Video Acquisition:

. ./cmv.func
fil_reg 15 0x01000100

Stop Video Acquisition:

fil_reg 15 0x0

Half HDMI Frequency (1080p60 -> 1080p30 or 1080p50 -> 1080p25)

pll_reg 22 0x2106


Full HDMI Frequency (1080p60)

pll_reg 22 0x2083




18.1 HDMI Output Parameters

. ./i2c.func
i2c_map 0x39 0x55 5 6 0x0 #RGB output
i2c_map 0x39 0x55 5 6 0x1 #YCbCr 422 output
i2c_map 0x39 0x55 5 6 0x2 #YCbCr 444 output




19 Update Firmware

u-boot access:

mount /dev/mmcblk0p1 /mnt/




20 Real Time FPN correction

FPN correction supports a 9bit signed value for each column and row.

9 bit signed integer means:

0xFF = 255 (highest possible positive value)
0x1FF = -1
0x100 = -256 (lowest possible negative value

To obtain the correct value (for -7 in this case) to set you can run:

echo $(( -7 & 0x1FF ))

which outputs (in decimal)

505

To set a value use syntax:

devmem address 32 value

or with actual values:

devmem 0x60304000 32 0xFF

Adresses are:

0x60300000 = even columns, 2048 total
0x60304000 = odd columns, 2048 total
0x60308000 = even rows, 2048 total (only first 1576 are used)
0x6030C000 = odd rows, 2048 total (only first 1576 are used)

each row/column address increases by 4 from current to next row/column.

Row/Column 0 is in the top left corner.




20.1 Balance (FPN profiling tool)

Download: http://vserver.13thfloor.at/Stuff/AXIOM/balance

balance can extract a DSNU profile from a darkframes and apply the created profile to correct an image.

Usage:

Extract FPN profile:

./balance -b -o darkframe.raw16 > profile.offsets

Apply FPN profile to image:

./balance -b -O profile.offsets image.raw16 > image_balanced.raw16

Only use the -b parameter if the image uses darkcolumns.


The profile.offsets file format is:

row-numer offset (double) then column number (double)

(3072+4096)x8 = 57344 Bytes = 57KB

One double is 8 bytes (64 bits)




20.2 In-camera FPN profiling

Point the camera at a totally evenly lit non-clipping same color surface like a white wall (out of focus helps to even out minor details) and run:

./cmv_rcn3 -azc

To help visualize how evenly lit a surface is this LUT can help:

./lut_conf.sh -M 0x100000 -N 4096




21 Matrix Color Conversion

Each color channel is going through 4 stages that each look like this: (I+A)*E+O

I = input (25bit unsigned)
A = adjustment (30bit signed)
E = matrix coefficient (18bit signed)
O = offset (48bit signed)

The four stages are connected together that the offset of stage N is the output of stage N-1. The offset of stage 1 is set in registers (32-35). The output of stage 4 is the final result.


For each channel that results in the following fomular:

Ri = SUMj ((Ij + Aij) * Eij) + Oi

Which in other words mean the result of matrix (Eij) multiplied with an input vector (Ij + Aij) plus an offset vector (Oi).

Currently all registers (beside input/output) are shortened to 16bit signed (this could be changed easily) values so the matrix is currently limited to 1/256 of the possible accuracy (due to a 8bit shift of the output).

All input /output register are tested for over/underflow and clipped to 12 bit (this can also be disabled).


Setting/Getting Registers:

. ./hdmi.func # load functions
mat_reg 0 # Read value of register 0 = A0
mat_reg 0 1 # Sett value of register 0 (A0) to 1

4x4 matrix coefficients (A0, A1, A2, A3, B0, B1, B2, B3, B4, C0, C1, C2, C3, D0, D1, D2, D3) are mapped to addresses 0 - 15. In registers 16 - 31 there are adjustment values and from 32 - 35 there are 4 offset values.

Input format per default is (R, G1, G2, B).

All values of the 4x4 matrix can be set conveniently with this script:

Syntax:

./mat4_conf.sh  A0 A1 A2 A3  B0 B1 B2 B3 B4  C0 C1 C2 C3  D0 D1 D2 D3    O1 O2 O3 O4

Examples:

./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1    0 0 0 0   # unity matrix but not optimal as both green channels are processed separately
./mat4_conf.sh  1 0 0 0  0 0.5 0.5 0  0 0.5 0.5 0  0 0 0 1    0 0 0 0   # the two green channels inside each 2x2 pixel block are averaged and output on both green pixels
./mat4_conf.sh  0 0 0 1  0 1 0 0  0 0 1 0  1 0 0 0    0 0 0 0   # red and blue are swapped
./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1    0.5 0 0 0    # red 50% brigther
./mat4_conf.sh  1 0 0 0  0 1 0 0  0 0 1 0  0 0 0 1.5    0 0 0 0    # blue multiplied with factor 1.5
./mat4_conf.sh  .25 .25 .25 .25  .25 .25 .25 .25  .25 .25 .25 .25  .25 .25 .25 .25    0 0 0 0    # black/white
./mat4_conf.sh  -1 0 0 0  0 -0.5 -0.5 0  0 -0.5 -0.5 0  0 0 0 -1    1 1 1 1    # negative


An RGB 3x3 matrix can easily be extended to a 4x4 matrix by using the coefficient for green and offsets twice or better to half both of the green coefficients and use the same value for both greens twice.

Eqn3.png

Eqn4.png




22 LUTs

The result from the matrix conversation goes straight into 4x12 bit Look Up Tables (LUT). The output is 16bit wide.

These 4096x16bit values can be set at the register address space: 0x8030xxxx [0x80300000 (ch0 - R), 0x80304000 (ch1 - G1), 0x80308000 (ch2 - G2), 0x8030C000 (ch3 - B)]

For creating classic gamma curves for the LUTs there is a tool called: lut_conf3

./lut_conf3 -h
This is ./lut_conf3 V1.5
options are:
-h        print this help message
-d        dump current values
-r        read index/value pairs
-w        write index/value pairs
-z        wrap on limits
-s <val>  scale input value
-m <val>  minimum output value
-N <val>  number of lut entries
-M <val>  maximum output value
-C <val>  sigmoid/sine center
-F <val>  output value factor
-O <val>  output value offset
-B <val>  memory mapping base
-S <val>  memory mapping size
-A <val>  memory mapping address
-G <val>  gamma value


Examples:

./lut_conf3 -N 4096 -M 65535 -G 0.6 -B 0x80300000 # create LUT of gamma curve for gamma value 0.6 and set it for red channel (ch0)
-N = number of entries
-M = maximum value of the curve
-G = gamma value
-B = channel address
-F = multiplication factor
-O = offset

Another tool that simplifies setting gamma curves is:

./gamma_conf.sh 0.6


Save LUT into a file (all 4 channels):

pmem -B 0x80000000 -m R0x80300000+0x10000/W > /tmp/lut.save

Load LUT from a file (all 4 channels):

pmem -B 0x80000000 -m W0x80300000+0x10000/W </tmp/lut.save

Write a LUT pair table into a file:

./lut_conf3 -d > gamma-table.save

Read a LUT pair table file back in:

./lut_conf3 -r < gamma-table.save