Difference between revisions of "AXIOM Beta/AXIOM Beta Software"

From apertus wiki
Jump to: navigation, search
m (→‎Operating System: Add "Pattern noise correction" section)
 
(186 intermediate revisions by 19 users not shown)
Line 1: Line 1:
[[File:BetaGettingStarted.jpg | thumb | 400px]]
==Misc Scripts==
 
 
=Getting Started=
 
==Connectivity==
===Prep your AXIOM Beta camera for use===
# Use a micro-USB cable to connect the camera's MicroZed development board (USB UART) to a computer. The MicroZed board is the backmost, red PCB. (There is another micro-USB socket on the Power Board, but that is the JTAG Interface.)
# Connect the ethernet port on the MicroZed to an ethernet port on your computer. You might have to use an ethernet adapter on newer, smaller machines which come without a native ethernet port.
# Connect the AC adapter to the camera's Power Board. (The power cord plugs into an adapter that connects to the Power Board; to power the camera off at a later point, you need not disconnect the adapter from the board but can just unplug the cord from the adapter.)
 
 
[[File:BetaGuide.jpg  | 500px]]
 
 
===Connect to the camera===
By default, the Beta requests an IP address via DHCP.
 
The camera is accessed via the Serial Console (USB UART).<br>
Drivers for bridging USB to UART [https://www.silabs.com/products/mcu/Pages/USBtoUARTBridgeVCPDrivers.aspx can be downloaded] from Silicon Labs' website.  We have successfully used Minicom, Screen or Putty to access UART.
 
Once the Beta is connected and powered on, it is listed as a USB device in the <code>/dev</code> directory of your file system, e.g. <code>/dev/ttyUSB0</code> (Linux) or <code>/dev/tty.SLAB_USBtoUART</code> (Mac OS X).<br>
You can use e.g. <code>ls -al /dev/ | grep -i usb</code> to list all USB devices connected to your machine.
 
 
To connect to the camera, use the command:
screen file_path 115200
 
e.g. (depending on your operating system):
screen /dev/ttyttyUSB0 115200
screen /dev/tty.SLAB_USBtoUART 115200
 
You might have to run the command with superuser rights, i.e.:
sudo screen /dev/ttyttyUSB0 115200
 
 
On successful connection, you will be prompted to enter user credentials needed for logging into the camera.<br>
If your terminal remains blank, try pressing enter.
 
The default credentials are:
user: root
password: beta
 
===Disconnect from the camera===
To disconnect from the camera again, use:
exit
The result will be a logout message followed by a new login prompt. To suspend or quit your <code>screen</code> session (and return to your regular terminal window) use one of the following commands:
CTRL+a CTRL+z
CTRL+a CTRL+\
 
 
It is required to use keypair authentication for SSH access to the Beta via Ethernet:
* Add your public key to /root/.ssh/authorized_keys
 
Notice: We had problems that screen could not copy/paste the entire key at once - had to split it copy/pasting two halves in the end.
 
==Setup==
 
The init script is automatically run at startup:
./kick.sh
 
This will initialize all systems and train the sensor communication.
 
==Capture an image==
 
write the image into file: snap.raw16 plus display it with imagemagick:
ssh root@*BETA-IP* "./cmv_snap3 -e 10ms" | tee snap.raw16 | display -size 4096x3072 -depth 16 gray:-
 
same for 12bit (more efficient, same data):
ssh root@*BETA-IP* "./cmv_snap3 -2 -e 10ms" | tee snap.raw12 | display -size 4096x3072 -depth 12 gray:-
 
==Overlay Images==
Remove overlay
./mimg -a -o -P 0
 
==Addititonal information==


Display voltages and current flow:
Display voltages and current flow:
  ./pac1720_info.sh
  ./pac1720_info.sh
Output:
Output:
  <nowiki>ZED_5V        5.0781 V [2080] +29.0625 mV [2e8]  +968.75 mA
  ZED_5V        5.0781 V [2080] +29.0625 mV [2e8]  +968.75 mA<br />
BETA_5V      5.1172 V [20c0] +26.6016 mV [2a9]  +886.72 mA
BETA_5V      5.1172 V [20c0] +26.6016 mV [2a9]  +886.72 mA<br />
HDN          3.2422 V [14c0] -0.0391 mV [fff]    -1.30 mA
HDN          3.2422 V [14c0] -0.0391 mV [fff]    -1.30 mA<br />
PCIE_N_V      3.2422 V [14c0] -0.0391 mV [fff]    -1.30 mA
PCIE_N_V      3.2422 V [14c0] -0.0391 mV [fff]    -1.30 mA<br />
HDS          3.2422 V [14c0] +0.0000 mV [000]    +0.00 mA
HDS          3.2422 V [14c0] +0.0000 mV [000]    +0.00 mA<br />
PCIE_S_V      3.2422 V [14c0] -0.0391 mV [fff]    -1.30 mA
PCIE_S_V      3.2422 V [14c0] -0.0391 mV [fff]    -1.30 mA<br />
RFW_V        3.2812 V [1500] +0.2734 mV [007]    +9.11 mA
RFW_V        3.2812 V [1500] +0.2734 mV [007]    +9.11 mA<br />
IOW_V        3.2422 V [14c0] +0.0000 mV [000]    +0.00 mA
IOW_V        3.2422 V [14c0] +0.0000 mV [000]    +0.00 mA<br />
RFE_V        3.2812 V [1500] +0.2344 mV [006]    +7.81 mA
RFE_V        3.2812 V [1500] +0.2344 mV [006]    +7.81 mA<br />
IOE_V        3.2812 V [1500] +0.0781 mV [002]    +2.60 mA
IOE_V        3.2812 V [1500] +0.0781 mV [002]    +2.60 mA<br />
VCCO_35      2.5000 V [1000] +0.6641 mV [011]    +22.14 mA
VCCO_35      2.5000 V [1000] +0.6641 mV [011]    +22.14 mA<br />
VCCO_13      2.4609 V [ fc0] +1.2500 mV [020]    +41.67 mA
VCCO_13      2.4609 V [ fc0] +1.2500 mV [020]    +41.67 mA<br />
PCIE_IO      2.4609 V [ fc0] -0.0391 mV [fff]    -1.30 mA
PCIE_IO      2.4609 V [ fc0] -0.0391 mV [fff]    -1.30 mA<br />
VCCO_34      2.4609 V [ fc0] +0.8203 mV [015]    +27.34 mA
VCCO_34      2.4609 V [ fc0] +0.8203 mV [015]    +27.34 mA<br />
W_VW          1.9922 V [ cc0] -0.0781 mV [ffe]    -2.60 mA
W_VW          1.9922 V [ cc0] -0.0781 mV [ffe]    -2.60 mA<br />
N_VW          3.1641 V [1440] +0.0000 mV [000]    +0.00 mA
N_VW          3.1641 V [1440] +0.0000 mV [000]    +0.00 mA<br />
N_VN          1.8750 V [ c00] +15.4297 mV [18b]  +514.32 mA
N_VN          1.8750 V [ c00] +15.4297 mV [18b]  +514.32 mA<br />
N_VE          3.1641 V [1440] +0.0000 mV [000]    +0.00 mA
N_VE          3.1641 V [1440] +0.0000 mV [000]    +0.00 mA<br />
E_VE          1.9922 V [ cc0] -0.0391 mV [fff]    -1.30 mA
E_VE          1.9922 V [ cc0] -0.0391 mV [fff]    -1.30 mA<br />
S_VE          1.9531 V [ c80] +0.0000 mV [000]    +0.00 mA
S_VE          1.9531 V [ c80] +0.0000 mV [000]    +0.00 mA<br />
S_VS          2.9297 V [12c0] +0.3906 mV [00a]    +13.02 mA
S_VS          2.9297 V [12c0] +0.3906 mV [00a]    +13.02 mA<br />
S_VW          1.9922 V [ cc0] -0.1562 mV [ffc]    -5.21 mA
S_VW          1.9922 V [ cc0] -0.1562 mV [ffc]    -5.21 mA
</nowiki>


Read Temperature on Zynq:
Read Temperature on Zynq:
Line 107: Line 31:
Output:  
Output:  
  ZYNQ Temp    49.9545 °C
  ZYNQ Temp    49.9545 °C
=Tools=
==cmv_reg==
Get and Set CMV12000 image sensor registers (CMV12000 sports 128x16 Bit registers).


Details are in the sensor datasheet: https://github.com/apertus-open-source-cinema/beta-hardware/tree/master/Datasheets
==Image Processing Nodes==


===Debayering===


'''Examples:'''
A planned feature is to generate this FPGA code block with "dynamic reconfiguration" meaning that the actual debayering algorithm can be replaced at any time by loading a new FPGA binary block at run-time.


Read register 115 (which contains the analog gain settings):
This tries to simplify creating custom debayering algorithms with a script like programming language that can be translated to FPGA code and loaded into the FPGA dynamically for testing.
cmv_reg 115


Return value:
===Peaking Proposal===
0x00
Means we are currently operating at analog gain x1 = unity gain


[[Peaking]] marks high image frequency areas with colored dot overlays. These marked areas are typically the ones "in-focus" currently so this is a handy tool to see where the focus lies with screens that have lower resolution than the camera is capturing.


'''Handy Custom Parameters:'''


Set register 115 to gain x2:
*color
cmv_reg 115 1
*frequency threshold


==set_gain.sh==
'''Potential Problems:'''


Set gain and related settings (ADC range and offsets).
* there are sharper and softer lenses so the threshold depends on the glass currently used. For a sharp lens the peaking could show areas as "in-focus" if they actually aren't and for softer lenses the peaking might never show up at all because the threshold is never reached


ssh root@cameraip "./set_gain.sh 1"
===Image Blow Up / Zoom Proposal===
ssh root@cameraip "./set_gain.sh 2"
ssh root@cameraip "./set_gain.sh 3/3"


==cmv_snap3==
Digital zoom into the center area of the image to check focus.  
Capture and store image snapshots and sequences.


Example:
As extra feature this zoomed area could be moved around the full sensor area.
ssh root@cameraip "./cmv_snap3 -B0x08000000 -x -N8 -L1024 -2 -e 5ms" >/tmp/test.seq12


-B setzt die buffer base und ist im augenblick erforderlich
[[File:20140909152450-look-around.jpg | 300px]]
0x08000000 bedeutet 128MB, ich hab das linux auf 128MB zusammengepfercht, damit bleibt 1024MB-128MB fuer sequenzen
bitte beachten, es gibt kein sicherheitsnetz und keine plausibilitaetschecks, d.h. wenn du addressen erreichst oder spezifizierst wo das linux laeuft, dann ist vermutlich ein reset notwendig
-x bedeutet das erste frame zu skippen (optional), da mir aufgefallen ist, dass das erste frame nach einer pause anders aussieht
-N setzt die anzahl der frames die zu capturen sind (bitte wieder beachten, wenn du ueber 0x3FFFFFFF kommst, dann ueberschreibst du den linux bereich)
-L setzt die anzahl der zeilen (nur gerade werte, max 3072)
-2 stellt auf 12bit output (einziges format das ich getestet habe :)
-e setzt die exposure wie gehabt


To split one seq12 file into multiple individual files:
This feature is also related to the "Look Around" where the viewfinder sees a larger image area than is being output to the clean-feed.
split -b <size> <file.seq12> <prefix>


<size> the amount of bytes of one image (4096 x number-of-lines x 12bit / 8bit)
This re-sampling method to scale up/down an image in real-time can be of rather low quality (nearest neighbor/bilinear/etc.) as it is only for preview purposes.
<prefix> name
optionally add --additional-suffix=.raw12 for the proper extension
-d or --numeric-suffixes for numeric numbering of the split files


Example for 1024 lines (input file: test.seq12):
== Clipping ==
split -b 6291456 -d test.seq12 output --additional-suffix=.raw12


Example for 1080 lines (input file: test.seq12):
scn_reg 28 0x00 # deactivate clipping<br />
  split -b 6635520 -d test.seq12 output --additional-suffix=.raw12
scn_reg 28 0x10 # activate low clipping<br />
scn_reg 28 0x20 # activate high clipping<br />
  scn_reg 28 0x30 # activate high+low clipping


==Operating System==


Use imagemagick to convert raw12 file into a color preview image:
At this moment we were able to reuse an Arch Linux image for the Zedboard on the Microzed. To do so, some software such as the FSBL and uboot were added.
cat test.raw12 | convert \( -size 4096x3072 -depth 12 gray:- \) \( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \) -sample 2048x1536 \( -clone 2,3 -average \) -delete 2,3 -swap 0,1 +swap -combine test_color.png
More information can be found here: http://stefan.konink.de/contrib/apertus/ I will commit myself on the production of a screencast of the entire bootstrap proces from the Xilinx software to booting the MicroZed.


Capture directly to DNG, without saving the raw12, in the camera:
I would suggest running Arch Linux on the AXIOM Beta for development purposes. If we need to shrink it down that will be quite trivial. Obviously we can take the embedded approach from there, as long as we don't fall in the trap of libc implementations with broken threading.


./cmv_snap3 -2 -b -r -e 10ms | raw2dng snap.DNG
==Pattern noise correction==


=General Info=
--rnfilter=1        : FIR filter for row noise correction from black columns<br />
Stop HDMI live stream:
--rnfilter=2        : FIR filter for row noise correction from black columns and per-row median differences in green channels<br />
  fil_reg 15 0
--fixrn            : Fix row noise by image filtering (slow, guesswork)<br />
Start HDMI live stream:
--fixpn            : Fix row and column noise (SLOW, guesswork)<br />
  fil_reg 15 0x01000100
  --fixrnt            : Temporal row noise fix (use with static backgrounds; recommended)<br />
==Operating System==
--fixpnt            : Temporal row/column noise fix (use with static backgrounds)<br />
  --no-blackcol-rn    : Disable row noise correction from black columns (they are still used to correct static offsets)<br />
--no-blackcol-ff    : Disable fixed frequency correction in black columns


At this moment we were able to reuse an Arch Linux image for the Zedboard on the Microzed. To do so, some software such as the FSBL and uboot were added.
Flat field correction:
More information can be found here: http://stefan.konink.de/contrib/apertus/ I will commit myself on the production of a screencast of the entire bootstrap proces from the Xilinx software to booting the MicroZed.


I would suggest running Arch Linux on the AXIOM Beta for development purposes. If we need to shrink it down that will be quite trivial. Obviously we can take the embedded approach from there, as long as we don't fall in the trap of libc implementations with broken threading.
--dchp              : Measure hot pixels to scale dark current frame<br />
--no-darkframe      : Disable dark frame (if darkframe-xN.pgm is present)<br />
--no-dcnuframe      : Disable dark current frame (if dcnuframe-xN.pgm is present)<br />
--no-gainframe      : Disable gain frame (if gainframe-xN.pgm is present)<br />
--no-clipframe      : Disable clip frame (if clipframe-xN.pgm is present)<br />
--no-blackcol      : Disable black reference column subtraction: - enabled by default if a dark frame is used. - reduces row noise and black level variations<br />
--calc-darkframe    : Average a dark frame from all input files<br />
--calc-dcnuframe    : Fit a dark frame (constant offset) and a dark current frame (exposure-dependent offset) from files with different exposures (starting point: 256 frames with exposures from 1 to 50 ms)<br />
--calc-gainframe    : Average a gain frame (aka flat field frame)<br />
--calc-clipframe    : Average a clip (overexposed) frame<br />
--check-darkframe  : Check image quality indicators on a dark frame


Debug options:


==Userspace==
--dump-regs        : Dump sensor registers from metadata block (no output DNG)<br />
Arch Linux comes with systemd, which has one advantage that the boot process is incredible fast. Standard tools such as sshd and dhcpcd have been preinstalled. We may need other tools such as ftp, webserver, etc.
--fixpn-dbg-denoised: Pattern noise: show denoised image<br />
--fixpn-dbg-noise  : Pattern noise: show noise image (original - denoised)<br />
--fixpn-dbg-mask    : Pattern noise: show masked areas (edges and highlights)<br />
--fixpn-dbg-col    : Pattern noise: debug columns (default: rows)<br />
--export-rownoise  : Export row noise data to octave (rownoise_data.m)<br />
--get-pixel:%d,%d  : Extract one pixel from all input files, at given coordinates, and save it to pixel.csv, including metadata. Skips DNG output.


* ftp; I would suggest vsftpd here
Example:
* webserver; I am able to modify cherokee with custom C code to directly talk to specific camera sections. Cherokee already powers the WiFi module of the GoPro.


One idea to store camera relevant parameters inside the camera and provide access from most programming languages is to use a database like http://en.wikipedia.org/wiki/Berkeley_DB
./raw2dng --fixrnt --pgm --black=120 frame%05d.dng

Latest revision as of 12:09, 14 October 2018

1 Misc Scripts

Display voltages and current flow: 

./pac1720_info.sh

Output: 

ZED_5V        	5.0781 V [2080] 	+29.0625 mV [2e8]   +968.75 mA

BETA_5V       	5.1172 V [20c0] 	+26.6016 mV [2a9]   +886.72 mA

HDN           	3.2422 V [14c0] 	 -0.0391 mV [fff]     -1.30 mA

PCIE_N_V      	3.2422 V [14c0] 	 -0.0391 mV [fff]     -1.30 mA

HDS           	3.2422 V [14c0] 	 +0.0000 mV [000]     +0.00 mA

PCIE_S_V      	3.2422 V [14c0] 	 -0.0391 mV [fff]     -1.30 mA

RFW_V         	3.2812 V [1500] 	 +0.2734 mV [007]     +9.11 mA

IOW_V         	3.2422 V [14c0] 	 +0.0000 mV [000]     +0.00 mA

RFE_V         	3.2812 V [1500] 	 +0.2344 mV [006]     +7.81 mA

IOE_V         	3.2812 V [1500] 	 +0.0781 mV [002]     +2.60 mA

VCCO_35       	2.5000 V [1000] 	 +0.6641 mV [011]    +22.14 mA

VCCO_13       	2.4609 V [ fc0] 	 +1.2500 mV [020]    +41.67 mA

PCIE_IO       	2.4609 V [ fc0] 	 -0.0391 mV [fff]     -1.30 mA

VCCO_34       	2.4609 V [ fc0] 	 +0.8203 mV [015]    +27.34 mA

W_VW          	1.9922 V [ cc0] 	 -0.0781 mV [ffe]     -2.60 mA

N_VW          	3.1641 V [1440] 	 +0.0000 mV [000]     +0.00 mA

N_VN          	1.8750 V [ c00] 	+15.4297 mV [18b]   +514.32 mA

N_VE          	3.1641 V [1440] 	 +0.0000 mV [000]     +0.00 mA

E_VE          	1.9922 V [ cc0] 	 -0.0391 mV [fff]     -1.30 mA

S_VE          	1.9531 V [ c80] 	 +0.0000 mV [000]     +0.00 mA

S_VS          	2.9297 V [12c0] 	 +0.3906 mV [00a]    +13.02 mA

S_VW          	1.9922 V [ cc0] 	 -0.1562 mV [ffc]     -5.21 mA

Read Temperature on Zynq: 

./zynq_info.sh

Output: 

ZYNQ Temp     	49.9545 °C

2 Image Processing Nodes

2.1 Debayering

A planned feature is to generate this FPGA code block with "dynamic reconfiguration" meaning that the actual debayering algorithm can be replaced at any time by loading a new FPGA binary block at run-time.

This tries to simplify creating custom debayering algorithms with a script like programming language that can be translated to FPGA code and loaded into the FPGA dynamically for testing.

2.2 Peaking Proposal

Peaking marks high image frequency areas with colored dot overlays. These marked areas are typically the ones "in-focus" currently so this is a handy tool to see where the focus lies with screens that have lower resolution than the camera is capturing.

Handy Custom Parameters:

  • color
  • frequency threshold

Potential Problems:

  • there are sharper and softer lenses so the threshold depends on the glass currently used. For a sharp lens the peaking could show areas as "in-focus" if they actually aren't and for softer lenses the peaking might never show up at all because the threshold is never reached

2.3 Image Blow Up / Zoom Proposal

Digital zoom into the center area of the image to check focus.

As extra feature this zoomed area could be moved around the full sensor area.

20140909152450-look-around.jpg

This feature is also related to the "Look Around" where the viewfinder sees a larger image area than is being output to the clean-feed.

This re-sampling method to scale up/down an image in real-time can be of rather low quality (nearest neighbor/bilinear/etc.) as it is only for preview purposes.

3 Clipping

scn_reg 28 0x00 # deactivate clipping

scn_reg 28 0x10 # activate low clipping

scn_reg 28 0x20 # activate high clipping

scn_reg 28 0x30 # activate high+low clipping

4 Operating System

At this moment we were able to reuse an Arch Linux image for the Zedboard on the Microzed. To do so, some software such as the FSBL and uboot were added. More information can be found here: http://stefan.konink.de/contrib/apertus/ I will commit myself on the production of a screencast of the entire bootstrap proces from the Xilinx software to booting the MicroZed.

I would suggest running Arch Linux on the AXIOM Beta for development purposes. If we need to shrink it down that will be quite trivial. Obviously we can take the embedded approach from there, as long as we don't fall in the trap of libc implementations with broken threading.

5 Pattern noise correction

--rnfilter=1        : FIR filter for row noise correction from black columns

--rnfilter=2        : FIR filter for row noise correction from black columns and per-row median differences in green channels

--fixrn             : Fix row noise by image filtering (slow, guesswork)

--fixpn             : Fix row and column noise (SLOW, guesswork)

--fixrnt            : Temporal row noise fix (use with static backgrounds; recommended)

--fixpnt            : Temporal row/column noise fix (use with static backgrounds)

--no-blackcol-rn    : Disable row noise correction from black columns (they are still used to correct static offsets)

--no-blackcol-ff    : Disable fixed frequency correction in black columns

Flat field correction: 

--dchp              : Measure hot pixels to scale dark current frame

--no-darkframe      : Disable dark frame (if darkframe-xN.pgm is present)

--no-dcnuframe      : Disable dark current frame (if dcnuframe-xN.pgm is present)

--no-gainframe      : Disable gain frame (if gainframe-xN.pgm is present)

--no-clipframe      : Disable clip frame (if clipframe-xN.pgm is present)

--no-blackcol       : Disable black reference column subtraction: - enabled by default if a dark frame is used. - reduces row noise and black level variations

--calc-darkframe    : Average a dark frame from all input files

--calc-dcnuframe    : Fit a dark frame (constant offset) and a dark current frame (exposure-dependent offset) from files with different exposures (starting point: 256 frames with exposures from 1 to 50 ms)

--calc-gainframe    : Average a gain frame (aka flat field frame)

--calc-clipframe    : Average a clip (overexposed) frame

--check-darkframe   : Check image quality indicators on a dark frame

Debug options: 

--dump-regs         : Dump sensor registers from metadata block (no output DNG)

--fixpn-dbg-denoised: Pattern noise: show denoised image

--fixpn-dbg-noise   : Pattern noise: show noise image (original - denoised)

--fixpn-dbg-mask    : Pattern noise: show masked areas (edges and highlights)

--fixpn-dbg-col     : Pattern noise: debug columns (default: rows)

--export-rownoise   : Export row noise data to octave (rownoise_data.m)

--get-pixel:%d,%d   : Extract one pixel from all input files, at given coordinates, and save it to pixel.csv, including metadata. Skips DNG output.

Example: 

./raw2dng --fixrnt --pgm --black=120 frame%05d.dng
Retrieved from "https://wiki.apertus.org/index.php?title=AXIOM_Beta/AXIOM_Beta_Software&oldid=7307"