Hamamatsu C15550-20UP support and multichannel CoaXPress grabber update

morgan

With the new multichannel CoaXPress grabber gateware, camera that needs multiple CoaXPress channels such as the Hamamatsu C15550-20UP is now supported.
During testing, I faced a couple of issue when setting up the Hamamatsu C15550-20UP, so only the following is tested and confirmed working for now.

The initial camera startup process is working correctly with some firmware workarounds
Frame triggering in free running mode trigger (similar to software trigger) is working. For the deterministic CXPLinkTrigger, I am in talks with Hamamtsu to figure how to setup the camera to accept CXPLinkTrigger.
Using ROI and ROIViewer, noise pattern was observed when capture an image. Due to the lack of C-mount lens at the moment, no focused image was captured.

As the time of writing, some of the CoaXPress grabber update PRs were merged to ARTIQ 10 (current BETA). You can follow the PR at

Mahmoud_Badawy_uWaterloo

Hi Morgan,

I'm working on integrating a Hamamatsu C15550-20UP with Kasli-SoC and the CoaXPress-SFP adapter. I saw your notes mentioning CXPLinkTrigger acceptance was still being resolved with Hamamatsu.

Did you make any progress on getting the camera to accept deterministic CXPLinkTrigger? If not, can I use the camera's external SMA input driven by an ARTIQ TTL output instead?

Any updates or guidance would be appreciated. Thank you very much for your efforts.

Mahmoud_Badawy_uWaterloo

Also, what is the correct cable to use? Is it DIN 1.0/2.3 to HD-BNC?

morgan

Did you make any progress on getting the camera to accept deterministic CXPLinkTrigger? If not, can I use the camera's external SMA input driven by an ARTIQ TTL output instead?

Unfortunately no progress so far as I'm waiting for Hamamtsu reply. Personally I haven't tested the SMA (Line0) trigger input but from the C15550-20UP Instruction manual it's unclear what TTL voltage level the camera can accept "External trigger input level TTL / 3.3 V LVCMOS", I will suggest asking Hamamtsu for clarification before connecting the ARTIQ 5V TTL to the camera.

Also, what is the correct cable to use? Is it DIN 1.0/2.3 to HD-BNC?

Yes DIN 1.0/2.3 (C15550-20UP side) to HD-BNC (CoaXPress-SFP adapter side) rated for CXP-6 will work. In my test setup, I used DIN 1.0/2.3 to BNC cables provided by Hamamatsu with Amphenol 031-70540-12G BNC to HD-BNC adapters.

Btw C15550-20UP requires four CXP-6 channels to work properly, so you will need ARTIQ 10 (current BETA), a Kasli-SoC and four CoaXPress-SFP adapters.

Mahmoud_Badawy_uWaterloo

Hi Morgan,

Thanks again for all your help.

I have done the following:

Built and flashed ARTIQ 10 (NAC3) gateware+firmware for Kasli-SoC v1.2 with the 4-channel coaxpress_sfp peripheral (on Vivado 2022.2).
Camera connects on all 4 channels — the C15550-20UP non-compliant workaround runs, all four channels pass the connection test, "camera setup complete", LEDs green.
Control plane confirmed: read32(0x0000) returns 0xC0A79AE5 (CoaXPress magic number).
Pulled the camera XML with read_local_xml().
Read the registers it exposes: AcqEnReg @ 0x8000 (AcquisitionStart), AcqStopReg @ 0x8004, ExposureTimeReg_Value @ 0xE204003C (reads 1152 → ₈.3 ms with the Mult/Div converter), Width/Height read 4096/2304.

Where I'm stuck is no frame: Writing AcqEnReg = 0x1 alone produces no frame (no received frame: 4096x2304 in the coremgmt log), and I confirmed on my end too that CXPLinkTrigger isn't accepted by this camera.

You used a software trigger, but the XML this camera produce is minimal. it only exposes DeviceControl / ImageFormatControl / AcquisitionControl, with AcquisitionMode = Continuous and Start/Stop, and no trigger register at all (no TriggerSource/TriggerMode/software-trigger).

How did you fire the software trigger over CoaXPress on this camera?

morgan

Hello,

Glad to hear the camera setup is OK on your end. That camera has so many software bugs and non-compliant issues I need to bypass in order to receive a frame. Also Hamamatsu mentioned they have no plan to fix them in later firmware version.

The trigger source entry is called "TriggerSelector" and the software trigger is called "AcquisitionStart". I called it software trigger because "AcquisitionStart" is triggered by writing a register (or "command" in GenICam terms) so for the confusion.

Here's the XML of my camera for your reference, maybe they axed some entries in later firmware.
https://files.m-labs.hk/s/gFye7skJGRsQksZ

And my (very messy) software trigger test kernel during development.
https://git.m-labs.hk/morgan/artiq-zynq/src/commit/51f4d5c79c9edd9ce97d17cae806ed6b811876e3/nac3_hamamatsu_cxp_test_kernel.py

Mahmoud_Badawy_uWaterloo

This is XML of my camera. It's way shorter than your XML. Looks like the versions are different.

<?xml version="1.0" encoding="utf-8"?>
<RegisterDescription
ModelName="C15550_20UP"
VendorName="HAMAMATSU"
StandardNameSpace="None"
SchemaMajorVersion="1" SchemaMinorVersion="1" SchemaSubMinorVersion="0"
MajorVersion="1" MinorVersion="0" SubMinorVersion="5"
ProductGuid="C20B1A45-C313-43AE-996D-731170CB9CEC"
VersionGuid="D8126BA2-B065-4115-90C8-9C2549775D92"
xmlns="http://www.genicam.org/GenApi/Version_1_1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.genicam.org/GenApi/Version_1_1
http://www.genicam.org/GenApi/GenApiSchema_Version_1_1.xsd">

<Category Name="Root">
<pFeature>DeviceControl</pFeature>
<pFeature>ImageFormatControl</pFeature>
<pFeature>AcquisitionControl</pFeature>
</Category>

<Port Name="Device"></Port>
<Integer Name="TLParamsLocked">
<Visibility>Invisible</Visibility>
<Value>0</Value>
<Min>0</Min>
<Max>1</Max>
</Integer>

<Category Name="DeviceControl">
<pFeature>DeviceScanType</pFeature>
</Category>

<Category Name="ImageFormatControl">
<pFeature>Width</pFeature>
<pFeature>Height</pFeature>
<pFeature>PixelFormat</pFeature>
<pFeature>TapGeometry</pFeature>
<pFeature>Img1StreamId</pFeature>
</Category>

<Category Name="AcquisitionControl">
<pFeature>AcquisitionMode</pFeature>
<pFeature>AcquisitionStart</pFeature>
<pFeature>AcquisitionStop</pFeature>
<pFeature>ExposureTime</pFeature>
</Category>

<Enumeration Name="DeviceScanType" NameSpace="Standard">
<DisplayName>Scan Type</DisplayName>
<Visibility>Expert</Visibility>
<EnumEntry Name="Areascan">
<Visibility>Invisible</Visibility>
<Value>0x00000000</Value>
</EnumEntry>



<Value>0x00000000</Value>
</Enumeration>

<Integer Name="Width" NameSpace="Standard">
<Visibility>Beginner</Visibility>
<pValue>ImageSizeReg_Width</pValue>
<Representation>Linear</Representation>
</Integer>
<IntReg Name="ImageSizeReg_Width">
<Address>0xE2020098</Address>
<Length>4</Length>

<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<Integer Name="Height" NameSpace="Standard">
<Visibility>Beginner</Visibility>
<pValue>ImageSizeReg_Hight</pValue>
<Representation>Linear</Representation>
</Integer>
<IntReg Name="ImageSizeReg_Hight">
<Address>0xE20200A0</Address>
<Length>4</Length>

<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<Command Name="AcquisitionStart" NameSpace="Standard">
<DisplayName>Acquisition Start</DisplayName>
<Visibility>Beginner</Visibility>
<pValue>AcqEnReg</pValue>
<CommandValue>0x1</CommandValue>
</Command>
<IntReg Name="AcqEnReg">
<Address>0x8000</Address>
<Length>4</Length>
<AccessMode>WO</AccessMode>
<pPort>Device</pPort>
<Endianess>BigEndian</Endianess>
</IntReg>

<Command Name="AcquisitionStop" NameSpace="Standard">
<DisplayName>Acquisition Stop</DisplayName>
<Visibility>Beginner</Visibility>
<pValue>AcqStopReg</pValue>
<CommandValue>0x1</CommandValue>
</Command>
<IntReg Name="AcqStopReg">
<Address>0x8004</Address>
<Length>4</Length>
<AccessMode>WO</AccessMode>
<pPort>Device</pPort>
<Endianess>BigEndian</Endianess>
</IntReg>

<Float Name="ExposureTime" NameSpace="Standard">
<DisplayName>ExposureTime</DisplayName>
<Visibility>Beginner</Visibility>
<pValue>ExposureTimeValue</pValue>
<pMin>ExposureTimeMin</pMin>
<pMax>ExposureTimeMax</pMax>
<pInc>ExposureTimeInc</pInc>
<Unit>us</Unit>
<Representation>PureNumber</Representation>
<DisplayNotation>Fixed</DisplayNotation>
<DisplayPrecision>3</DisplayPrecision>
</Float>

<SwissKnife Name="ExposureTimeInc">
<pVariable Name="MULT">ExposureTimeReg_Mult</pVariable>
<pVariable Name="DIV">ExposureTimeReg_Div</pVariable>
<Formula> (MULT * 1000000) / DIV </Formula>
</SwissKnife>

<SwissKnife Name="ExposureTimeMin">
<pVariable Name="MULT">ExposureTimeReg_Mult</pVariable>
<pVariable Name="DIV">ExposureTimeReg_Div</pVariable>
<pVariable Name="MIN">ExposureTimeReg_Min</pVariable>
<Formula> (MIN * MULT * 1000000) / DIV </Formula>
</SwissKnife>

<SwissKnife Name="ExposureTimeMax">
<pVariable Name="MULT">ExposureTimeReg_Mult</pVariable>
<pVariable Name="DIV">ExposureTimeReg_Div</pVariable>
<pVariable Name="MAX">ExposureTimeReg_Max</pVariable>
<Formula> (MAX * MULT * 1000000) / DIV </Formula>
</SwissKnife>

<Converter Name="ExposureTimeValue">
<pVariable Name="MULT">ExposureTimeReg_Mult</pVariable>
<pVariable Name="DIV">ExposureTimeReg_Div</pVariable>
<FormulaTo> (FROM * DIV) / (MULT * 1000000) </FormulaTo> 
<FormulaFrom> (TO * MULT * 1000000) / DIV </FormulaFrom> 
<pValue>ExposureTimeReg_Value</pValue>
<Slope>Increasing</Slope>
</Converter>

<IntReg Name="ExposureTimeReg_Mult">
<Visibility>Invisible</Visibility>
<Address>0xE204002C</Address>
<Length>4</Length>
<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<IntReg Name="ExposureTimeReg_Div">
<Visibility>Invisible</Visibility>
<Address>0xE2040030</Address>
<Length>4</Length>
<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<IntReg Name="ExposureTimeReg_Min">
<Visibility>Invisible</Visibility>
<Address>0xE2040034</Address>
<Length>4</Length>
<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<IntReg Name="ExposureTimeReg_Max">
<Visibility>Invisible</Visibility>
<Address>0xE2040038</Address>
<Length>4</Length>
<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<IntReg Name="ExposureTimeReg_Value">
<Visibility>Invisible</Visibility>
<Address>0xE204003C</Address>
<Length>4</Length>
<AccessMode>RW</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<Enumeration Name="AcquisitionMode" NameSpace="Standard">
<DisplayName>Acquisition Mode</DisplayName>
<Visibility>Beginner</Visibility>
<EnumEntry Name="Continuous">
<Visibility>Invisible</Visibility>
<Value>0x00000001</Value>
</EnumEntry>
<Value>0x00000001</Value>
</Enumeration>

<Enumeration Name="PixelFormat" NameSpace="Standard">
<DisplayName>Pixel Format</DisplayName>
<Visibility>Beginner</Visibility>
<EnumEntry Name="Mono8">
<Visibility>Invisible</Visibility>
<Value>8</Value>
</EnumEntry>
<EnumEntry Name="Mono12">
<Visibility>Invisible</Visibility>
<Value>12</Value>
</EnumEntry>
<EnumEntry Name="Mono16">
<Visibility>Invisible</Visibility>
<Value>16</Value>
</EnumEntry>
<pValue>PixelSizeReg</pValue>
</Enumeration>
<IntReg Name="PixelSizeReg">
<Visibility>Invisible</Visibility>
<Address>0xE20200FC</Address>
<Length>4</Length>

<AccessMode>RO</AccessMode>
<pPort>Device</pPort>
<Sign>Unsigned</Sign>
<Endianess>BigEndian</Endianess>
</IntReg>

<Enumeration Name="TapGeometry">
<Visibility>Beginner</Visibility>
<EnumEntry Name="Cxp_1x_1y">
<Value>0</Value>
</EnumEntry>
<Value>0</Value>
</Enumeration>

<Enumeration Name="Img1StreamId">
<Visibility>Beginner</Visibility>
<EnumEntry Name="Stream1Id">
<Value>1</Value>
</EnumEntry>
<Value>1</Value>
</Enumeration>

</RegisterDescription>

Mahmoud_Badawy_uWaterloo

I get many Invalid Address errors as we have different firmware versions. Not sure what the next steps should be. I still haven't got any frames.

morgan

You should reach out to Hamamatsu to see whether this is a firmware bug and fixable via a firmware upgrade. It's very strange the XML is missing a lot feature settings (Ultra quiet scan, lightsheet, external trigger setting) advertise in their technical notes and instruction manual.

Mahmoud_Badawy_uWaterloo

Hi Morgan,

Quick update on the Hamamatsu CoaXPress integration. I figured out why my scripts were halting earlier: I was accidentally testing the newer ORCA-Quest II (C15550-22UP) thinking it was the older C15550-20UP!

To test this, I borrowed a real C15550-20UP from another experiment in the lab, and my code works perfectly! Because the camera is locked in continuous mode, I wrote a "Hit and Run" script that opens the FPGA ROI gate, sends AcquisitionStart, waits 1 ms, and instantly sends AcquisitionStop. This successfully bypasses the Kasli-SoC buffer overflow, and I am able to reliably pull both the fast pixel sums (input_mu) and 50x50 2D frames using ROIViewer.

Note on the XML: I noticed that the XML manifest dumped from this borrowed 20UP is actually missing a lot of the details and trigger controls that are present in your XML file. It seems both cameras are running a stripped-down firmware. I have already sent an email to Hamamatsu support to ask them about these missing registers.

I need to get the C15550-22UP working for my actual setup, but the exact same code halts on the 22UP when trying to read the FPGA FIFO.

Here is what I’ve diagnosed on the 22UP so far:

The LEDs blink rapidly, meaning it is successfully exposing and pushing data down the CoaXPress cable.

I polled the GenICam registers, and it is outputting the exact same format as the 20UP (4096x2304, 16-bit Mono).

I dumped the XML manifest, and Hamamatsu literally copy-pasted the stripped-down 20UP's firmware onto it (it even still says ModelName="C15550_20UP").

Since the Python code hangs while the camera is transmitting, it seems the Kasli-SoC gateware is likely rejecting the 22UP's incoming data before it ever reaches my script. I suspect something subtle changed in the data stream that the current CoaXPress receiver doesn't recognize.

I want to take a crack at integrating the C15550-22UP myself, but I don't know a lot of the low-level details yet. Could you give me a general guideline on how to approach this?

Specifically, how would I go about debugging what is actually getting dropped, and which parts of the ARTIQ codebase would I need to touch to add support for it? I’d love to dive in if you can point me in the right direction.

Thanks!

morgan

Assuming Hamamatsu LED's follow the CXP standard and C15550-22UP is blinking Green at 12.5 Hz, it should be sending the stream data payload to SoC.

To debug this, we need to

familiarize ourselves with CoaXPress gateware layout
familiarize ourselves with the stream data payload from the CXP standard (CXP V2.1 standard CXP-001-2021)
- Namely 9.5 Stream Data Packet and 10.4.6.2 Rectangular Image Heade
capture the stream data payload packet to see whether the packet is CXP compilant and paste the log here so we can decode it together.

Gateware layout

CrossBarSwitch
- This module aggregates multiple CXP channels together and form a continuous stream data for downstream consumption (i.e. dropping the Stream Packet Header (SPH) and Stream Packet Trailer (SPT))
- One thing to note is that the CrossBarDispatch will drop stream data with a mismatch stream_id
- the firmware set the stream_id we want and the gateware will drop the rest
- The C15550_20UP I had required bypassing this check as the stream_id keep changing each packet it sent.
ImageDecoder
- This module only decodes Rectangular Image Stream and unpack the pixel for ROI consumption
- ImageHeaderReader module read the header and send it to firmware. So we can printout received frame: 4096x2304 with MONO16 format
- If this module cannot decode the header (e.g. the camera send deprecated Arbitrary Image Stream), the log won't show received frame... and the downstream ROIs will keep waiting for the pixel to be decoded and unpacked.

Packet capture

I cherry-pick a LogicAnalzyer buffer gateware I used during development. And I already prepare some suspect data_sink we might want to capture. Changing the data_sink requires a full gateware rebuild so this might take a while to get what we want to see (e.g. a non-compliant packet). Let me annotate the uart log of triggering a CXP compliant Balser boa2448-250cm (Two CXP-12 channel) and data_sink = cls.cxp_grabber.stream_decoder.switch.arbiter.source

[    21.691757s] DEBUG(libboard_artiq::cxp_camera_setup): camera detected at linerate 3.125 Gbps
[    21.694543s]  INFO(libboard_artiq::cxp_grabber): camera detected, setting up camera...
[    21.979669s] DEBUG(libboard_artiq::cxp_camera_setup): camera recommends using 2 channels
[    22.183613s] DEBUG(libboard_artiq::cxp_camera_setup): set host connection id to = 0x6303
[    22.186217s] DEBUG(libboard_artiq::cxp_camera_setup): camera's CoaXPress revision is 2.0
[    22.194269s] DEBUG(libboard_artiq::cxp_camera_setup): both camera and cxp grabber support CoaXPress 2.0, switch to CoaXPress 2.0 protocol now
[    22.207332s] DEBUG(libboard_artiq::cxp_camera_setup): changing linerate to 12.5 Gbps
[    22.416643s] DEBUG(libboard_artiq::cxp_camera_setup): image1 stream id = 0x0
[    22.579305s] DEBUG(libboard_artiq::cxp_camera_setup): camera channel #0 sent 29513 test packets, CXP grabber received 29513 packets
[    22.585476s] DEBUG(libboard_artiq::cxp_camera_setup): channel #0 passed connection test
[    22.593697s] DEBUG(libboard_artiq::cxp_camera_setup): camera channel #1 sent 29513 test packets, CXP grabber received 29513 packets
[    22.605266s] DEBUG(libboard_artiq::cxp_camera_setup): channel #1 passed connection test
[    22.613251s]  INFO(libboard_artiq::cxp_grabber): camera setup complete
[    25.007695s] DEBUG(ksupport::kernel::core1): Core1 started
[    25.007745s] DEBUG(dyld): ELF target: 5272 bytes, align to 1000, allocated at 082BE000
[    25.015442s] DEBUG(dyld): Relocating 672 rela, 0 rel, 0 pltrel
[    25.021323s] DEBUG(ksupport::kernel::core1): kernel loaded
[    25.026893s]  INFO(ksupport::kernel::core1): kernel starting
[    26.222781s]  INFO(ksupport::kernel::core1): kernel finished
[    26.240553s] DEBUG(ksupport::kernel::core1): Core1 started
[    26.240581s] DEBUG(dyld): ELF target: 1360 bytes, align to 1000, allocated at 082BD000
[    26.248289s] DEBUG(dyld): Relocating 132 rela, 0 rel, 0 pltrel
[    26.254122s] DEBUG(ksupport::kernel::core1): kernel loaded
[    26.259867s]  INFO(ksupport::kernel::core1): kernel starting
[    26.265646s]  INFO(ksupport::kernel::core1): kernel finished
[    26.271034s]  INFO(runtime::comms): peer closed connection
[    26.276325s]  INFO(runtime::comms): no idle kernel found
[    26.281627s]  INFO(libboard_artiq::cxp_grabber): received CXP linktrigger ack
[    26.288737s]  INFO(libboard_artiq::cxp_grabber): received frame: 2448x2048 with MONO8 format
Logic analzyer receive data
0x0000 : 0x00 0x00 0x00 0x00* # <====== new payload, StreamID = 0
0x0001 : 0x00 0x00 0x00 0x00  # PacketTag = 0
0x0002 : 0x00 0x00 0x00 0x00 
0x0003 : 0x04 0x04 0x04 0x04  # DsizeP = 0x0004 words (32-bits = 1 word)
0x0004 : 0x7c 0x7c 0x7c 0x7c* # Stream marker K28.3
0x0005 : 0x01 0x01 0x01 0x01  # Rectangular Image Header Indication
0x0006 : 0x00 0x00 0x00 0x00  # StreamID = 0
0x0007 : 0x00 0x00 0x00 0x00  # SourceTag(15:8)
0x0008 : 0x00 0x00 0x00 0x00* # <====== new payload, StreamID = 0
0x0009 : 0x01 0x01 0x01 0x01  # PacketTag = 1
0x000a : 0x00 0x00 0x00 0x00  
0x000b : 0x08 0x08 0x08 0x08  # DsizeP = 0x0008 words
0x000c : 0x00 0x00 0x00 0x00* # SourceTag(7:0)
0x000d : 0x00 0x00 0x00 0x00 
0x000e : 0x09 0x09 0x09 0x09 
0x000f : 0x90 0x90 0x90 0x90  # Xsize = 0x00990 = 2448, same as "received frame" :D
0x0010 : 0x00 0x00 0x00 0x00*
0x0011 : 0x00 0x00 0x00 0x00 
0x0012 : 0x00 0x00 0x00 0x00  # Xoffs = 0
0x0013 : 0x00 0x00 0x00 0x00  # Ysize(23:16)
0x0014 : 0x00 0x00 0x00 0x00* # <====== new payload, StreamID = 0
0x0015 : 0x02 0x02 0x02 0x02  # PacketTag = 2
0x0016 : 0x00 0x00 0x00 0x00 
0x0017 : 0x0d 0x0d 0x0d 0x0d  # DsizeP = 0x000d = 13
0x0018 : 0x08 0x08 0x08 0x08* # Ysize(15:8)
0x0019 : 0x00 0x00 0x00 0x00  # Ysize = 0x000800 = 2048, same as "received frame" :D
0x001a : 0x00 0x00 0x00 0x00  
0x001b : 0x00 0x00 0x00 0x00 
0x001c : 0x08 0x08 0x08 0x08* # Yoffs = 0x0008
0x001d : 0x00 0x00 0x00 0x00 
0x001e : 0x02 0x02 0x02 0x02 
0x001f : 0x64 0x64 0x64 0x64  # DsizeL = 0x000264 = 612 words, 612*32-bit = 19584 bits per line == Xsize * pixel format = 2448*8-bit = 19584 bits per line correct :D
0x0020 : 0x01 0x01 0x01 0x01*
0x0021 : 0x01 0x01 0x01 0x01  # PixelIF = 0x0101, MONO8 same as "received frame" :D
0x0022 : 0x00 0x00 0x00 0x00 
0x0023 : 0x00 0x00 0x00 0x00  # TapG = 0x0000 = 1X-1Y
0x0024 : 0x00 0x00 0x00 0x00* # Flags = 0x00
0x0025 : 0xde 0xad 0x00 0x01  # This is the 0xDEAD padding, 0x01 means only 1 words is valid, i.e. the Flas
0x0026 : 0x00 0x00 0x00 0x00  # <====== new payload, StreamID = 0 
0x0027 : 0x03 0x03 0x03 0x03  # PacketTag = 3
0x0028 : 0x00 0x00 0x00 0x00*
0x0029 : 0x2a 0x2a 0x2a 0x2a  # DsizeP = 0x002a
0x002a : 0x7c 0x7c 0x7c 0x7c  <- distance between last KCode = 38
0x002b : 0x02 0x02 0x02 0x02 
0x002c : 0xff 0xff 0xff 0xff*
0x002d : 0xff 0xff 0xff 0xff 
0x002e : 0xff 0xff 0xff 0xff 
0x002f : 0xff 0xff 0xff 0xff 
0x0030 : 0xff 0xff 0xff 0xff*
0x0031 : 0xff 0xff 0xff 0xff 
0x0032 : 0xff 0xff 0xff 0xff 
0x0033 : 0xff 0xff 0xff 0xff 
0x0034 : 0xff 0xff 0xff 0xff*
0x0035 : 0xff 0xff 0xff 0xff 
0x0036 : 0xff 0xff 0xff 0xff 
0x0037 : 0xff 0xff 0xff 0xff 
0x0038 : 0xff 0xff 0xff 0xff*
0x0039 : 0xff 0xff 0xff 0xff 
0x003a : 0xff 0xff 0xff 0xff 
0x003b : 0xff 0xff 0xff 0xff 
0x003c : 0xff 0xff 0xff 0xff*
...

Mahmoud_Badawy_uWaterloo

Hi Morgan,

Thanks a lot for your efforts.

Using your notes, I looked at the CrossBarSwitch stream-ID filtering. During the handshake, the 22UP reports image1 stream id = 0x0, so the firmware enables the strict filter (passthrough = 0). But when the 22UP actually transmits video, its stream-ID tags were most probably erratic/out-of-spec, so the CrossBarDispatch drops every frame, which is why my script hung waiting on the FIFO while the camera was clearly transmitting. It's the same non-compliance you hit on your 20UP (stream_id changing per packet), just masked on mine because the 22UP does report a valid ID at setup, so the firmware doesn't fall back to passthrough on its own.

I forced csr::cxp_grabber::stream_decoder_stream_passthrough_write(1) unconditionally in cxp_camera_setup (keeping the ID filter off even when the camera reports a valid ID). After rebuild + reflash, the UART immediately showed:

INFO(libboard_artiq::cxp_grabber): received frame: 4096x2304 with MONO16 format

The ImageDecoder unpacks the stream fine, so the header format is compliant, it was purely the stream-ID filter dropping the packets, not a decode problem. So I don't think I even need the LogicAnalyzer packet capture at this point.

Now the questions that remains are:

what is your feedback on that?
can this go into ARTIQ upstream so I don't diverge?
Right now this is a local firmware patch (hard-forcing passthrough), and I'd really rather not maintain a fork that drifts from the ARTIQ codebase. Is there a way to handle this upstream? for example, a JSON option or a per-camera override to force stream-ID passthrough on, for cameras that report a valid ID at setup but then send non-compliant tags? That would cover both your 20UP (changing IDs) and my 22UP (valid-then-corrupted IDs) without anyone hand-editing the firmware.

Thanks again. Your gateware explanation saved me a lot of time.

morgan

Nice!🎉 Glad the stream_id bypass works on your camera. Seems like 20UP and 22UP used the same gateware for preparing the streams packets and the workaround applies to both cameras.

just masked on mine because the 22UP does report a valid ID at setup

Actually 20UP also report a "valid stream id" of image1 stream id = 0x00, but the camera detection trigger properly on 20UP and branch off to implementing the workaround instead of using the unless stream id.

As for upstreaming a proper workaround, we can reuse the camera detection logic for implementing the 20UP workaround. As this logic didn't trigger on 22UP, there is a good chance the content DEVICE_MODEL_NAME 0x2020 register is different.

To add a new match arm for the 22UP,
1) we first need to compile a fw that prints the register content as ascii string. Adding debug!("{:?}", unsafe { str::from_utf8_unchecked(&bytes) }.split("\0").next()) after async_read_bytes should work.
2) Add a new match arm in fw. Let say the string we got from step (1) is "C15550-22UP", we can add

Some(name @ "C15550-22UP") => {
            debug!(
                "{} is a non-compliant CoaXPress camera, implementing workaround...",
                name
            );
            Ok((false, None))
        }

The false here means the don't ignore test_channels_stability result, as 22UP setup without any issue (At least there are some fw improvement compare to 20UP, the 20UP TestPacketCountTx[3] register value will turn bad when changing speed to CXP6)
The None here means no stream_id and it will trigger stream_id workaround logic.

To upstream this patch, we have two options
1) If you have a gitea account, you can raise a PR on artiq-zynq repo and I will do some cleanup and add some comment if necessary to get it merged.
2) I will raise a PR and credit you as a co-author. Then you can test out the PR patch. If it's OK, we will merge it.

Guest

Hi Morgan,

Apologies for the delay in getting back to you on this!

I finally had the chance to test it, and your proposed code was exactly right. I dropped in your Some(name @ "C15550-22UP") => { ... Ok((false, None)) } match arm, and I am now getting a stable stream of 4096x2304 MONO16 frames!

Just to confirm your suspicions from testing:

The exact string reported by the camera is indeed "C15550-22UP".

You were right about the stability tests, they pass perfectly now. My UART logs showed all 4 channels sending and receiving exactly 25,908 packets with zero errors at CXP6.

Even though it passes the test and reports a valid ID at setup, it still requires your None stream ID passthrough workaround to actually receive the frames without the gateware dropping them.

Since your code snippet was spot on, could you go ahead and open the PR for this and I will test it and report the UART logs to the PR?

I also have two quick questions as we are hoping to run some experiments involving the camera soon:
Is it possible to run this setup at 12 Gbps (CXP-12) instead of 6 Gbps (CXP-6)?
Are we currently able to use ARTIQ 10 reliably in production for these experiments?

Thanks again for the help getting this running!

Mahmoud_Badawy_uWaterloo

opps looks like I wasn't logged in. @morgan

morgan

Thanks for testing out the code snippet. Great to hear the code snippet works 😃. I opened a PR to add the necessary workaround https://git.m-labs.hk/M-Labs/artiq-zynq/pulls/494 and credited you as co-author. Can you help me test the PR and report back if it's ok or not.

Is it possible to run this setup at 12 Gbps (CXP-12) instead of 6 Gbps (CXP-6)?

Since C15550-22UP only support up to CXP-6, we cannot run this setup at CXP-10 or CXP-12. If you got a new camera that needs four channels of CXP-12, we only need to set "max_cxp_speed" : "CXP-12" in the json and re-compile the gateware.

From my experience, getting four channels of CXP-12 to pass timing in vivado is flaky (some json works but some doesn't). So I will suggest against setting "max_cxp_speed" : "CXP-12" all the time unless the camera you are using really need the CXP-12.

Are we currently able to use ARTIQ 10 reliably in production for these experiments?

We are introducing the new compiler NAC3 on ARTIQ 10 (BETA) please try it out for your experiments and if you face any issue please let us know and we will fix it asap.

Mahmoud_Badawy_uWaterloo

Hi Morgan,

Thanks for setting up the PR and for the co-author credit! I pulled the branch and tested the patch on our setup. I can confirm it works perfectly. A picture of two ions is attached.

By the way, I have a question about the specific timing during an experiment. Let's suppose the camera fires and I receive a frame. How much time does it actually take for the Kasli to process and give me the ROI viewer array or the intensity count? Does waiting for that data block the ARTIQ timeline, and how should we account for that delay in our code?

I tried to benchmark the image rate, and I get an image every 80 ms. I know for sure that at least 60 ms of these 80 ms is coming from the camera. But the remaining 20 ms is mysterious to me. I still have to characterize the intensity sum. But can the ROI viewer take 20 ms to fetch the data into the kernel?

Typo in the image should be 50x40

Klea_Dhimitri_HamamatsuCorp

Hi Everyone,

I wanted to add that in order to use the new update from M-labs on the multichannel CoaXPress grabber, you will need a GenICam option installed into your ORCA QUEST cameras.

How to know if your camera has the GenICam option:
• If your current software can display the firmware version of the camera, there should be a “G” as a prefix in the camera firmware Version if your camera has GenICam. i.e. – “G3.10”
• If you cannot find it –
o Install the latest DCAM-API for Windows, USB Camera module on any Windows 11 Intel x64 PC which has USB 3.0 or newer ports – such as a portable laptop.
o Power off the camera, unplug all CoaXPress (CXP) cables to the camera temporarily, and connect the Quest to a USB 3.x port.
o Open Device Manager and confirm the “Hamamatsu C15550” shows properly and running under USB Serial bus devices section.
o Run the DCAM Configurator tool – it is inside the DCAM-API download. You can access it from Setup main installer program under Tools.
o Under “Hardware” when the camera is detected, there should be a “G” as a prefix in the camera firmware Version if your camera has GenICam. i.e. – “G3.10”
o Close DCAM Configurator, power off the camera, disconnect the USB, reconnect the CXP, then continue.

How do I upgrade my camera to GenICam:
• Reachout to your local Hamamatsu Photonics Subsidiary and ask for a quote for a GenICam upgrade. The GenICam upgrade will be available to purchase. Please be sure to provide the camera part number and camera serial number as the upgrade will serial number specific.
• GenICam upgrade part number will be M17492-01/ M17492-01A for ORCA QUEST Camera (C15550-20UP)
• GenICam upgrade part number will be M17492-02/ M17492-02A for ORCA QUEST 2 Camera (C15550-22UP)

If you want to upgrade your ORCA QUEST camera to use GenICam contact your local Hamamatsu Photonics Subsidiary for further quoting and assistance.

I will note that here in at the Hamamatsu Corporation office in NJ we have ARTIQ hardware but, we have not yet validated the new CoaXpress grabber board to see if it works with the ORCA QUEST and ORCA QUEST 2 Camera.

We will validate operation once the board is commercially readily available.

morgan

Great news, the C15550-22UP workaround PR is merged.🎉

Let's suppose the camera fires and I receive a frame. How much time does it actually take for the Kasli to process and give me the ROI viewer array or the intensity count?

The rough process time (ignoring the packet overhead and assume the bus is always full) formula:

(20 cycles + 1 cycle * image_decoder_ratio + total_pixel_count * pixel_size / (32 * num of cxp channels)) * cxp_gt_rx period
In C15550-22UP case, it will be (20 cycles + 2 cycles + 1.179*10⁶ cycle) * (6.4 ns) = 7.55 ms

The exact timing depends on alot of factors but it is constant between frames with the same setup.

how the camera splits the pixel data stream
- the SPH and SPT increase the link overhead and reduce pixel data rate

the line rate the CXP is operating in
- freq of CXP IP core (cxp_gt_rx clockdomain) changed based on the link, freq of cxp_gt_rx = linerate / 40 bit (e.g. CXP-6/40 = 6.25 Gbps/40 = 156.25 MHz)
the pixel format
- The ImageDecoder pixel unpacker is actually a gearbox and the ratio depends on the pixel format
- The ROI engines and ROIViewer takes in 4 * num of cxp channels pixels per cxp_gt_rx cycle but the ImageDecoder bus width is 32 * num of cxp channels. So the ratio would be
- 8-bit (MONO-8): 1:1
- 10-bit (MONO-10): 1:1.25
- 12-bit (MONO-12): 1:1.5
- 14-bit (MONO-14): 1.1.75
- 16-bit (MONO-16): 1:2

Does waiting for that data block the ARTIQ timeline, and how should we account for that delay in our code?

The input_mu will block the ARTIQ timeline, usually I advance the timeline before calling input_mu.

I know for sure that at least 60 ms of these 80 ms is coming from the camera. But the remaining 20 ms is mysterious to me.

The ROI Viewer is designed for adjusting focusing and not designed to be fast or real time monitoring. Basically the viewer is a 4096 pixel FIFO that connects to the CPU CSR bus and CPU need to manually check FIFO has data and pop the FIFO to get each pixel out. The 20 ms is very close to what I get from my Basler (₁₅ ms) if I used the continuous acquisition mode mode.

If you want to the sum of accumulate pixels in a rectangular area, you should use the gateware accelerated ROI engines which are directly RTIO input FIFO.