Trouble installing second dds (urukul ad9910)

Hhoogeroo · 1 May

when running sinara tester I get:

****** Sinara system tester ******

*** Testing LEDs.
Check for blinking. Press ENTER when done.
Testing LED: led0

Testing LED: led1

*** Testing TTL outputs.
Outputs are tested in groups of 4. Touch each TTL connector
with the oscilloscope probe tip, and check that the number of
pulses corresponds to its number in the group.
Press ENTER when done.
Testing TTL outputs: ttl4, ttl5, ttl6, ttl7.

*** Testing TTL inputs.
TTL device to use as stimulus (default: ttl4): 
Connect ttl4 to ttl0. Press ENTER when done.

FAILED # **i just skipped past these, they do work however.**
Connect ttl4 to ttl1. Press ENTER when done.

FAILED 
Connect ttl4 to ttl2. Press ENTER when done.

FAILED
Connect ttl4 to ttl3. Press ENTER when done.

FAILED
*** Testing Urukul DDSes.
urukul0_cpld: initializing CPLD...
urukul0_cpld: testing attenuator digital control...
urukul0_cpld: done
urukul1_cpld: initializing CPLD...
Traceback (most recent call last):
  File "/nix/store/vxaim7i00a8h749i3x7gybw1nhxndpyl-python3.12-artiq-9.9140+4158954.beta/bin/.artiq_sinara_tester-wrapped", line 9, in <module>
    sys.exit(main())
             ^^^^^^
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/frontend/artiq_sinara_tester.py", line 986, in main
    experiment.run(tests)
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/frontend/artiq_sinara_tester.py", line 937, in run
    getattr(self, f"test_{name}")()
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/frontend/artiq_sinara_tester.py", line 304, in test_urukuls
    self.init_urukul(cpld)
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/language/core.py", line 54, in run_on_core
    return getattr(self, arg).run(run_on_core, ((self,) + k_args), k_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/coredevice/core.py", line 177, in run
    self._run_compiled(kernel_library, embedding_map, symbolizer, demangler)
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/coredevice/core.py", line 166, in _run_compiled
    self.comm.serve(embedding_map, symbolizer, demangler)
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/coredevice/comm_kernel.py", line 739, in serve
    self._serve_exception(embedding_map, symbolizer, demangler)
  File "/nix/store/jw63d7mkpfjiy8qasfbg1s92mfpfv4w7-python3-3.12.8-env/lib/python3.12/site-packages/artiq/coredevice/comm_kernel.py", line 721, in _serve_exception
    raise python_exn
ValueError: Urukul proto_rev mismatch

Ssrenblad · 2 May

Double check that you have connected your new Urukul to the correct ports. You can use the link below for reference:
https://git.m-labs.hk/sinara-hw/assembly/src/branch/master/src/hw/kasli_soc.md

Ssrenblad · 2 May

hoogeroo

When compiling the idle kernel, make sure that its compiling for the Kasli-SoC (ARM) and not the Kasli v2 (RISC-V). Perhaps the wrong (or an old) device_db.py was used to artiq_compile idle_kernel.py?

Hhoogeroo · 5 May

@srenblad Hey I just got back into the lab, thank you for your pointers, I'll give them a go and get back to you thank you

Hhoogeroo · 5 May

srenblad Hey Srenblad, I did attempt this, although now I am running into issues where I can no longer interact with my box over ethernet, and the error led is on, so I'll have to figure that out :")

Ssrenblad · 8 May

hoogeroo

Have you tried getting the UART log?

Additionally, check that the correct Urukul firmware version has been flashed. Binaries are available on Hydra.

Hhoogeroo · 21 May

srenblad

Hey there, I was eventually able to fix the problem.

My first problem was that I seemed to have some problem with my idle kernel when following the advice of , resulting in it thinking that I had multiple kernels running at once. I fixed this by using core-mgmt write to config the kernel again.

Once I was able to communicate with my device again, it was as you said, I had plugged my urukuls into the wrong eem slots, mostly due to the unexpected ordering, thank you for that diagram . I am now capable of using both urukul dds cards.

I however have ran into a new separate problem, one that I did occasionally run into when I was using the single urukul. That is that I get a set fraction of the set frequency, what seems to be 4/5 the set value, e.g if I set 125Mhz I get 100Mhz, 100 then 80, etc.

I see there are some others that have had similar problems, and its likely due to something regarding the clocking. I have been attempting to change values such as my reference clock, or flashing new rtio_clock values through the coremgmt config methods. However it seems I am still unware of the correct method to go about this.

I do have the following in my description.json file:


       {
            "type": "urukul",
            "dds": "ad9910",
            "hw_rev": "v1.3",
            "ports": [8,9],
            "clk_sel": 0,
}

As far as I'm aware "clk_sel":0 should use the internal clock of the urukul, although I'm not entirely sure if that means it uses its on separate value or that set by doing artiq_coremgmt -D IP config write -s rtio_clock int_100. I'm also not entirely sure I should be using hw_rev=1.3, but it doesn't seem to help if I change that, and from what I can tell in the documentation its the correct one.

I did originally have the MMCX cables connected between the urukuls and my core Kasli_Soc. If they are a better way of doing this then I'd be happy to do that.

Thanks for your previous help by the way @srenblad , It's much appreciated.

Thanks for any pointers

Ooccheung · 21 May

The refclk of AD9910 is 125 MHz by default. You are running it at 100 MHz.

I suggest you to explore the refclk and pll_n options in the JSON schema.
https://github.com/m-labs/artiq/blob/d7a380db01b05ef7b5d372584ab2f3568ea3b777/artiq/coredevice/ad9910.py#L116-L119
https://github.com/m-labs/artiq/blob/d7a380db01b05ef7b5d372584ab2f3568ea3b777/artiq/coredevice/urukul.py#L630-L631

Available JSON configs:
https://github.com/m-labs/artiq/blob/d7a380db01b05ef7b5d372584ab2f3568ea3b777/artiq/coredevice/coredevice_generic.schema.json#L281-L340

The AD9910 sysclk is generally run at 1 GHz, so I think pll_n should be 40.

After you update your JSON file, please update your device_db.py and do appropriate initialization after updating these values.

If any of these terminologies and number looks confusing to you, please checkout the AD9910 datasheet and Urukul schematics to understand the clock distribution.

Hhoogeroo · 22 May

occheung Thank you for this info, I'll have a look into it

Hhoogeroo · 22 May

occheung

Hey I've been looking into the documentation regarding things like refclk and pll_n. I've come to notice that I don't think my changes are being applied through after flashing. I've currently been booting with the ./local_run.sh script (which I think should work), but I see that it might not actually be applying my changes after writing my new gateware and make files.

I have attempted to use something like:
artiq_coremgmt -D IP config write -f boot /home/user/artiq-zynq/build/boot.bin

yet the same happens for both methods, despite the boot file indeed seemingly being updated when I create my new gateware. I assume I'm missing an obvious step here, but I'll continue looking just in case.

My current discription.json:


{
    "target": "kasli_soc",
    "min_artiq_version": "9.0",
    "variant": "variant",
    "hw_rev": "v1.1",
    "base": "standalone",
    "core_addr": "IP",
    "peripherals": [
        {
            "type": "grabber",
            "ports": [0, 1]
        },
        {
            "type": "dio",
            "board": "DIO_BNC",
            "ports": [2],
            "bank_direction_low": "input",
            "bank_direction_high": "output"
        },
        {
            "type": "urukul",
            "dds": "ad9910",
            "ports": [3],
            "clk_sel": 0,
            "pll_n": 40     <--- this is just the current value I thought I'd try change, however values like refclk, clk_div, etc, 
don't seem to change. I have tested this by simply running 
sudo nano gateware/kasli_soc.py -g ../build/gateware description.json

        },
        {
            "type": "sampler",
            "ports": [5, 6]
        },
        {
            "type": "fastino",
            "ports": [7],
            "hw_rev": "v1.1",
            "log2_width": 5
        },
        {
            "type": "urukul",
            "dds": "ad9910",
            "hw_rev": "v1.1",    <--- have tried changing around hw_rev, as well as removing it, 
but no changes there either it seems.
            "ports": [8],
            "clk_sel": 0
            	
        }
    ]
}

I am almost 100% certain this is the description that is being used.

Another side note is I see the red rf light leds are turned on apon power cycling. These can be cleared if I change the values in the new device db back, and then run artiq_sinara_tester -o urukuls (if I don't change the values I get the standard errors of not being able to create the called device e.g urukul0_ch0). This working with the changed values also likely indicates something wrong regarding the actual flashing process.

Hopefully that extra info adds more context, for now I'll keep at it. Thanks to you all for the help

Edit: there are also no errors in my jtag log, but I should probably provide it:


[     0.019990s]  INFO(szl): Simple Zynq Loader starting...
[     0.025205s] DEBUG(libboard_zynq::clocks::source): Set ARM_PLL to 2000000000 Hz
[     0.007041s] DEBUG(libboard_zynq::clocks::source): Set IO_PLL to 1000000000 Hz
[     0.016261s] DEBUG(libboard_zynq::clocks::source): Set DDR_PLL to 1066666666 Hz
[     0.023608s] DEBUG(libboard_zynq::ddr): DDR 3x/2x clocks: 533333328/355555552
[     0.030769s] DEBUG(libboard_zynq::ddr): DDR DCI clock: 10062892 Hz (divisors=2*53)
[     0.042000s] DEBUG(libboard_zynq::sdio): Reset SDIO!
[     0.046948s] DEBUG(libboard_zynq::sdio): Changing clock frequency to 400000
[     0.053895s]  INFO(szl): Card inserted. Mounting file system.
[     0.083156s] DEBUG(libboard_zynq::sdio): Changing clock frequency to 25000000
[     0.090209s] DEBUG(libboard_zynq::sdio::sd_card): Getting bus width
[     0.098363s] DEBUG(libboard_zynq::sdio::sd_card): 4 bit support
[     0.104261s] DEBUG(libboard_zynq::sdio::sd_card): Changing bus width
[     0.112000s] DEBUG(libboard_zynq::sdio): Set block size to 512
[     0.120152s] DEBUG(libconfig::sd_reader): Partition ID: C
[     0.126107s]  INFO(szl): Loading gateware
[     0.130392s] DEBUG(libconfig::bootgen): Partition header pointer = C80
[     0.137080s] DEBUG(libconfig::bootgen): Unencrypted length = C64DD
[     0.143243s] DEBUG(libconfig::bootgen): Partition start address: A900
[     0.494330s] DEBUG(libboard_zynq::devc): Invalidate DCache for bitstream buffer
[     0.506903s] DEBUG(libboard_zynq::devc): Init preload FPGA
[     0.512369s] DEBUG(libboard_zynq::devc): Toggling PROG_B
[     0.543135s] DEBUG(libboard_zynq::devc): Waiting for done
[     0.548514s] DEBUG(libboard_zynq::devc): Init postload FPGA
[     0.554068s]  INFO(szl): Loading runtime
[     0.559850s] DEBUG(libconfig::bootgen): Partition header pointer = C80
[     0.566540s] DEBUG(libconfig::bootgen): Unencrypted length = A33C
[     0.572618s] DEBUG(libconfig::bootgen): Unencrypted length = 2FB2E
[     0.578779s] DEBUG(libconfig::bootgen): Partition start address: D0DE0
[     0.670109s]  INFO(szl): Preparing for runtime execution
[     0.675748s]  INFO(szl): executing payload
[     0.000067s]  INFO(runtime): NAR3/Zynq7000 starting...
[     0.005236s]  INFO(runtime): gateware ident: 9+unknown;variant
[     0.016243s]  INFO(libboard_zynq::i2c): PCA9548 detected
[     0.175084s]  INFO(runtime::rtio_clocking): using internal 125MHz RTIO clock
[     0.565529s]  INFO(libboard_artiq::si5324): waiting for Si5324 lock...
[     6.188281s]  INFO(libboard_artiq::si5324):   ...locked
[     6.193490s]  INFO(runtime::rtio_clocking): Switching SYS clocks...
[     6.249988s]  INFO(runtime::rtio_clocking): SYS CLK switched successfully
[     6.256759s]  INFO(runtime::rtio_clocking): SYS PLL locked
[     6.267423s]  INFO(libboard_zynq::i2c): PCA9548 detected
[     6.303986s]  INFO(runtime::comms): network addresses: MAC=MAC_Add IPv4=IP IPv6-LL=xxxx IPv6: no configured address
[     6.332669s]  INFO(runtime::rtio_mgt): SED spreading disabled by default
[     6.343064s]  INFO(runtime::comms): loading idle kernel
[     6.348319s]  INFO(runtime::comms): running idle kernel
[     6.353525s]  INFO(ksupport::kernel::core1): kernel starting
[     9.841082s]  INFO(libboard_zynq::eth): eth: got Link { speed: S1000, duplex: Full }

Ooccheung · 22 May

These AD9910 configs are written to AD9910 by the kernel. Gateware/firmware isn't affected by this and flashing shouldn't help. Did you regenerate your device_db.py?

Also, given that you didn't change refclk, I would expect the AD9910 driver to complain during initialization.

Please keep the original hw_rev of urukul.

Hhoogeroo · 23 May

occheung

Howdy,

I messed around a bit more and potentially found a fix. I believe that despite me updating my device_db.py file, which was generated by using artiq_ddb_template -o device_db.py, I wasn't using that device_db during the flashing process. I was originally using the ./local_run.sh, but I have changed over to using artiq_coremgmt, and making sure I have the updated device_db.py in the directory of which I run the command in.

For future use:

What I changed (a lot at once, so the specific fix I'm not sure about):

I removed the hw_rev param (which should use the default value provided by the kernel) and also made sure both my urukuls had the same settings (I think in the device_db.py, urukul0 was the urukul1 in the json file, which I'm not sure why, there's a chance it wasn't but none the less). This is fine for me as I want both cards to be the same.

I set the rtio_clock on my Kasli_Soc the internal 100Mhz singal with, artiq_coremgmt -D 192.168.1.75 config write -s rtio_clock int_100

I set "refclk": 100e6, which is 100MHz,
I set the pll_en to 40 as recommended by @occheung ,

Current description.json file:

{
    "target": "kasli_soc",
    "min_artiq_version": "9.0",
    "variant": "<varient>",
    "hw_rev": "v1.1",
    "base": "standalone",
    "core_addr": "<IP>",
    "peripherals": [
        {
            "type": "grabber",
            "ports": [0, 1]
        },
        {
            "type": "dio",
            "board": "DIO_BNC",
            "ports": [2],
            "bank_direction_low": "input",
            "bank_direction_high": "output"
        },
        {
            "type": "urukul",
            "dds": "ad9910",
            "ports": [3],
            "refclk":100e6,
            "clk_sel": 0,
            "pll_n": 40
        },
        {
            "type": "sampler",
            "ports": [5, 6]
        },
        {
            "type": "fastino",
            "ports": [7],
            "hw_rev": "v1.1",
            "log2_width": 5
        },
        {
            "type": "urukul",
            "dds": "ad9910",
            "ports": [8],
            "refclk":100e6,
            "clk_sel": 0,
            "pll_n": 40
        }

    ]
}

After doing this I ran:

CD ~/artiq-zynq
nix develop --impure
CD src
artiq_ddb_template - o device_db.py   varient.json #generate device_db.py
gateware/kasli_soc.py -g ../build/gateware varient.json #generate new gateware
make TARGET=kasli_soc GWARGS="/home/user/artiq-zynq/src/varient.json" runtime #generate new make file
exit #out of nix dev
CD src #go back into src
cp device_db.py ../build #copy device_db.py into build
CD ../build #go into build
mkbootimage boot.bif boot.bin #make new boot.bin

splits into two methods from here.
For ease of access to any potential errors connect to uart log, find out which usb port your device is plugged into:

python -m serial.tools.list_ports -v 

/dev/ttyUSB2         #this was the one mine was under (I think its usually the case)
    desc: Quad RS232-HS - Quad RS232-HS


stty 115200 < /dev/ttyUSB2 
cat /dev/ttyUSB2

Method 1 (sd card boot):
Copy boot.bin into the SD card used for the artiq crate, then power on the crate (making sure to have switches to SD), I did this through the files system on my device, but you might need to mount the SD card then copy files onto it (ask google)

Method 2 (flash via jtag (I think its jtag)):

Inside /artiq-zynq/build:

artiq_coremgmt config write -f boot /home/user/artiq-zynq/build/boot.bin #can add -D IP if needed
artiq_coremgmt reboot

I then copied my device_db.py file into the directory in which I run my experiments, and voila, fixed.

Ssrenblad · 23 May

hoogeroo

For future reference:

Method 2 (flash via jtag (I think its jtag)):

It's not JTAG, the boot.bin is copied over the Ethernet connection and written to the SD card directly (after passing CRC). The method you show should still work without a USB cable connected. To enable JTAG on the Kasli-SoC, you also need to flip the DIP switch on the board from SD->JTAG.