Urukul DDS synchronisation/PLL error lights on

lmcc-lab

Hi all,

I should be considered a newby with ARTIQ based hardware, so please forgive me if this is a simple question.
We recently got a second Urukul card, but after opening our chassis I realised that we didn't have enough EEM slots because we have the Kasli V1.5 controller, which only has 8 EEM slots. No big deal, I ordered the backplane adaptor, and while I waited for that to arrive I turned everything back on (without any changes, so no extra Urukul card) and I noticed that the RED indicator light was on. In the Urukul wiki it says;

Red LED indicating DDS sychronization/PLL error or software-controlled function

When I run the following minimal example code, I get no significant output.

from artiq.experiment import *

class RFOnOffTest(EnvExperiment):
    """RF on off test"""
    def build(self):
        self.setattr_device("core")
        self.setattr_device("urukul0_cpld")
        self.setattr_device("suservo0")
        self.setattr_device("suservo0_ch0")

    def prepare(self):
        self.rf_on = 1
        self.frequency_MHz = 180
        self.amplitude = 1.0

    @kernel
    def run(self):
        self.core.reset()

        self.suservo0_ch0.set_dds(profile=0, frequency=self.frequency_MHz * 1e6, offset=0.0)
        self.suservo0_ch0.set_y(0, self.amplitude_norm)  # DDS amplitude to max manually
        self.suservo0_ch0.set(en_out=self.rf_on, en_iir=0, profile=0)  # turn the RF switch on, but disable the integrator
        self.suservo0_ch0.get_y(0)

This should turn on Urukul channel 0 with full amplitude at 180 MHz. We do have SUServo enabled in our system.
This used to work before I turned it off and tried installing the second Urukul, but now it doesn't. When I run this I do see a slight increase in 'amplitude' when watching the oscilliscope, but it's just noise, and this increase is very small. I do not get any error message from ARTIQ and the green indicator light on Urukul comes on when the RF is supposed to be on.

I have confirmed;

the EEM cables are connected in the correct slots,
the MMCX clock cable is connected to the clocker module,
the clocker front SMA outputs work (125 MHz +-1 V square wave),
the issue persists even with all other cards (except Kasli, clocker and Urukul) disconnected,
the issue persists even if I swap the Urukul card with the new Urukul card.

I am unsure what else I can do. Is there a possibility that there is a software issue? Can I reset the RAM or should I re-flash Kasli board? Any help will be greatly appreciated.

Regards,

Liam

sb10q

Are you using correct firmware and device database?

lmcc-lab

sb10q I believe so. Nothing has changed hardware wise, only that I was going to install a new Urukul card but didn't end up doing anything because I didn't have enough EEM slots.

Nevertheless, what would be the best way to check that I'm using the correct firmware and device database?

esavkin

lmcc-lab Nevertheless, what would be the best way to check that I'm using the correct firmware and device database?

I would try running artiq_sinara_tester -o urukuls to see if it works with code we use for testing every crate

lmcc-lab

esavkin Thank you for your recommendation, I had no idea this test module existed. Once it got to the Urukul testing I got this error

Traceback (most recent call last):
  File "C:\Users\Ion\anaconda3\envs\artiqsu_old\Scripts\artiq_sinara_tester-script.py", line 9, in <module>
    sys.exit(main())
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\frontend\artiq_sinara_tester.py", line 591, in main
    experiment.run()
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\frontend\artiq_sinara_tester.py", line 571, in run
    self.test_urukuls()
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\frontend\artiq_sinara_tester.py", line 256, in test_urukuls
    self.init_urukul(cpld)
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\language\core.py", line 54, in run_on_core
    return getattr(self, arg).run(run_on_core, ((self,) + k_args), k_kwargs)
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\coredevice\core.py", line 137, in run
    self.comm.serve(embedding_map, symbolizer, demangler)
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\coredevice\comm_kernel.py", line 652, in serve
    self._serve_exception(embedding_map, symbolizer, demangler)
  File "C:\Users\Ion\AppData\Roaming\Python\Python38\site-packages\artiq\coredevice\comm_kernel.py", line 644, in _serve_exception
    raise python_exn
ValueError: Urukul proto_rev mismatch

I believe it may have to do with the fact that we have SUservo enabled. Does this mean anything to you? To add extra details, I have also included my device_db.py contents:


# Autogenerated for the griffith variant
core_addr = "192.168.1.75"

device_db = {
    "core": {
        "type": "local",
        "module": "artiq.coredevice.core",
        "class": "Core",
        "arguments": {"host": core_addr, "ref_period": 1e-09,  "target": "or1k"},
    },
    "core_log": {
        "type": "controller",
        "host": "::1",
        "port": 1068,
        "command": "aqctl_corelog -p {port} --bind {bind} " + core_addr
    },
    "core_cache": {
        "type": "local",
        "module": "artiq.coredevice.cache",
        "class": "CoreCache"
    },
    "core_dma": {
        "type": "local",
        "module": "artiq.coredevice.dma",
        "class": "CoreDMA"
    },

    "i2c_switch0": {
        "type": "local",
        "module": "artiq.coredevice.i2c",
        "class": "PCA9548",
        "arguments": {"address": 0xe0}
    },
    "i2c_switch1": {
        "type": "local",
        "module": "artiq.coredevice.i2c",
        "class": "PCA9548",
        "arguments": {"address": 0xe2}
    },
}

# standalone peripherals

device_db["ttl0"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLInOut",
    "arguments": {"channel": 0x000000},
}


device_db["ttl1"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLInOut",
    "arguments": {"channel": 0x000001},
}


device_db["ttl2"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLInOut",
    "arguments": {"channel": 0x000002},
}


device_db["ttl3"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLInOut",
    "arguments": {"channel": 0x000003},
}


device_db["ttl4"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000004},
}


device_db["ttl5"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000005},
}


device_db["ttl6"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000006},
}


device_db["ttl7"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000007},
}


device_db["ttl8"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000008},
}


device_db["ttl9"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000009},
}


device_db["ttl10"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00000a},
}


device_db["ttl11"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00000b},
}


device_db["ttl12"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00000c},
}


device_db["ttl13"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00000d},
}


device_db["ttl14"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00000e},
}


device_db["ttl15"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00000f},
}


device_db["suservo0_ch0"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000010, "servo_device": "suservo0"}
}

device_db["suservo0_ch1"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000011, "servo_device": "suservo0"}
}

device_db["suservo0_ch2"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000012, "servo_device": "suservo0"}
}

device_db["suservo0_ch3"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000013, "servo_device": "suservo0"}
}

device_db["suservo0_ch4"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000014, "servo_device": "suservo0"}
}

device_db["suservo0_ch5"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000015, "servo_device": "suservo0"}
}

device_db["suservo0_ch6"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000016, "servo_device": "suservo0"}
}

device_db["suservo0_ch7"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "Channel",
    "arguments": {"channel": 0x000017, "servo_device": "suservo0"}
}

device_db["suservo0"] = {
    "type": "local",
    "module": "artiq.coredevice.suservo",
    "class": "SUServo",
    "arguments": {
        "channel": 0x000018,
        "pgia_device": "spi_sampler0_pgia",
        "cpld0_device": "urukul0_cpld",
        "cpld1_device": "urukul1_cpld",
        "dds0_device": "urukul0_dds",
        "dds1_device": "urukul1_dds"
    }
}

device_db["spi_sampler0_pgia"] = {
    "type": "local",
    "module": "artiq.coredevice.spi2",
    "class": "SPIMaster",
    "arguments": {"channel": 0x000019}
}

device_db["spi_urukul0"] = {
    "type": "local",
    "module": "artiq.coredevice.spi2",
    "class": "SPIMaster",
    "arguments": {"channel": 0x00001a}
}
device_db["urukul0_cpld"] = {
    "type": "local",
    "module": "artiq.coredevice.urukul",
    "class": "CPLD",
    "arguments": {
        "spi_device": "spi_urukul0",
        "refclk": 125000000.0,
        "clk_sel": 2
    }
}
device_db["urukul0_dds"] = {
    "type": "local",
    "module": "artiq.coredevice.ad9910",
    "class": "AD9910",
    "arguments": {
        "pll_n": 32,
        "chip_select": 3,
        "cpld_device": "urukul0_cpld"
    }
}

device_db["spi_urukul1"] = {
    "type": "local",
    "module": "artiq.coredevice.spi2",
    "class": "SPIMaster",
    "arguments": {"channel": 0x00001b}
}
device_db["urukul1_cpld"] = {
    "type": "local",
    "module": "artiq.coredevice.urukul",
    "class": "CPLD",
    "arguments": {
        "spi_device": "spi_urukul1",
        "refclk": 125000000.0,
        "clk_sel": 2
    }
}
device_db["urukul1_dds"] = {
    "type": "local",
    "module": "artiq.coredevice.ad9910",
    "class": "AD9910",
    "arguments": {
        "pll_n": 32,
        "chip_select": 3,
        "cpld_device": "urukul1_cpld"
    }
}

device_db["spi_zotino0"] = {
    "type": "local",
    "module": "artiq.coredevice.spi2",
    "class": "SPIMaster",
    "arguments": {"channel": 0x00001c}
}
device_db["ttl_zotino0_ldac"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00001d}
}
device_db["ttl_zotino0_clr"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00001e}
}
device_db["zotino0"] = {
    "type": "local",
    "module": "artiq.coredevice.zotino",
    "class": "Zotino",
    "arguments": {
        "spi_device": "spi_zotino0",
        "ldac_device": "ttl_zotino0_ldac",
        "clr_device": "ttl_zotino0_clr"
    }
}

device_db["led0"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x00001f}
}

device_db["led1"] = {
    "type": "local",
    "module": "artiq.coredevice.ttl",
    "class": "TTLOut",
    "arguments": {"channel": 0x000020}
}

esavkin

lmcc-lab ValueError: Urukul proto_rev mismatch

This typically happens when there is a mismatch between actual EEM ports connected and the JSON/gateware description provided. Are they actually connected as you previously reported (be aware of reversed EEM ports on Urukul relative to Kasli, use the labels on the board to check it)? Also please check that the wires are actually connected (pushed until the end from both sides). If that didn't help, I would suggest trying another wires, we must have shipped them with the Urukul.

lmcc-lab

esavkin Thank you all for your responses. So I got it working, and this is what I did.

I first switched the EEM cables, replaced the MMCX cable, then replaced the EEM cables on Urukul. The issue persisted. Then, because I'm a noob, I decided to re-start artiq_master after power cycling the Sinara hardware, and, after initializing SU-servo which we do by running this experiment

from artiq.experiment import *


class InitSUServo(EnvExperiment):
    """Initialise the SU servo"""
    def build(self):
        self.setattr_device("core")
        self.setattr_device("urukul0_cpld")
        self.setattr_device("suservo0")
        self.setattr_device("suservo0_ch0")
        self.setattr_device("suservo0_ch1")
        self.setattr_device("suservo0_ch2")
        self.setattr_device("suservo0_ch3")

    @kernel
    def run(self):
        self.core.reset()
        # initialize the servo + hardware (normally only done in your startup script)
        self.suservo0.init()
        #
        # # the servo is currently inactive, so active it
        # # after this
        # #  - the servo will continuously (once per ~us) read ADC samples and store them in RAM.
        # #    These are the "x" values
        # #  - it will also continuously update the DDS frequency, phase and amplitude from the values stored in RAM
        # #    The amplitude is the servo "y" value
        # #  - if the IIRs are enabled for a DDS channel and the RF switch is open then the servo will update the
        # #    y value by integrating the x values. Otherwise, the y values are held constant (e.g. use this to manually
        # #   set them
        #
        self.suservo0.set_config(enable=1)

        g = 1
        self.suservo0.set_pgia_mu(0, g)  # set gain on Sampler channel 0 to 10^g
        self.suservo0.set_pgia_mu(3, g)  # set gain on Sampler channel 3 to 10^g
        self.suservo0.set_pgia_mu(2, g)  # set gain on Sampler channel 2 to 10^g

        # # suservo_channels = [self.suservo0_ch0, self.suservo0_ch1]
        # # selected_channel = suservo_channels[self.urukul_channel]
        #
        # # now, before playing with the feedback, let's check we can get some life out of the DDSs
        #
        # # first, set the Urukul attenuation to desired value
        self.suservo0.cpld0.set_att(0, 13.0)  # changed from 17.0 to 13.0 27/06/2022 accounting for 2.5 dB transmission loss in BNC cable
                                              # 13.0  15/11/2022
        self.suservo0.cpld0.set_att(1, 14.5)  # changed from 17.0 to 14.5 27/06/2022 accounting for 2.0 dB transmission loss in BNC cable
                                              # 14.5  15/11/2022
        self.suservo0.cpld0.set_att(2, 14.5)  # changed from 16.5 to 14.5 20/06/2022 accounting for 2 dB transmission loss
        self.suservo0.cpld0.set_att(3, 10.0)  # changed from 15 to 10 20/06/2022 accounting for 5 dB transmission loss

        self.suservo0_ch0.get_y(0)
        delay(5 * us)
        self.suservo0_ch1.get_y(1)
        delay(5 * us)
        self.suservo0_ch2.get_y(2)
        delay(5 * us)
        self.suservo0_ch3.get_y(2)

        # This will also perform a test of the ports that are plugged in.

Then the red LED's turned off. I continued having some issues getting an output from Urukul but after another power cycle and ARTIQ reset it was working, including SUServo.

I do want to also point out that even with this now working, Urukul was still failing when running artiq_sinara_tester -o urukuls, with the same error message. It would be good to have a resolution to this too, as this would allow us to hardware test our crate in the future for debugging. I might open a seperate thread for this isssue.

Thanks again for both of your advice 🙂

esavkin

lmcc-lab I may had mistake on this, I think it should have been artiq_sinara_tester -o suservos, I'm sorry for misleading information.

lmcc-lab

esavkin Oh okay cool. Thank you so much for that though. I will try using that test-suite another day and close this thread with an update of how that went.

esavkin

lmcc-lab Any updates?

lmcc-lab

esavkin Hi, sorry for the super late reply. I have tested the suservos using artiq_sinara_tester -o suservos and that worked well. I have tested it recently as I have finally gotten around to upgrading the gateware to enable the second Urukul. All working well.