MTDDS Phaser DRTIO/DDMA limitations

debry

I'm interested in some more information about what the phaser is capable of as a DRTIO node with the MTDDS gateware, which we have now installed (default 250 MHz/26 tones) and functioning.

My application is that we do pulse shaping of a single frequency tone by calling set_amplitude() every 100 ns, and this works with both the old and new gateware when using standard DMA. However, I would like to free our Kasli-SOC to do other real-time operations (feeding back on inputs in a way that cannot be pre-recorded as DMA), and so I was hoping to use the distributed DMA feature to record the sequence in the phaser's memory instead of the Kasli-SOC's. We have the following code that works for short (100 updates) sequences:

    @kernel
    def record_sequence_ddma(self,
                             dma_name,
                             asf_values,
                             dt_mu):
        self.core.break_realtime()
        n = len(asf_values)

        with self.core_dma.record(dma_name, enable_ddma=True):
            for i in range(n):
                self.phaser_tone.set_amplitude_mu(asf_values[i])
                delay_mu(dt_mu)
        self.core.break_realtime()

    @kernel
    def playback_ddma(self, dma_name, repeats=1):
        self.core.break_realtime()
        handle = self.core_dma.get_handle(dma_name)
        self.core.break_realtime()
        for _ in range(repeats):
            self.core_dma.playback_handle(handle)
            delay(1*us)

However, there are two issues:

A very long delay (many ms) is required before DMA playback to avoid underflow errors
Anything more that 128 amplitude updates in the DDMA sequence causes the experiment to hang

Is this a fundamental limitation of the memory available on the phaser? We will need more like 2000 amplitude updates for our pulses to be smooth enough. Is there some other way to accomplish our goal of freeing up our Kasli to perform other operations?

I know the phaser was designed more for outputting many tones than for the sort of pulse shaping we are doing, but the pulse shapes we need cannot be faithfully represented by a sum of 26 oscillators. We can also consider moving the phaser to our satellite crate and doing DDMA on the satellite Kasli, which would hopefully free up the master Kasli for other operations.

claytonho

debry
my understanding with phaser (the non-mtdds version, though) is that the many-ms delay you’re experiencing is from saturation of the event FIFOs - we also cap out at a similar number of update events
does ddma fix FIFO saturation?

spaqin

DDMA does indeed take extra time to set up - whole DMA trace has to be sifted through, each event has to be separated for master and each satellite - that takes time; that has to be transmitted over DRTIO auxiliary protocol - yes that takes time; and start command also takes a bit longer than DMA itself. So it's not always the best choice, and recording should be done early in the experiment before anything else is run.

Why 128+ updates causes a hang, not sure - sounds like a bug, will try to reproduce and file a ticket if I find something. Thanks for the sample code, it will certainly make it easier.

Have you considered subkernels? That would free up the master to do basically anything and no recording would have to be done - the compiled subkernel would be already on the Phaser.

debry

Thanks both for the info! Good to know on the timing. I think we could work around that, and it’s the 128-event max that’s the real issue. I’ve attached a full experiment below to help with reproduction, where the first DMA sequence plays and the second hangs and does not output anything.

spaqin I had thought about sub kernels, but I’ve found that at least on the Kasli I really needed the DMA to push the events at the < 100 ns rate we need, and I didn’t think a sub kernel would fix that. I’ll give it a shot today though and see how quickly a sub kernel running on the phaser is able to update the amplitude.

from artiq.experiment import *
import numpy as np


class PhaserDDMADemo(EnvExperiment):
    """Phaser DDMA demo"""

    def build(self):
        self.setattr_device("core")
        self.setattr_device("core_dma")
        self.setattr_device("phaser_drtio_mtdds0_fpga")
        self.setattr_device("phaser_drtio_mtdds0_channel0")
        self.setattr_device("trigger")

    def prepare(self):
        self.phaser_fpga = self.phaser_drtio_mtdds0_fpga
        self.phaser_ch = self.phaser_drtio_mtdds0_channel0

        n = 128
        duration_mu = self.core.seconds_to_mu(10*ms)
        self.dt_mu = max(8, duration_mu // n)

        # Build a sin^2 envelope in amplitude.
        x = np.linspace(0.0, np.pi, n)
        amps = 0.9 * np.sin(x) ** 2
        self.asf_values_128 = np.asarray([self.phaser_ch.ddss[0].amplitude_to_asf(float(a)) for a in amps], dtype=np.int32)

        n = 129
        duration_mu = self.core.seconds_to_mu(1*ms)
        self.dt_mu = max(8, duration_mu // n)

        # Build a sin^2 envelope in linear amplitude.
        x = np.linspace(0.0, np.pi, n)
        amps = 0.9 * np.sin(x) ** 2
        self.asf_values_129 = np.asarray([self.phaser_ch.ddss[0].amplitude_to_asf(float(a)) for a in amps], dtype=np.int32)

    @kernel
    def run(self):
        self.core.reset()
        self.core.break_realtime()

        self.phaser_fpga.init()
        self.core.break_realtime()
        self.phaser_ch.init()
        self.core.break_realtime()

        self.phaser_ch.attenuator.set_att(0.0)
        delay(1*us)
        self.phaser_ch.servo.enable_iir(False)
        delay(1*us)
        self.phaser_ch.select_dac_source(1)
        delay(1*us)

        self.phaser_ch.ddss[0].enable_phase_accumulator(True)
        delay(1*us)
        self.phaser_ch.ddss[0].set_frequency(1*MHz)
        delay(1*us)
        self.phaser_ch.ddss[0].set_amplitude(0.0)
        delay(1*us)

        self.core.break_realtime()

        # Record 128 sample DDMA
        self.record_sequence_ddma("128", self.asf_values_128, self.dt_mu)
        # Record 129 sample DDMA
        self.record_sequence_ddma("129", self.asf_values_129, self.dt_mu)

        self.core.break_realtime()
        delay(50*ms)

        # Playback (runs OK)
        self.trigger.pulse(1*us)
        self.playback_ddma("128")

        
        self.core.break_realtime()

        # Playback (hangs)
        delay(10*ms)
        self.playback_ddma("129")


    @kernel
    def play_sequence_mu(self,
                         asf_values,
                         dt_mu):
        
        n = len(asf_values)

        for i in range(n):
            self.phaser_ch.ddss[0].set_amplitude_mu(asf_values[i])
            delay_mu(dt_mu)

    @kernel
    def record_sequence_ddma(self,
                             dma_name,
                             asf_values,
                             dt_mu):
        self.core.break_realtime()

        with self.core_dma.record(dma_name, enable_ddma=True):
            self.play_sequence_mu(asf_values, dt_mu)

        self.core.break_realtime()

    @kernel
    def playback_ddma(self, dma_name):
        self.core.break_realtime()
        handle = self.core_dma.get_handle(dma_name)
        self.core.break_realtime()
        self.core_dma.playback_handle(handle)
        delay(1*us)

debry

spaqin I got the subkernel working, but this seems to struggle with long sequences at fast rates. I can play back short (approx 100 update) sequences at 100 ns update rate, but for 300 us long sequences I have to slow the update rate down to 400 ns for it to work, otherwise the updates just stop at some point. Code is attached below, including a toggle to use local DMA from the Kasli.
Attempting to play back a 200 us pulse with 200 ns update rate with phaser subkernel:

The same pulse (200 us, 200 ns rate) using Kasli DMA

from artiq.experiment import *
import numpy as np


class PhaserSubkernelDemo(EnvExperiment):
    """Phaser Subkernel demo"""

    def build(self):
        self.setattr_device("core")
        self.setattr_device("core_dma")
        self.setattr_device("phaser_drtio_mtdds0_fpga")
        self.setattr_device("phaser_drtio_mtdds0_channel0")
        self.setattr_device("trigger")

        self.setattr_argument("duration", NumberValue(default=100*us, unit="us"))
        self.setattr_argument("update_period", NumberValue(default=100*ns, unit="ns"))
        self.setattr_argument("use_subkernel", BooleanValue(default=True))

    def prepare(self):
        self.phaser_fpga = self.phaser_drtio_mtdds0_fpga
        self.phaser_ch = self.phaser_drtio_mtdds0_channel0

        duration_mu = self.core.seconds_to_mu(self.duration)
        self.dt_mu = self.core.seconds_to_mu(self.update_period)
        x = np.linspace(0.0, np.pi, duration_mu // self.dt_mu)
        amps = 0.9 * np.sin(x) ** 2
        self.asf_values = np.asarray([self.phaser_ch.ddss[0].amplitude_to_asf(float(a)) for a in amps], dtype=np.int32)
        print(self.asf_values, self.dt_mu)

    @kernel
    def run(self):
        self.core.reset()
        self.core.break_realtime()

        self.phaser_fpga.init()
        self.core.break_realtime()
        self.phaser_ch.init()
        self.core.break_realtime()

        self.phaser_ch.attenuator.set_att(0.0)
        delay(1*us)
        self.phaser_ch.servo.enable_iir(False)
        delay(1*us)
        self.phaser_ch.select_dac_source(1)
        delay(1*us)

        self.phaser_ch.ddss[0].enable_phase_accumulator(True)
        delay(1*us)
        self.phaser_ch.ddss[0].set_frequency(10*MHz)
        delay(1*us)
        self.phaser_ch.ddss[0].set_amplitude(0.0)
        delay(1*us)

        self.core.break_realtime()
        with self.core_dma.record("phaser_subkernel_demo", enable_ddma=False):
            self.trigger.pulse(1*us)
            self.playback_local_phaser()
        self.core.break_realtime()
        handle = self.core_dma.get_handle("phaser_subkernel_demo")
        self.core.break_realtime()

        if self.use_subkernel: # Run subkernel on the phaser
            self.playback_phaser_subkernel()
            self.core.break_realtime()
            self.trigger.pulse(1*us)
        else: # Run playback using local core DMA
            self.core_dma.playback_handle(handle)

    @kernel
    def playback_local_phaser(self):
        self.phaser_ch.ddss[0].set_amplitude(0.0)
        delay(10*us)

        for i in range(len(self.asf_values)):
            self.phaser_ch.ddss[0].set_amplitude_mu(self.asf_values[i])
            delay_mu(self.dt_mu)
        
        self.phaser_ch.ddss[0].set_amplitude(0.0)
        self.core.break_realtime()

    # Subkernel on the phaser
    @subkernel(destination=5)
    def playback_phaser_subkernel(self):
        self.core.break_realtime()
        self.phaser_ch.ddss[0].set_amplitude(0.0)
        delay(1*us)

        for i in range(len(self.asf_values)):
            self.phaser_ch.ddss[0].set_amplitude_mu(self.asf_values[i])
            delay_mu(self.dt_mu)
        
        self.phaser_ch.ddss[0].set_amplitude(0.0)
        self.core.break_realtime()