Run loop on core device in the background

bodokaiser

Hello everyone,

We would like to change the Urukuls output frequency with as few latency as possible by combining interactive arguments with a background loop.

The idea is to run a background loop on the core device, which resolves an interactive argument from the host computer (or maybe at some point from the core device's memory), compares it with the previous value and if they differ performs an update of the Urukul's DDS chip.

The code would look somewhat like:

from artiq.experiment import EnvExperiment, NumberValue
from artiq.experiment import kernel, delay


class ScanExperiment(EnvExperiment):

    def build(self):
        self.setattr_device("core")
        self.setattr_device("urukul0_cpld")
        self.setattr_device("urukul0_ch0")

        self.setattr_argument("background_sleep", NumberValue(100, unit="ms"))
        self.setattr_argument("attenuation", NumberValue(10, unit="dB"))
        self.setattr_argument("frequency", NumberValue(100, unit="MHz", precision=0))

    def run(self):
        self.init()
        self.background()

        while True:
            with self.frequency(title="Frequency", unit="MHz") as interactive:
                interactive.setattr_argument(
                    "frequency",
                    NumberValue(interactive.frequency, unit="MHz", precision=0),
                )

    @kernel
    def init(self):
        self.core.reset()
        self.core.break_realtime()

        self.urukul0_cpld.init()
        self.urukul0_ch0.init()
        self.urukul0_ch0.cpld.init()
        self.urukul0_ch0.sw.on()

    @kernel
    def background(self):
        current_frequency = self.frequency
        current_attenuation = self.attenuation

        self.core.break_realtime()
        self.urukul0_ch0.set_att(self.attenuation)
        self.urukul0_ch0.set(self.frequency)

        while True:
            target_frequency = self.frequency
            target_attenuation = self.attenuation

            print("frequency: ", current_frequency, " -> ", target_frequency)

            if target_frequency != current_frequency:
                current_frequency = target_frequency
                self.urukul0_ch0.set(target_frequency)
            if target_attenuation != current_attenuation:
                current_attenuation = target_attenuation
                self.urukul0_ch0.set_att(target_attenuation)

            delay(self.background_sleep)

As you might expect, the code won't work as self.background blocks the main thread as it contains another while loop.

Is there some ARTIQ feature, which could run self.background on a core device in the background without blocking the main thread on the host?

If not, what would be alternative approaches here? Precompile background and run it every t seconds?

architeuthis

It's easy enough to run something on the host without blocking on the core device -- from the core device it's an asynchronous RPC, from the host it's just running a separate piece of Python. I think the issue rather is that you're trying to pass information back and forth between host and core device while blocking neither of them, which is not trivial. Two thoughts:

(1) From a quick look at your example here, why not ask for interactive arguments directly from the kernel or in a synchronous RPC? The kernel will block until the information is submitted, but that's the case for interactive arguments regardless, and your background loop doesn't seem to be doing anything besides waiting for input. As far as 'as little latency as possible' goes, interactive arguments have to be entered and submitted by the user anyway, so they're inherently incredibly slow opposed to the timescales ARTIQ normally operates on.

(2) Rather than run an experiment it sounds like what you want might be best served by MonInj, which is the feature that allows setting ('injecting') the state of (some) peripherals directly without needing to wrap it in an experiment. It's currently undocumented (which should change soon) but you can investigate it yourself in the MonInj tab of the dashboard.

bodokaiser

Hi @architeuthis, good idea to have the kernel run the interactive code on the host via synchronous RPC. I implemented your proposal in the following experiment:

from artiq.experiment import EnvExperiment, NumberValue
from artiq.experiment import kernel


class ScanExperiment(EnvExperiment):

    def build(self):
        self.setattr_device("core")
        self.setattr_device("urukul0_cpld")
        self.setattr_device("urukul0_ch0")

        self.setattr_argument("attenuation", NumberValue(10, unit="dB"))
        self.setattr_argument("frequency", NumberValue(10e6, unit="MHz", precision=0))

    def run(self):
        self.init()
        self.background()

        print("run finished")

    @kernel
    def init(self):
        self.core.reset()
        self.core.break_realtime()

        self.urukul0_cpld.init()
        self.urukul0_ch0.init()
        self.urukul0_ch0.cpld.init()
        self.urukul0_ch0.sw.on()

    def update_frequency(self) -> float:
        with self.interactive(title="Arguments") as interactive:
            interactive.setattr_argument(
                "frequency",
                NumberValue(self.frequency, unit="MHz", precision=0),
            )
        self.frequency = interactive.frequency

        return self.frequency

    @kernel
    def background(self):
        current_frequency = self.frequency
        current_attenuation = self.attenuation

        self.core.break_realtime()
        self.urukul0_ch0.set_att(current_attenuation)
        self.urukul0_ch0.set(current_frequency)

        while True:
            target_frequency = self.update_frequency()

            if target_frequency != current_frequency:
                self.core.break_realtime()
                self.urukul0_ch0.set(target_frequency)
                current_frequency = target_frequency

Can we scale this approach to multiple background loops?

For instance, can we have one loop on the host handling self.interactive in another thread and then have self.background1, self.background2, ..., getting the current value for self.frequency from the host via syncronous RPC?

Regarding Moninj, my understanding is that it involves constructing an EnvExperiment class that can be submitted by the scheduler. Would you agree? If so, I believe you're indirectly highlighting a crucial conceptual issue: by using interactive arguments, we risk losing track of the parameters. This likely contradicts the original intent of submitting one experiment per parameter set.

architeuthis

bodokaiser

Can we scale this approach to multiple background loops?

Meaning what exactly? You can only have one synchronous RPC at once, if that's what you mean; but why would you want to have multiple requests for the same arguments out at the same time? A synchronous RPC is not really 'background' in a meaningful sense; the kernel waits for it to return.

Regarding Moninj, my understanding is that it involves constructing an EnvExperiment class that can be submitted by the scheduler. Would you agree?

No. The purpose of MonInj is to 'override' the values of channels without having to run any sort of experiment, or even during an experiment. You can use it at any time.

bodokaiser

architeuthis

Meaning what exactly? You can only have one synchronous RPC at once, if that's what you mean; but why would you want to have multiple requests for the same arguments out at the same time? A synchronous RPC is not really 'background' in a meaningful sense; the kernel waits for it to return.

I would like to find a "coding style" for writing maintainable, e.g., readable, modular, and semantically logic code for ARTIQ that goes beyond the usual lining up of instructions and delays for our research group.

One idea I have is inspired by coroutines, which I envision to read like

from artiq.experiment import EnvExperiment, NumberValue, BooleanValue
from artiq.experiment import subkernel, kernel, rpc


class CoroutineExperiment(EnvExperiment):

    def build(self):
        self.setattr_device("core")
        self.setattr_device("ttl0")
        self.setattr_device("urukul0_cpld")
        self.setattr_device("urukul0_ch0")

        self.setattr_argument("logic_level", BooleanValue(False))
        self.setattr_argument("frequency", NumberValue(10e6, unit="MHz", precision=0))

    def run(self):
        self.init()

        # start background routines
        self.poll_logic_level()
        self.poll_frequency()

        while True:
            # constantly allow the user to update the frequency
            self.user_input()

    def user_input(self):
        with self.interactive(title="Arguments") as interactive:
            interactive.setattr_argument(
                "frequency",
                NumberValue(self.frequency, unit="MHz", precision=0),
            )
        self.frequency = interactive.frequency

    @kernel
    def init(self):
        self.core.reset()
        self.core.break_realtime()

        self.urukul0_cpld.init()
        self.urukul0_ch0.init()
        self.urukul0_ch0.cpld.init()
        self.urukul0_ch0.sw.on()

    @rpc
    def get_frequency(self) -> float:
        return self.frequency

    @rpc
    def get_logic_level(self) -> bool:
        return self.logic_level

    @subkernel(destination=1)
    def poll_frequency(self):
        # store the initial frequency setting
        current_frequency = self.frequency

        # break out of realtime and set the frequency
        self.core.break_realtime()
        self.urukul0_ch0.set(current_frequency)

        while True:
            # get the latest frequency from the host
            target_frequency = self.get_frequency()

            # update the frequency if it has changed
            if target_frequency != current_frequency:
                self.core.break_realtime()
                self.urukul0_ch0.set(target_frequency)
                current_frequency = target_frequency

    @subkernel(destination=2)
    def poll_logic_level(self):
        # store the initial logic level setting
        current_logic_level = self.logic_level

        # break out of realtime and set the logic level
        self.core.break_realtime()
        self.ttl0.output(current_logic_level)

        while True:
            # get the latest logic level from the host
            target_logic_level = self.get_logic_level()

            # update the logic level if it has changed
            if target_logic_level != current_logic_level:
                self.core.break_realtime()
                self.ttl0.output(target_logic_level)
                current_logic_level = target_logic_level

The advantage of such an approach (if it scales) would be that it allows fast parameter changes, runs functions on the core device closest to the respective EEM module (by using the @subkernel decorator), and allows code to be organized independently from another.

Of course, if you have other suggestions on organizing a large ARTIQ codebase, I am happy to hear them. (So far I am only aware of the DAX ARTIQ codebase from Duke university).

No. The purpose of MonInj is to 'override' the values of channels without having to run any sort of experiment, or even during an experiment. You can use it at any time.

Well, for the Urukul it works that way, see set frequency in moninj.py ?

For the TTL devices, it appears to me from comm_moninj.py that we are directly sending DRTIO packages to the core device?

architeuthis

bodokaiser

Well, for the Urukul it works that way, see set frequency in moninj.py ?

For the TTL devices, it appears to me from comm_moninj.py that we are directly sending DRTIO packages to the core device?

Ah, admittedly this outstrips my knowledge of how to use specific peripherals -- one thing I can contribute is that Urukul MonInj was recently a topic of discussion, if you haven't seen these discussions yet.

The advantage of such an approach (if it scales) would be that it allows fast parameter changes, runs functions on the core device closest to the respective EEM module (by using the @subkernel decorator), and allows code to be organized independently from another.

The specific problem here will be that subkernels don't support RPCs. Only the master is in direct communication with the host machine. Additionally, the 'self' that subkernels work on is a copy made at compile time, and later changes will not be reflected.

bodokaiser

architeuthis I see, so that is still an active area of research.

Regarding the question on how to organize large ARTIQ code bases, do you have any recommendations or resources we can study (beside the mentioned DAX repos)?