I ran into a nasty problem with numpy array: The values of a returned numpy array changed depending if I had an additional print(arr) line or not. While I have solved the issue for me, I am still curious what is actually happening.

Here are a few similar examples. In all cases the output is not what I'd expect. To me the problem looks a bit like a use after free issue, but I might be wrong. In all cases the output was reproducible for me, I didn't get random output.

Example 1

from artiq.experiment import *
import numpy as np

class NumpyTypes1(EnvExperiment):
    def build(self):
        self.setattr_device("core")

    @kernel
    def func(self):
        if self.core.get_rtio_counter_mu() % 2:
            return np.array([30,20,10], dtype=np.int32)
        else:
            return np.array([10,20,30], dtype=np.int32)

    def run(self):
        print(self.func())

The (reproducible) output I observe: [ 1 20 1342176984]
Output I'd expect: [30,20,10] or [10,20,30]

The if-branch in the method func is probably required that the compiler does not inline the array. Without the if, the output becomes [10 20 30] (as expected).

Example 2

from artiq.experiment import *
import numpy as np

class NumpyTypes2(EnvExperiment):
    def build(self):
        self.setattr_device("core")

    @kernel
    def func(self):
        if self.core.get_rtio_counter_mu() % 2:
            return np.array([30,20,10], dtype=np.int32)
        else:
            return np.array([10,20,30], dtype=np.int32)

    @kernel
    def run(self):
        print(self.func())

The (reproducible) output I observe: [1342176948 20 30]
Output I'd expect: [30,20,10] or [10,20,30]

The only difference between NumpyTypes1 and NumpyTypes2 is that run is a kernel method in in NumpyTypes2 but not in NumpyTypes1. It is also curious, that the numbers 1342176984 and 1342176948 are so similar.

Example 3

from artiq.experiment import *
import numpy as np

class NumpyTypes3(EnvExperiment):
    def build(self):
        self.setattr_device("core")

    @kernel
    def func(self):
        if self.core.get_rtio_counter_mu() % 2:
            a = 30
            b = 20
            c = 10
        else:
            a = 10
            b = 20
            c = 30
        
        return np.array([a,b,c], dtype=np.int32)

    @kernel
    def func2(self):
        return self.func()

    @kernel
    def run(self):
        print(self.func())
        print(self.func2())

Output:

[10 20 30]
[        10 1342176880         30]

Expected output:

[10 20 30]
[10 20 30]

func returns the expected array, while func2 does not, even though func2 is just a tiny wrapper. But apparently, it is sufficient that the compiler cannot infer the right types anymore.

You cannot return values that are allocated on the stack. This is equivalent to this well-known C code problem:

char *func() {
  char abc[3];
  abc[0] = 30;
  abc[1] = 20;
  abc[2] = 10;
  return abc;
}

Under NAC3 this will become a compilation error.

See https://github.com/m-labs/artiq/pull/2239 for a fix that catches all your original examples. The full issue is still thorny and will require further-reaching modifications of the lifetime tracking code, but it turns out that basic use of numpy.array and friends can be handled in quite a straightforward fashion (even if the values can still be escaped by passing them to another function first).