Artiq 7: Python packages installed through flake.nix are not imported

willow_01

We have an Artiq 7 system installed via Nix on an Ubuntu Linux machine. I was trying to install some Python packages through the flake.nix which is originally from the Installing via Nix page of Artiq 7 online manual. I literally copied the flake.nix file contents from the webpage and pasted in a file with same name in an empty directory on my Linux computer. I uncommented a few Python packages in the file (ps.pandas, ps.matplotlib ... ) and added ps.influxdb. I then opened a terminal and navigated to the directory and ran $ nix shell. The flake.lock file was created and no error was reported. However, when I started the python in that directory and import packages installed in this way (for example, pandas), I got ModuleNotFoundEorror: No module named 'pandas'.

I searched in the forum and didn't found any closely related posts. This may not be an issue for most of the Artiq users but just curious that if anyone can help on this...
Many thanks in advance...

fsagbuya

Seems to be fine when I tested it.

Python 3.9.16 (main, Dec  6 2022, 18:36:13) 
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import artiq
>>> import pandas

You can check if you are already in the nix shell by verifying that the Python version is 3.9.16. Make sure that the ps packages are inside the pkgs.python3.withPackages.

willow_01

fsagbuya Thanks a lot for your reply. I checked that the Python version is 3.9.16 when l launched the Python after doing $ nix shell. Could you please elaborate a bit more on how to check if the ps packages are inside the pkgs.python3.withPackages or how to find the pkgs.python3.withPackages?
(I am first-time user of Nix therefore more detailed instructions are appreciated.)

Python 3.9.16 (main, Dec 6 2022, 18:36:13)
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import artiq
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'

fsagbuya

From the artiq flake:

    (pkgs.python3.withPackages(ps: [
            # List desired Python packages here.
            aqmain.artiq
            #ps.paramiko  # needed if and only if flashing boards remotely (artiq_flash -H)
            #aqextra.flake8-artiq

            # The NixOS package collection contains many other packages that you may find
            # interesting. Here are some examples:
+           ps.pandas
            #ps.numpy
            #ps.scipy
            #ps.numba
+           ps.matplotlib
+           ps.influxdb
            # or if you need Qt (will recompile):
            #(ps.matplotlib.override { enableQt = true; })
            #ps.bokeh
            #ps.cirq
            #ps.qiskit
          ]))

willow_01

fsagbuya This portion of the flake.nix is exactly the same as in my flake.nix. I am confused...
Are there any other things needed to be checked??

fsagbuya

Not sure what the issue is, since there are no additional error logs. Could you try using devShells instead? See: https://forum.m-labs.hk/d/951-help-on-nix-manager-package-installs/2

willow_01

fsagbuya I tried the nix develop with devShells and now it seems everything works as expected. I can call python 3.9.16 there and import my local installed python packages i.e. pandas, matplotlib, influxdb etc.

In the flake.nix file, I changed the line defaultPackage.x86_64-linux = pkgs.buildEnv { to devShells.x86_64-linux.default = pkgs.mkShell { and then paths = [ to packages = [. This seems working fine.

Here is the modified flake.nix file in case anyone else needs it for reference:

{
  inputs.artiq.url = "git+https://github.com/m-labs/artiq.git?ref=release-7";
  inputs.extrapkg.url = "git+https://git.m-labs.hk/M-Labs/artiq-extrapkg.git?ref=release-7";
  inputs.extrapkg.inputs.artiq.follows = "artiq";
  outputs = { self, artiq, extrapkg }:
    let
      pkgs = artiq.inputs.nixpkgs.legacyPackages.x86_64-linux;
      aqmain = artiq.packages.x86_64-linux;
      aqextra = extrapkg.packages.x86_64-linux;
    in {
      #defaultPackage.x86_64-linux = pkgs.buildEnv {
      devShells.x86_64-linux.default = pkgs.mkShell {
        name = "artiq-env";
        #paths = [
        packages = [
          # ========================================
          # EDIT BELOW
          # ========================================
          (pkgs.python3.withPackages(ps: [
            # List desired Python packages here.
            aqmain.artiq
            #ps.paramiko  # needed if and only if flashing boards remotely (artiq_flash -H)
            #aqextra.flake8-artiq

            # The NixOS package collection contains many other packages that you may find
            # interesting. Here are some examples:
            ps.pandas
            #ps.numpy
            #ps.scipy
            # or if you need Qt (will recompile):
            #(ps.matplotlib.override { enableQt = true; })
            #ps.bokeh
            #ps.cirq
            #ps.qiskit
          ]))
          #aqextra.korad_ka3005p
          #aqextra.novatech409b
          # List desired non-Python packages here
          #aqmain.openocd-bscanspi  # needed if and only if flashing boards
          # Other potentially interesting packages from the NixOS package collection:
          #pkgs.gtkwave
          #pkgs.spyder
          #pkgs.R
          #pkgs.julia
          # ========================================
          # EDIT ABOVE
          # ========================================
        ];
      };
    };
  nixConfig = {  # work around https://github.com/NixOS/nix/issues/6771
    extra-trusted-public-keys = "nixbld.m-labs.hk-1:5aSRVA5b320xbNvu30tqxVPXpld73bhtOeH6uAjRyHc=";
    extra-substituters = "https://nixbld.m-labs.hk";
  };
}

Thanks a lot for your help!!