Access Directories

Guest

I'm writing code for dashboard scripts. I'd like some of them to be able to access the results of other experiments. To achieve this, the code I write needs access to the location of the results folder. How can I access this? The code will be run by artiq_master after submission, at which point calls to os.getcwd() should return the location where the hdf5 file after completion will ultimately be saved, so I could just go a few directories up from there, one level to get to the hour of the day, two levels to get to the day of the year, etc. Seems silly to hard code this. Are there any environment variables I could access at runtime or class variables of the scheduler, etc that would give me the absolute path to the results directory?

Same question for the directory in which the master is running.

Thanks!
Dave

bukehu

Hi!
There are currently no "environment variables" that you may use directly, but the results folder is always located under the artiq-master home folder (where the repository folder is located). We do not have a feature to change the results folder location, see this discussion.

I personally think your current approach is reasonable. You may also simply hardcode your home directory location and go down to the results folder.

Buke

dly

Hi Dave,
As @bukehu replied, the results directory is not exposed, because in the experiment's prepare(), each time we make a new formatted directory relative to the master's running directory and move into it. You can check this implementation in the source code (https://git.m-labs.hk/M-Labs/artiq) artiq/artiq/master/worker_impy.py:

if action == "build":
                start_time = time.time()
                rid = obj["rid"]
                expid = obj["expid"]
                if "devarg_override" in expid:
                    device_mgr.devarg_override = expid["devarg_override"]
                if "file" in expid:
                    if obj["wd"] is not None:
                        # Using repository
                        experiment_file = os.path.join(obj["wd"], expid["file"])
                        repository_path = obj["wd"]
                    else:
                        experiment_file = expid["file"]
                        repository_path = None
                    setup_diagnostics(experiment_file, repository_path)
                    exp = get_experiment_from_file(experiment_file, expid["class_name"])
                else:
                    setup_diagnostics("<none>", None)
                    exp = get_experiment_from_content(expid["content"], expid["class_name"])
                device_mgr.virtual_devices["scheduler"].set_run_info(
                    rid, obj["pipeline_name"], expid, obj["priority"])
                start_local_time = time.localtime(start_time)
                dirname = os.path.join("results",
                                       time.strftime("%Y-%m-%d", start_local_time),
                                       time.strftime("%H", start_local_time))
                os.makedirs(dirname, exist_ok=True)
                # os.environ['ARTIQ_RESULTS_HOME'] = os.path.abspath(".")
                os.chdir(dirname)
                argument_mgr = ArgumentManager(expid["arguments"])
                exp_inst = exp((device_mgr, dataset_mgr, argument_mgr, {}))
                argument_mgr.check_unprocessed_arguments()
                put_completed()

If you are using cloned repository and full flake, you could also add a environment variable by yourself, like the above commented os.environ['ARTIQ_RESULTS_HOME'] = os.path.abspath("."), before os.chdir(), then you can access it in your experiment.