Artiq Sinara Tester Errors

armo13579

I am setting up a new Artiq system and wanted to use the artiq_sinara_tester.py file provided here:
https://github.com/m-labs/artiq/blob/master/artiq/frontend/artiq_sinara_tester.py

The artiq is communicating with the computer and responds to a ping, but when I run the tester file the following error comes up:

Traceback (most recent call last):
File "/home/artiq/miniconda3/envs/artiq/bin/artiq_run", line 10, in <module>
sys.exit(main())
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/frontend/artiq_run.py", line 224, in main
return run(with_file=True)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/frontend/artiq_run.py", line 210, in run
raise exn
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/frontend/artiq_run.py", line 203, in run
exp_inst.run()
TypeError: run() missing 1 required positional argument: 'tests'

Any recommendations for potential fixes?
Thanks!

sb10q

Which system is it and where did you purchase it from?

sb10q

Anyway it looks like you are trying to run artiq_sinara_tester via artiq_run, which is not the correct way to use it. Just execute the artiq_sinara_tester command instead.

armo13579

When executing the artiq_sinara_tester command, the following error occurs:

Traceback (most recent call last):
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/master/worker_db.py", line 88, in get
dev = _create_device(desc, self)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/master/worker_db.py", line 27, in _create_device
return device_class(device_mgr, **desc.get("arguments", {}))
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/coredevice/core.py", line 82, in init
raise ValueError("Unsupported target")
ValueError: Unsupported target

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/artiq/miniconda3/envs/artiq/bin/artiq_sinara_tester", line 10, in <module>
sys.exit(main())
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/frontend/artiq_sinara_tester.py", line 589, in main
experiment = SinaraTester((device_mgr, None, None, None))
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/language/environment.py", line 244, in init
self.build(*args, **kwargs)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/frontend/artiq_sinara_tester.py", line 44, in build
self.setattr_device("core")
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/language/environment.py", line 328, in setattr_device
setattr(self, key, self.get_device(key))
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/language/environment.py", line 321, in get_device
return self.__device_mgr.get(key)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.8/site-packages/artiq/master/worker_db.py", line 90, in get
raise DeviceError("Failed to create device '{}'"
artiq.master.worker_db.DeviceError: Failed to create device 'core'

armo13579

This system is the waterloo2 purchased from M-labs.

armo13579

The above error occurred with the computer running 6.7665.83af3b75. After removing that environment and downloading Artiq 7.8047.16393ef.beta instead, the following error occurs when running artiq_sinara_tester:

****** Sinara system tester ******

Mismatch between gateware (7.0.69b6426.beta) and software (7.8044.c440f9f.beta) versions
Previous kernel did not cleanly finish
Traceback (most recent call last):
File "/home/artiq/miniconda3/envs/artiq/bin/artiq_sinara_tester", line 10, in <module>
sys.exit(main())
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/frontend/artiq_sinara_tester.py", line 788, in main
experiment.run(tests)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/frontend/artiq_sinara_tester.py", line 735, in run
self.core.reset()
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/language/core.py", line 54, in run_on_core
return getattr(self, arg).run(run_on_core, ((self,) + k_args), k_kwargs)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/coredevice/core.py", line 140, in run
self._run_compiled(kernel_library, embedding_map, symbolizer, demangler)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/coredevice/core.py", line 130, in run_compiled
self.comm.serve(embedding_map, symbolizer, demangler)
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/coredevice/comm_kernel.py", line 703, in serve
self.read_header()
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/coredevice/comm_kernel.py", line 249, in read_header
sync_byte = self.read(1)[0]
File "/home/artiq/miniconda3/envs/artiq/lib/python3.9/site-packages/artiq/coredevice/comm_kernel.py", line 237, in _read
raise ConnectionResetError("Core device connection closed unexpectedly")
ConnectionResetError: Core device connection closed unexpectedly

sb10q

The first error is expected, ARTIQ versions 6 and 7 use different processor architectures and are not compatible.

The second is more surprising. Are you running anything on the device in parallel with sinara_tester? Can you grab a log of the device e.g. with artiq_coremgmt log when the problem occurs?

armo13579

Running artiq_coremgmt log results in the following:

[ 0.000015s] INFO(runtime): ARTIQ runtime starting...
[ 0.003937s] INFO(runtime): software ident 7.0.69b6426.beta;waterloo2master
[ 0.010998s] INFO(runtime): gateware ident 7.0.69b6426.beta;waterloo2master
[ 0.018079s] INFO(runtime): log level set to INFO by default
[ 0.023797s] INFO(runtime): UART log level set to INFO by default
[ 0.154608s] WARN(runtime::rtio_clocking): rtio_clock setting not recognised. Falling back to default.
[ 0.162712s] INFO(runtime::rtio_clocking): using internal 125MHz RTIO clock
[ 0.439200s] INFO(board_artiq::si5324): waiting for Si5324 lock...
[ 2.832933s] INFO(board_artiq::si5324): ...locked
[ 2.838069s] INFO(runtime::rtio_clocking::crg): Using internal RTIO clock
[ 2.869192s] INFO(runtime): network addresses: MAC=e8-eb-1b-45-c1-fe IPv4=192.168.1.75 IPv6-LL=fe80::eaeb:1bff:fe45:c1fe IPv6=no configured address
[ 2.882898s] WARN(board_artiq::drtio_routing): length of the configured routing table is incorrect
[ 2.890655s] INFO(board_artiq::drtio_routing): could not read routing table from configuration, using default
[ 2.900665s] INFO(board_artiq::drtio_routing): routing table: RoutingTable { 0: 0; 1: 1 0; 2: 2 0; 3: 3 0; }
[ 2.928634s] INFO(runtime::mgmt): management interface active
[ 2.940665s] INFO(runtime::session): accepting network sessions
[ 2.953759s] INFO(runtime::session): running startup kernel
[ 2.958210s] INFO(runtime::session): no startup kernel found
[ 2.964027s] INFO(runtime::session): no connection, starting idle kernel
[ 2.970860s] INFO(runtime::session): no idle kernel found
[ 2.976263s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 13.034815s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 13.039718s] INFO(runtime::rtio_mgt::drtio): [DEST#0] destination is up
[ 13.246897s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 23.159964s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 23.364987s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 43.319604s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 43.525002s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 63.681094s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 63.886517s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 84.042662s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 84.248048s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 104.404122s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 104.609534s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 124.765617s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 124.971066s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 145.127184s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 145.332550s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 165.488704s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 165.694038s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 185.850167s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 186.055599s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 206.211631s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 206.417065s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 226.573238s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 226.778567s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 246.934671s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 247.140075s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 267.296244s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 267.501630s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 287.657687s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 287.863095s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 308.019235s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 308.224640s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 328.380776s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 328.586102s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 348.742221s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 348.947602s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 369.103734s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 369.309170s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 389.465281s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 389.670671s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 409.826802s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 410.032137s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 430.188266s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 430.393641s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 450.549783s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 450.755148s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 470.911334s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 471.116692s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 491.272825s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 491.478158s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 511.634315s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 511.839666s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 531.995808s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 532.201232s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 552.357300s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 552.562688s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 572.718796s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 572.924229s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 593.080383s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 593.285731s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 613.441886s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 613.647202s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 633.803300s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 634.008715s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 654.164844s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 654.370258s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 665.326137s] INFO(runtime::mgmt): new connection from 192.168.1.10:58083
[ 674.526368s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 674.731782s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 694.887850s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 695.093275s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 715.249350s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 715.454757s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 735.610877s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 735.816287s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 755.972356s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 756.177771s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 776.333917s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 776.539304s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 796.695409s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 796.900839s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 817.056904s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 817.262338s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 837.418463s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 837.623790s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 857.779982s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 857.985325s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 878.141482s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 878.346851s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 898.502977s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 898.708364s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 918.864431s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 919.069845s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 939.225988s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 939.431389s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 959.587433s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 959.792848s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 979.948945s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 980.154406s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1000.310480s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1000.515851s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1020.672024s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1020.877400s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1041.033470s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1041.238885s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1061.395020s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1061.600374s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1081.756561s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1081.961915s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1102.118049s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1102.323442s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1122.479503s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1122.684968s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1142.841052s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1143.046471s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1163.202549s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1163.407983s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1183.564022s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1183.769464s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1203.925522s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1204.130998s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1224.287042s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1224.492495s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1244.648568s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1244.854000s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1265.010121s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1265.215477s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1285.371604s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1285.576991s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1305.733153s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1305.938541s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1326.094597s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1326.300017s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1346.456132s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1346.661554s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1366.817682s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1367.023050s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1387.179119s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1387.384504s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1407.540698s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1407.746031s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1427.902173s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1428.107578s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1448.263613s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1448.469071s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1468.625147s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1468.830576s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1488.986688s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1489.192064s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1509.348224s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1509.553549s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1529.709681s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1529.915058s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1550.071257s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1550.276632s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1570.432690s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1570.638141s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1590.794186s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1590.999630s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1611.155745s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1611.361106s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1631.517291s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1631.722641s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1651.878777s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1652.084158s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1672.240228s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1672.445657s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1692.601786s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1692.807158s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1712.963291s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1713.168677s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1733.324752s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1733.530181s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1753.686275s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1753.891672s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1774.047819s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1774.253177s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1794.409331s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1794.614659s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1814.770790s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1814.976194s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1835.132271s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1835.337705s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1855.493846s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1855.699247s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1875.855314s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1876.060740s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1896.216865s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1896.422216s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1916.578297s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1916.783722s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1936.939877s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1937.145250s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1957.301360s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1957.506786s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1977.662878s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1977.868243s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 1998.024416s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 1998.229779s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2018.385931s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2018.591256s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2038.747400s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2038.952758s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2059.108850s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2059.314327s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2079.470459s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2079.675784s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2099.831935s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2100.037339s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2120.193413s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2120.398832s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2140.554985s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2140.760358s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2160.916433s] ERROR(runtime::rtio_mgt::drtio): [LINK#0] ping failed
[ 2161.121826s] INFO(runtime::rtio_mgt::drtio): [LINK#0] link RX became up, pinging
[ 2172.255595s] INFO(runtime::mgmt): new connection from 192.168.1.10:50709

sb10q

How are you connecting the DRTIO satellite? Is it powered up and is everything OK with it? Please check its log as well using the serial console on USB.
If you cannot quickly get DRTIO working properly, let's focus on the original issue at first. Disconnect the cable completely so DRTIO errors stop flooding the log.
I don't see connections from artiq_sinara_tester in this log. Please run it and capture the log again (without rebooting Kasli), which should contain more information about the problem.

armo13579

Running log with satellite disconnected after running artiq_sinara_tester (running artiq_sinara_tester resulted in the same error response as previously stated):

(artiq) artiq@artiq-VirtualBox:~/tester$ artiq_coremgmt log
[ 0.000015s] INFO(runtime): ARTIQ runtime starting...
[ 0.003937s] INFO(runtime): software ident 7.0.69b6426.beta;waterloo2master
[ 0.010998s] INFO(runtime): gateware ident 7.0.69b6426.beta;waterloo2master
[ 0.018079s] INFO(runtime): log level set to INFO by default
[ 0.023797s] INFO(runtime): UART log level set to INFO by default
[ 0.154608s] WARN(runtime::rtio_clocking): rtio_clock setting not recognised. Falling back to default.
[ 0.162712s] INFO(runtime::rtio_clocking): using internal 125MHz RTIO clock
[ 0.439200s] INFO(board_artiq::si5324): waiting for Si5324 lock...
[ 8.436124s] INFO(board_artiq::si5324): ...locked
[ 8.441258s] INFO(runtime::rtio_clocking::crg): Using internal RTIO clock
[ 8.472382s] INFO(runtime): network addresses: MAC=e8-eb-1b-45-c1-fe IPv4=192.168.1.75 IPv6-LL=fe80::eaeb:1bff:fe45:c1fe IPv6=no configured address
[ 8.486087s] WARN(board_artiq::drtio_routing): length of the configured routing table is incorrect
[ 8.493843s] INFO(board_artiq::drtio_routing): could not read routing table from configuration, using default
[ 8.503854s] INFO(board_artiq::drtio_routing): routing table: RoutingTable { 0: 0; 1: 1 0; 2: 2 0; 3: 3 0; }
[ 8.517814s] INFO(runtime::mgmt): management interface active
[ 8.529845s] INFO(runtime::session): accepting network sessions
[ 8.542939s] INFO(runtime::session): running startup kernel
[ 8.547391s] INFO(runtime::session): no startup kernel found
[ 8.553208s] INFO(runtime::session): no connection, starting idle kernel
[ 8.560040s] INFO(runtime::session): no idle kernel found
[ 8.565460s] INFO(runtime::rtio_mgt::drtio): [DEST#0] destination is up
[ 19.026260s] INFO(runtime::mgmt): new connection from 192.168.1.10:52010
[ 62.576399s] INFO(runtime::session): new connection from 192.168.1.10:52013
[ 62.628999s] INFO(kernel): panic at ksupport/lib.rs:523:5: Exception(LoadFault) at PC 0x450601a4, trap value 0x45061014
[ 62.639097s] ERROR(runtime::session): session aborted: unexpected request RunAborted from kernel CPU
[ 62.650636s] INFO(runtime::session): no connection, starting idle kernel
[ 62.656289s] INFO(runtime::session): no idle kernel found
[ 239.740733s] INFO(runtime::mgmt): new connection from 192.168.1.10:52026
[ 271.784098s] INFO(runtime::mgmt): new connection from 192.168.1.10:52031
[ 291.103402s] INFO(runtime::session): new connection from 192.168.1.10:52033
[ 291.156102s] INFO(kernel): panic at ksupport/lib.rs:523:5: Exception(LoadFault) at PC 0x450601a4, trap value 0x45061014
[ 291.166202s] ERROR(runtime::session): session aborted: unexpected request RunAborted from kernel CPU
[ 291.177790s] INFO(runtime::session): no connection, starting idle kernel
[ 291.183437s] INFO(runtime::session): no idle kernel found
[ 294.233856s] INFO(runtime::mgmt): new connection from 192.168.1.10:52034