Kasli-SoC rejecting connection with host computer - ARTIQ 9

benjaminb

While setting up a Kasli-SoC machine with ARTIQ 9 I am running into connection errors whenever I try to get the log or reboot the machine using artiq_coremgmt log or artiq_coremgmt reboot.
Here is the error message from trying to run artiq_coremgmt log:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:/msys64/clang64/bin/artiq_coremgmt.exe/__main__.py", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/frontend/artiq_coremgmt.py", line 149, in main
    print(mgmt.get_log(), end="")
          ~~~~~~~~~~~~^^
  File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/coredevice/comm_mgmt.py", line 127, in get_log
    self._read_expect(Reply.LogContent)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/coredevice/comm_mgmt.py", line 108, in _read_expect
    header = self._read_header()
  File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/coredevice/comm_mgmt.py", line 102, in _read_header
    ty = Reply(*struct.unpack("B", self._read(1)))
                                   ~~~~~~~~~~^^^
  File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/coredevice/comm_mgmt.py", line 95, in _read
    rn = self.socket.recv(min(8192, length - len(r)))
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

I can get the log if I use artiq_coremgmt -s 1 log, but it does not work with other DRTIO destinations. Here is the log after both commands are run:

[  7893.433405s]  INFO(runtime::mgmt): received connection
[  7893.435102s]  WARN(runtime::mgmt): connection terminated: UnrecognizedPacket
[  7942.578747s]  INFO(runtime::mgmt): received connection

A similar error is generated when I try to run artiq_coremgmt reboot. Additionally, if I try to run artiq_session, I run into a separate but similar error:

<session> starting master
<master> ARTIQ master is now ready.
<session> starting controller manager
<ctlmgr> ARTIQ controller manager is now running.
<ctlmgr> ERROR:controller(core_log):artiq.frontend.aqctl_corelog:Logging connection terminating with exception
<ctlmgr> Traceback (most recent call last):
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/asyncio/windows_events.py", line 463, in finish_socket_func
<ctlmgr>     return ov.getresult()
<ctlmgr>            ~~~~~~~~~~~~^^
<ctlmgr> OSError: [WinError 64] The specified network name is no longer available
<ctlmgr>
<ctlmgr> During handling of the above exception, another exception occurred:
<ctlmgr>
<ctlmgr> Traceback (most recent call last):
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/frontend/aqctl_corelog.py", line 67, in get_logs
<ctlmgr>     length, = struct.unpack(endian + "l", await reader.readexactly(4))
<ctlmgr>                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/asyncio/streams.py", line 769, in readexactly
<ctlmgr>     await self._wait_for_data('readexactly')
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/asyncio/streams.py", line 539, in _wait_for_data
<ctlmgr>     await self._waiter
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/asyncio/proactor_events.py", line 286, in _loop_reading
<ctlmgr>     length = fut.result()
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/asyncio/windows_events.py", line 804, in _poll
<ctlmgr>     value = callback(transferred, key, ov)
<ctlmgr>   File "C:/msys64/clang64/lib/python3.14/asyncio/windows_events.py", line 467, in finish_socket_func
<ctlmgr>     raise ConnectionResetError(*exc.args)
<ctlmgr> ConnectionResetError: [WinError 64] The specified network name is no longer available
<master> ERROR:controller(core_log):artiq.frontend.aqctl_corelog:Logging connection terminating with exception
<master> Traceback (most recent call last):
<master>   File "C:/msys64/clang64/lib/python3.14/asyncio/windows_events.py", line 463, in finish_socket_func
<master>     return ov.getresult()
<master>            ~~~~~~~~~~~~^^
<master> OSError: [WinError 64] The specified network name is no longer available
<master>
<master> During handling of the above exception, another exception occurred:
<master>
<master> Traceback (most recent call last):
<master>   File "C:/msys64/clang64/lib/python3.14/site-packages/artiq/frontend/aqctl_corelog.py", line 67, in get_logs
<master>     length, = struct.unpack(endian + "l", await reader.readexactly(4))
<master>                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
<master>   File "C:/msys64/clang64/lib/python3.14/asyncio/streams.py", line 769, in readexactly
<master>     await self._wait_for_data('readexactly')
<master>   File "C:/msys64/clang64/lib/python3.14/asyncio/streams.py", line 539, in _wait_for_data
<master>     await self._waiter
<master>   File "C:/msys64/clang64/lib/python3.14/asyncio/proactor_events.py", line 286, in _loop_reading
<master>     length = fut.result()
<master>   File "C:/msys64/clang64/lib/python3.14/asyncio/windows_events.py", line 804, in _poll
<master>     value = callback(transferred, key, ov)
<master>   File "C:/msys64/clang64/lib/python3.14/asyncio/windows_events.py", line 467, in finish_socket_func
<master>     raise ConnectionResetError(*exc.args)
<master> ConnectionResetError: [WinError 64] The specified network name is no longer available
<ctlmgr> WARNING:ctlmgr(ESC-MZ03JWVM):artiq_comtools.ctlmgr:Controller core_log exited
<ctlmgr> WARNING:ctlmgr(ESC-MZ03JWVM):artiq_comtools.ctlmgr:Restarting in 5.0 seconds
<master> WARNING:ctlmgr(ESC-MZ03JWVM):artiq_comtools.ctlmgr:Controller core_log exited
<master> WARNING:ctlmgr(ESC-MZ03JWVM):artiq_comtools.ctlmgr:Restarting in 5.0 seconds

Here is the log after running artiq_session:

[  8116.505753s]  INFO(runtime::mgmt): received connection
[  8116.515380s]  WARN(runtime::mgmt): connection terminated: UnrecognizedPacket
[  8116.515395s]  INFO(runtime::moninj): received connection
[  8121.760633s]  INFO(runtime::mgmt): received connection
[  8121.768405s]  WARN(runtime::mgmt): connection terminated: UnrecognizedPacket
[  8123.495696s]  WARN(runtime::moninj): connection terminated: network error: illegal operation
[  8288.258092s]  INFO(runtime::mgmt): received connection

I have both the machine and the host computer on the same IP subnet and they ping successfully. It seems like the machine is unable to recognize some commands so it closes the socket connection prematurely. What is causing the issue?

jfernand

Maybe your ARTIQ software is not the same version your Kasli was flashed with? What does the log say about software version? In the log, just after rebooting, it might say something like...

[0.003899s] INFO(runtime): software ident 8.89614344d44e;<place><date>

benjaminb

jfernand

Here is the gateware version listed in the boot log:

[     0.005238s]  INFO(runtime): gateware ident: 8+fd5cbc9f;byu

Is the version old enough that it wouldn't be compatible with ARTIQ 10? For reference, the host computer has ARTIQ v10.10213+6ebff44.beta (we just recently ran pacman -Syu for updates so the host computer is no longer running ARTIQ 9).

Guest

benjaminb Is the version old enough that it wouldn't be compatible with ARTIQ 10?

Yes. ARTIQ-10 is only compatible with itself.

Recommendation is to install ARTIQ-9 and flash ARTIQ-9 firmware into your boards.

benjaminb we just recently ran pacman -Syu for updates so the host computer is no longer running ARTIQ 9

If you select the correct channel then it would stay on -9.

benjaminb

How is the correct channel selected to stay on -9?

jose

The current stable version is Artiq-9, so to rollback to that version you should edit your C:\msys64\etc\pacman.conf file and add at the end:

[artiq]
SigLevel = Optional TrustAll
Server = https://msys2.m-labs.hk/artiq

You can find the detailed steps here