timeout

ChenglongSun

Hi, I used artiq to control kc705 in our experiment. But, it's awalys reporting mistakes "timeout" recently.
Here 's what it says,
root:Terminating with exception (timeout: timed out)
Traceback (most recent call last):
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\master\worker_impl.py", line 228, in main
exp_inst.run()
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\language\core.py", line 54, in run_on_core
return getattr(self, arg).run(run_on_core, ((self,) + k_args), k_kwargs)
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\coredevice\core.py", line 118, in run
self.comm.check_ident()
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\coredevice\comm_generic.py", line 227, in check_ident
self.write_empty(H2DMsgType.IDENT_REQUEST)
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\coredevice\comm_generic.py", line 194, in write_empty
self.write_header(ty)
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\coredevice\comm_generic.py", line 176, in _write_header
self.open()
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\coredevice\comm_tcp.py", line 46, in open
self.socket = initialize_connection(self.host, self.port)
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\site-packages\artiq\coredevice\comm_tcp.py", line 28, in initialize_connection
sock = socket.create_connection((host, port), 5.0)
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\socket.py", line 711, in create_connection
raise err
File "D:\ProgramData\Anaconda3\envs\artiq_2_2\lib\socket.py", line 702, in create_connection
sock.connect(sa)
socket.timeout: timed

sb10q

Check network settings and look at UART output when the KC705 is booting.

ChenglongSun

sb10q Hi，sb 10q ,I have upgraded my software from artiq2 to artiq 4.0, and reburn the artiq 4 for my FPGA. No 'timeout' reported for the time being.

ChenglongSun

hi, sb! Even if I use artiq 4.0, but it always 'timeout'. I look at the UART output,
Here's what it says,
[09:13:56.780] +0000: 88800003 d8032008 e9630000 9c220000
[09:13:56.780] +0010: 44004800 8441fffc d7e14ffc d7e117f8
[09:13:56.780] +0020: 8c410000 9c21ffac d3e2e7f4 d7e2d7f0
[09:13:56.780] +0030: d7e2c7ec d7e2b7e8 d7e2a7e4 d7e297e0
[09:13:56.780] @ 0x000000
[09:13:56.780] +0000: ffffffff ffffffff ffffffff ffffffff
[09:13:56.780] +0010: ffffffff ffffffff ffffffff ffffffff
[09:13:56.780] +0020: 000000bb 11220044 ffffffff ffffffff
[09:13:56.780] +0030: aa995566 20000000 3003e001 0000026b
[09:13:56.780] panic at runtime/main.rs:343:13: exception Alignment at PC 0x40035014, EA 0x3
[09:13:56.780] backtrace for software version 4.0.dev+1311.gbd968211;nist_clock:
[09:13:56.780] 0x400032a8
[09:13:56.800] 0x40042ac8
[09:13:56.821] 0x40002cf0
[09:13:56.821] 0x400010d0
[09:13:56.821] halting.
[09:13:56.846] use artiq_coreconfig write -s panic_reset 1 to restart instead
[09:13:56.846]
I don't know how to solve the problem.

sb10q

ChenglongSun [09:13:56.780] +0000: 88800003 d8032008 e9630000 9c220000
[09:13:56.780] +0010: 44004800 8441fffc d7e14ffc d7e117f8
[09:13:56.780] +0020: 8c410000 9c21ffac d3e2e7f4 d7e2d7f0
[09:13:56.780] +0030: d7e2c7ec d7e2b7e8 d7e2a7e4 d7e297e0
[09:13:56.780] @ 0x000000
[09:13:56.780] +0000: ffffffff ffffffff ffffffff ffffffff
[09:13:56.780] +0010: ffffffff ffffffff ffffffff ffffffff
[09:13:56.780] +0020: 000000bb 11220044 ffffffff ffffffff
[09:13:56.780] +0030: aa995566 20000000 3003e001 0000026b

What is this? Does the UART actually output these exact characters?
Is there any text before this?

ChenglongSun

hi，I use serial communication software to monitor the UART actually output.
When I restart FPGA, it will show these under characters.
[19:42:53.213] MiSoC Bootloader
[19:42:53.213] Copyright (c) 2017-2018 M-Labs Limited
[19:42:53.213]
[19:42:53.213] Bootloader CRC passed
[19:42:53.236] Gateware ident 4.0.dev+1311.gbd968211;nist_clock
[19:42:53.236] Initializing SDRAM...
[19:42:53.236] Write leveling scan:
[19:42:53.280] Module 7:
[19:42:53.280] 00000111111111111100000000000000
[19:42:53.280] Module 6:
[19:42:53.280] 00001111111111111000000000000000
[19:42:53.280] Module 5:
[19:42:53.280] 00000000111111111111100000100000
[19:42:53.280] Module 4:
[19:42:53.280] 00000000011111111111110000110000
[19:42:53.280] Module 3:
[19:42:53.280] 10000000000000111111111111111111
[19:42:53.280] Module 2:
[19:42:53.280] 10000000000000111111111111111111
[19:42:53.280] Module 1:
[19:42:53.280] 11000000000000011111111111111111
[19:42:53.280] Module 0:
[19:42:53.280] 11100000000000000111111111111111
[19:42:53.280] Write leveling: 17* 15* 14* 14* 8 8 4 5 done
[19:42:53.280] Read bitslip: 7 6 5 4 3 2
[19:42:53.280] Read leveling scan:
[19:42:53.280] Module 7:
[19:42:53.280] 00011111111110000000000000000000
[19:42:53.351] Module 6:
[19:42:53.351] 00111111111111000000000000000000
[19:42:53.411] Module 5:
[19:42:53.411] 00000111111111100000000000000000
[19:42:53.481] Module 4:
[19:42:53.481] 00000011111111110000000000000000
[19:42:53.552] Module 3:
[19:42:53.552] 00000000000111111111100000000000
[19:42:53.612] Module 2:
[19:42:53.612] 00000000000111111111000000000000
[19:42:53.680] Module 1:
[19:42:53.680] 11111111110000000000000000000001
[19:42:53.741] Module 0:
[19:42:53.741] 01111111110000000000000000000000
[19:42:53.808] Read leveling: 7+-5 7+-5 9+-5 10+-5 16+-4 15+-4 4+-4 5+-4 done
[19:42:53.978] SDRAM initialized
[19:42:53.978] Memory test passed
[19:42:55.315]
[19:42:55.315] Booting from flash...
[19:42:55.315] Starting firmware.
[19:42:55.708] [ 0.000005s] INFO(runtime): ARTIQ runtime starting...
[19:42:55.763] [ 0.003855s] INFO(runtime): software ident 4.0.dev+1311.gbd968211;nist_clock
[19:42:55.763] [ 0.010988s] INFO(runtime): gateware ident 4.0.dev+1311.gbd968211;nist_clock
[19:42:55.763] [ 0.018126s] INFO(runtime): log level set to INFO by default
[19:42:55.763] [ 0.023843s] INFO(runtime): UART log level set to INFO by default
[19:42:55.763] [ 0.029994s] INFO(runtime): press 'e' to erase startup and idle kernels...
[19:42:55.763] [ 1.029006s] INFO(runtime): continuing boot
[19:42:56.769] [ 1.032196s] INFO(runtime): using MAC address 00-0a-35-03-51-1f
[19:42:56.769] [ 1.037978s] INFO(runtime): using IP address 192.168.1.202
[19:42:56.769] [ 1.047624s] INFO(runtime::rtio_mgt): using internal RTIO clock (by default)
[19:42:56.779] [ 1.055026s] INFO(runtime::mgmt): management interface active
[19:42:56.785] [ 1.068191s] INFO(runtime::session): accepting network sessions
[19:42:56.799] [ 1.082261s] INFO(runtime::session): running startup kernel
[19:42:56.830] [ 1.086701s] INFO(runtime::session): no startup kernel found
[19:42:56.830] [ 1.092434s] INFO(runtime::session): no connection, starting idle kernel
[19:42:56.830] [ 1.099298s] INFO(runtime::session): no idle kernel found
[19:42:56.830] [ 15.548629s] INFO(runtime::moninj): new connection from 192.168.1.103:50369
[19:43:11.280] [ 22.106201s] INFO(runtime::session): new connection from 192.168.1.103:50370
[19:43:17.838] [ 22.142900s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:17.873] [ 22.490214s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:18.220] [ 22.833817s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:18.564] [ 23.209632s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:18.940] [ 23.551277s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:19.281] [ 23.892784s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:19.623] [ 24.268767s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:19.999] [ 24.402044s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.132] [ 24.548082s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.278] [ 24.681023s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.411] [ 24.814291s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.544] [ 24.947569s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.677] [ 25.094344s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.824] [ 25.226854s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:20.957] [ 25.392769s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:21.122] [ 25.538410s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:21.268] [ 25.682592s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:21.412] [ 25.828785s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:21.559] [ 25.929300s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:21.659] [ 25.944536s] INFO(runtime::session): no connection, starting idle kernel
[19:43:21.681] [ 25.950480s] INFO(runtime::session): no idle kernel found
[19:43:21.681] [ 38.182256s] INFO(runtime::session): new connection from 192.168.1.103:50371
[19:43:33.914] [ 38.220136s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:33.950] [ 38.233383s] INFO(runtime::session): no connection, starting idle kernel
[19:43:33.970] [ 38.239375s] INFO(runtime::session): no idle kernel found
[19:43:33.970] [ 48.593036s] INFO(runtime::session): new connection from 192.168.1.103:50372
[19:43:44.324] [ 48.630669s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:44.360] [ 48.644257s] INFO(runtime::session): no connection, starting idle kernel
[19:43:44.380] [ 48.650247s] INFO(runtime::session): no idle kernel found
[19:43:44.380] [ 63.672365s] INFO(runtime::session): new connection from 192.168.1.103:50375
[19:43:59.403] [ 63.710119s] INFO(runtime::kern_hwreq): resetting RTIO
[19:43:59.439] [ 63.723364s] INFO(runtime::session): no connection, starting idle kernel
[19:43:59.459] [ 63.729355s] INFO(runtime::session): no idle kernel found
When I run my program, sometimes artiq will report 'timeout'. Meanwhile ,the UART output is as follows.
[09:13:56.780] +0000: 88800003 d8032008 e9630000 9c220000
[09:13:56.780] +0010: 44004800 8441fffc d7e14ffc d7e117f8
[09:13:56.780] +0020: 8c410000 9c21ffac d3e2e7f4 d7e2d7f0
[09:13:56.780] +0030: d7e2c7ec d7e2b7e8 d7e2a7e4 d7e297e0
[09:13:56.780] @ 0x000000
[09:13:56.780] +0000: ffffffff ffffffff ffffffff ffffffff
[09:13:56.780] +0010: ffffffff ffffffff ffffffff ffffffff
[09:13:56.780] +0020: 000000bb 11220044 ffffffff ffffffff
[09:13:56.780] +0030: aa995566 20000000 3003e001 0000026b
[09:13:56.780] panic at runtime/main.rs:343:13: exception Alignment at PC 0x40035014, EA 0x3
[09:13:56.780] backtrace for software version 4.0.dev+1311.gbd968211;nist_clock:
[09:13:56.780] 0x400032a8
[09:13:56.800] 0x40042ac8
[09:13:56.821] 0x40002cf0
[09:13:56.821] 0x400010d0
[09:13:56.821] halting.
[09:13:56.846] use artiq_coreconfig write -s panic_reset 1 to restart instead
I just want to know how to solve 'timeout', because before no this problem.

sb10q

Try replacing the SDRAM module in the KC705. Sometimes that helps.