Satellite hardware not accessible after gateware update

Today I upgraded my ARTIQ gateware from version 5 to version 8. (It has been a long time coming!) I am able to access hardware connected to the master after upgrading, but not hardware connected to the satellite.

I have flashed the 8.8881 gateware onto the master and the 8.8864 gateware onto the satellite. I performed the flashes via

artiq_flash -d berkeley3master

when plugged into the master and

artiq_flash -d berkeley3satellite

when plugged into the satellite. I then regenerated my device_db.py file via

artiq_ddb_template -s 1 satellite.json -o device_db.py master.json

After doing this I run artiq_master and artiq_dashboard. Once doing this I am able to successfully connect to hardware on the master (i.e. change TTL levels, etc.), but am unable to do so on the satellite. When I run artiq_coremgmt log I get the following:

alpha@control-system:~/control$ cd ~/control/
alpha@control-system:~/control$ artiq_coremgmt log
[     0.000014s]  INFO(runtime): ARTIQ runtime starting...
[     0.003904s]  INFO(runtime): software ident 8.8881+e8da758;berkeley3master
[     0.010772s]  INFO(runtime): gateware ident 8.8881+e8da758;berkeley3master
[     0.017662s]  INFO(runtime): log level set to INFO by default
[     0.023376s]  INFO(runtime): UART log level set to INFO by default
[     0.029763s]  WARN(runtime::rtio_clocking): Using legacy rtio_clock setting ('i'). Falling back to default. This will be deprecated.
[     0.041415s]  INFO(runtime::rtio_clocking): Clocking has already been set up.
[     0.074043s]  INFO(runtime): network addresses: MAC=04-91-62-c7-0b-dc IPv4=192.168.1.75/0 IPv6-LL=fe80::691:62ff:fec7:bdc/10 IPv6=no configured address
[     0.089636s]  INFO(board_artiq::drtio_routing): routing table: RoutingTable { 0: 0; 1: 1 0; }
[     0.098234s]  WARN(runtime::rtio_mgt): error reading device map (key not found), device names will not be available in RTIO error messages
[     0.109397s]  INFO(runtime::rtio_mgt): SED spreading disabled by default
[     0.117551s]  INFO(runtime::mgmt): management interface active
[     0.128780s]  INFO(runtime::session): accepting network sessions
[     0.133504s]  INFO(runtime::session): running startup kernel
[     0.139361s]  INFO(runtime::session): no startup kernel found
[     0.144878s]  INFO(runtime::session): no connection, starting idle kernel
[     0.159410s]  INFO(runtime::session): no idle kernel found
[     0.163700s]  INFO(runtime::rtio_mgt::drtio): [DEST#0] destination is up
[     5.189552s]  INFO(runtime::mgmt): new connection from 192.168.1.83:51606
[   410.933205s]  INFO(runtime::mgmt): new connection from 192.168.1.83:45550
alpha@control-system:~/control$ 

As far as I can tell no attempt is being made to connect to DEST#1, which is the satellite.

Looking at the satellite serial output I get the following:

alpha@control-system:~/control$ stty 115200 < /dev/ttyUSB2
alpha@control-system:~/control$ cat /dev/ttyUSB2
��
 __  __ _ ____         ____ 
|  \/  (_) ___|  ___  / ___|
| |\/| | \___ \ / _ \| |    
| |  | | |___) | (_) | |___ 
|_|  |_|_|____/ \___/ \____|

MiSoC Bootloader
Copyright (c) 2017-2024 M-Labs Limited

Bootloader CRC passed
Gateware ident 8.8864+f20b605;berkeley3satellite
Initializing SDRAM...
Read leveling scan:
Module 1:
00000001111111111000000000000000
Module 0:
00000001111111111000000000000000
Read leveling: 11+-5 11+-5 done
SDRAM initialized
Memory test passed

Booting from flash...
Starting firmware.
[     0.000017s]  INFO(satman): ARTIQ satellite manager starting...
[     0.004571s]  INFO(satman): software ident 8.8864+f20b605;berkeley3satellite
[     0.011617s]  INFO(satman): gateware ident 8.8864+f20b605;berkeley3satellite
[     0.419325s]  INFO(board_artiq::si5324): waiting for Si5324 lock...
[     2.363512s]  INFO(board_artiq::si5324):   ...locked
[     2.367085s]  INFO(satman): Switching sys clock, rebooting...

 __  __ _ ____         ____ 
|  \/  (_) ___|  ___  / ___|
| |\/| | \___ \ / _ \| |    
| |  | | |___) | (_) | |___ 
|_|  |_|_|____/ \___/ \____|

MiSoC Bootloader
Copyright (c) 2017-2024 M-Labs Limited

Bootloader CRC passed
Gateware ident 8.8864+f20b605;berkeley3satellite
Initializing SDRAM...
Read leveling scan:
Module 1:
00000001111111111000000000000000
Module 0:
00000001111111111000000000000000
Read leveling: 11+-5 11+-5 done
SDRAM initialized
Memory test passed

Booting from flash...
Starting firmware.
[     0.000017s]  INFO(satman): ARTIQ satellite manager starting...
[     0.004573s]  INFO(satman): software ident 8.8864+f20b605;berkeley3satellite
[     0.011619s]  INFO(satman): gateware ident 8.8864+f20b605;berkeley3satellite
[     0.163942s]  INFO(satman): Clocking has already been set up.
[     0.169394s]  INFO(satman): uplink is up, switching to recovered clock
[     0.202626s]  INFO(board_artiq::si5324): waiting for Si5324 lock...
[     1.962496s]  INFO(board_artiq::si5324):   ...locked
[     4.297536s]  INFO(board_artiq::si5324::siphaser): calibration successful, lead: 22, width: 429 (344deg)

No errors are being printed out. Did I make a mistake in configuring the satellite?

Also note that prior to flashing ARTIQ 8 I was able to access the satellite hardware without any issues.