Also, more related to the actual question at hand- I like the idea of running something with higher priority as a way to pause a dashboard experiment which makes regular calls to pause and check_pause. But what if I want the experiment to pause itself??
One nice reason to do this would be to have a long and detailed spectroscopy run check back on a known line frequency now and then, to make sure that the spectroscopic lightsource hasn't come unlocked. If the program found this to be the case, it would ideally pause itself, wait for me to relock the laser and run some higher priority script to update the known line frequencies in case of any drift, and then go back to operation.
The problem I'm encountering is that if I just have the experiment call an RPC that waits around for user input, if too much time passes it just throws a ConnectionReset Error once it finally receives user input (for example 20 minutes later) and tries to get going.