Help request with scheduling stages

lriesebos

han94ros the information you provided does not allow me to understand at what exact line the underflow happens. Regardless, it is likely that you will have an underflow exception during the second "active stage" loop. You add 1 second of slack at the beginning of the kernel, which is probably enough for the first "active stage" loop. At the end of that loop, you sample your ADC's, which will result in negative slack. In the last line of your kernel, you add a delay which could give you positive slack again if large enough, but if not, you will experience an underflow exception at the start of the second loop. You will need to make sure to regain slack after every iteration, increasing the existing 1 second slack at the start of your kernel outside of the loop will not help.

han94ros

lriesebos Thanks for the reply. The issue is that I need to run these iterations (active stages) successively without delay. Each stage needs to run for example for 200ms and the whole sequence of stages needs to run within the desired time (back to back), therefore I had gone for a single kernel as you had suggested. I cannot therefore place a delay within each stage/loop iteration. Do you believe I would have no choice but to introduce this delay, roughly in the span of microseconds?

lriesebos

han94ros First of all, you will need to confirm that my hypothesis is correct. If that is the case, then you could try one of the following ideas. The current driver of the sampler does not seem to provide direct control of buffering (i.e. scheduling the sampling on a timestamp and recovering the results at a later time). If possible you could try to modify the driver such that it does, allowing you to not lose all slack when sampling. Alternatively, you could modify the code to first schedule all DDS operations without sampling, revert the timeline, and then schedule only the sampling operations at the right time. You will need a large amount of initial slack to do this kind of timeline reversing.

han94ros

lriesebos You were absolutely bang on about the ADC creating the negative slack! I have measured the slack after each operation and there is a dramatic decrease in the slack after the ADC sampling at the end.

INFO:worker(7397,Modulation_frequency_test_129.py):print:1154110192 slack 12 INFO:worker(7397,Modulation_frequency_test_129.py):print:-118192 slack 1

Have you seen examples of this type of timeline reversing.

han94ros

lriesebos would timeline_reversing simply involve using at_mu

han94ros

@sb10q Would the modification of the driver to provide direct control of buffering be a difficult task, could you please advise me on how I could go about this?

lriesebos

han94ros Yes, at_mu() is probably what you will use to revert the timeline to earlier decided timestamps allowing you to schedule the sampling operations after all DDS operations are scheduled. I do not have a ready-made example for you at this moment.

han94ros

lriesebos No thats fine, thanks for the confirmation.

han94ros

I have saved all the output signals (DDS, Analog and TTL) using the DMA and I am able to play this back correctly within a given period for each stage. Unfortunately I also need to include the sampler in each stage and this is causing RTIO underflow issues as the RTIO wall clock and the kernel timeline cursor are being brought to the same point. I have been advised to revert the timeline using at_mu and then schedule the sampler. However even while doing this I have not been able to remove the underflow error. I have been trying to integrate the sampler in the main script for the last two weeks, with and without the use of the DMA to save the output signals, but neither method has been successful. Please find the snippet of the relevant section of the code attached. I would really appreciate advice on this as this is the last section required to get the entire system running and I have been stuck on this for two weeks.

        
        
    @kernel
def run(self):
    self.core.reset()
    self.record()
    for m in self.list_of_active_stages:
        
        self.list_of_pulses_handle[m] = self.core_dma.get_handle(self.list_of_stages[m])
        print(self.list_of_pulses_handle, "yes")
        
    
    delay(2 *s)
    # self.core.break_realtime()
        
    for h in range(self.number_of_repetitions):
        # i = 0 
        # i = np.int64(i)
        for i in self.list_of_active_stages:
            
            t0 = now_mu()
        
            self.core_dma.playback_handle(self.list_of_pulses_handle[i])
            self.list_of_t1[i] = now_mu()
            
            


            delay((self.list_t_duration[i] * ns) - (now_mu()-t0)*ns )

    
    delay(5 * s)      
    for hh in range(self.number_of_repetitions):
        
        for ii in self.list_of_active_stages:
            

            at_mu(self.list_of_t1[ii])
            self.sampler0.sample(self.list_of_sampler_values[ii])

han94ros

Can someone please explain the difference between
delay((self.list_t_duration[i] * ns) - (now_mu()-t0)*ns )
and

t1 = now_mu()
delay((self.list_t_duration[i] * ns) - (t1-t0)*ns )

The latter is allowing my script to work whereas the former is not. Is it an issue with recursion with python/artiq?
This is allowing the original script to work without the use of timeline reversal and the inclusion of the sampler. I can run the script down to 300*us given the slack of sampler0.sample being roughly 100us. Can someone explain this using time concepts.

lriesebos

han94ros Does that mean your code is working as expected at this moment?

For the two different code snippets: your conversions are not that great, which might cause unexpected issues with 32/64 bit integer casting. Timestamps (such as returned by now_mu()) are in machine units (int64) and should be used with delay_mu(). If you want to convert them to seconds for using with delay(), use self.core.mu_to_seconds().

han94ros

lriesebos Hey
I had to introduce DMA functionality to save all the output signals first but after that I was fixated on the sampler as the culprit when the issue was my conversions in the delay function. Once I realised the time stamps where the issue it was easy to solve. I have implemented your advice above and the whole sequence is working - I have to test it carefully before I can celebrate though! I will be sure to let you know if everything goes as expected. Thanks for the help.

han94ros

@jonhood Hi Jon, I was wondering if you have time to look at my sequence which utilises ramps and DMA. The whole sequence is running only when I remove the sampler. The previous successful run I mentioned in this thread was a red herring due to accidental scaling of the delay. The sampler brings the kernel cursor and RTIO wall clock together and my subsequent delay is not sufficient to provide enough positive slack for the next iteration (which represents a stage). I cannot increase this delay as each stage needs to run for a set period of time. The issue is not to do with the usage of DMA as the sequence runs when the sampler sample function is commented out and I have tested the script in seconds and machine units e.g delay_mu. I need to sample in real time e.g whilst playing back the DMA therefore recording the samples is not a possibility. Recording the delay for each stage with the DMA Obviously there are no problems with ARTIQ or the Sinara hardware rather it is a code architectural issue and previous posters in this thread have suggested the use of timeline reversal in addition to DMA, do you have experience of this. I would really appreciate it if you or your graduate student @dpeana could advise me on this issue. I believe this is the last piece required for completion of the project.

han94ros

This is the relevant section of the code and my attempt at time line reversal. Am I missing a trick because I don't believe it should be this difficult to include the sampler.

@lriesebos Do you have any ideas?
`

 @kernel
def run(self):
    self.core.reset()
    self.record()
    for m in self.list_of_active_stages:
    
    self.list_of_pulses_handle[m] = self.core_dma.get_handle(self.list_of_stages[m])
        
    
    delay(1 *s)
        
    for h in range(self.number_of_repetitions):

        for i in self.list_of_active_stages:
            
            t0 = now_mu()
            
            self.ttlcounter_0.gate_rising((self.list_t_duration[i]*0.9) * ns)
            self.ttlcounter_1.gate_rising((self.list_t_duration[i]*0.9) * ns)
            self.ttlcounter_2.gate_rising((self.list_t_duration[i]*0.9) * ns)
            self.ttlcounter_3.gate_rising((self.list_t_duration[i]*0.9) * ns)
            
            self.sampler0.sample(self.list_of_sampler_values[i])

            
            self.list_of_t__1[i] = now_mu()
            
        
            #self.core_dma.playback_handle(self.list_of_pulses_handle[i])

             
            

            self.list_of_ttl_rising_edges_0[i] = (self.ttlcounter_0.fetch_count())
            self.list_of_ttl_rising_edges_1[i] = (self.ttlcounter_1.fetch_count())
            self.list_of_ttl_rising_edges_2[i] = (self.ttlcounter_2.fetch_count())
            self.list_of_ttl_rising_edges_3[i] = (self.ttlcounter_3.fetch_count())
            
            delay((self.list_t_duration[i] * ns) - ((self.core.mu_to_seconds((now_mu()-t0)))))
            
    delay(1 *s)
        
    for hh in range(self.number_of_repetitions):

        for ii in self.list_of_active_stages:
            
            
            delay(5 *s )


            at_mu(self.list_of_t__1[ii])
            self.core_dma.playback_handle(self.list_of_pulses_handle[ii])`

lriesebos

han94ros the code you just shared does not seem to correctly implement the ideas I was trying to explain earlier. Currently you are first doing the sampling, which are the events that are supposed to be added last to the timeline. Now, after the first time sampling, your slack will be negative regardless. That means the second time the gate_rising() section of the code will run, you will get an underflow (unless that delay in self.list_t_duration is large enough, but earlier you indicated these delays are short).

Maybe it is more effective if you just outline how your pulsing/sampling timeline looks like, and I can probably write you some concept code.

han94ros

lriesebos I don't know why I posted this section of code, I had tested it and found an underflow error. I should have clarified that I already tested your original suggestion, please see the code below. I had got an underflow error here and therefore I tried the code I had posted above.

    
@kernel
def run(self):
    self.core.reset()
    self.record()
    for m in self.list_of_active_stages:
    
    
    
    self.list_of_pulses_handle[m] = self.core_dma.get_handle(self.list_of_stages[m])

        
    
    delay(1 *s)

    
        
    for h in range(self.number_of_repetitions):

        for i in self.list_of_active_stages:
            
    

            self.ttlcounter_0.gate_rising((self.list_t_duration[i]*0.9) * ns)
            self.ttlcounter_1.gate_rising((self.list_t_duration[i]*0.9) * ns)
            
            self.ttlcounter_2.gate_rising((self.list_t_duration[i]*0.9) * ns)
            self.ttlcounter_3.gate_rising((self.list_t_duration[i]*0.9) * ns)
            

            
            
            t0 = now_mu()

        
            self.core_dma.playback_handle(self.list_of_pulses_handle[i])

            at_mu(self.list_of_t__1[i])




            
            self.list_of_ttl_rising_edges_0[i] = (self.ttlcounter_0.fetch_count())
            self.list_of_ttl_rising_edges_1[i] = (self.ttlcounter_1.fetch_count())
            self.list_of_ttl_rising_edges_2[i] = (self.ttlcounter_2.fetch_count())
            self.list_of_ttl_rising_edges_3[i] = (self.ttlcounter_3.fetch_count())


            delay((self.list_t_duration[i] * ns) - ((self.core.mu_to_seconds((now_mu()-t0)))))
   
                
        
        
        for hh in range(self.number_of_repetitions):

            for ii in self.list_of_active_stages:
                

                at_mu(self.list_of_t__1[ii])
                self.sampler0.sample(self.list_of_sampler_values[ii])`

Even then i'm not sure this is exactly implementing what you have suggested before. Anyways I will explain what I am clearly trying to do. There are 7 stages in total,

STAGE 1: time duration: 200ms, running amplitude and frequency modulation and zotino ramping all in parallel. These output signals are first recorded in the DMA and then played back in the run section I have posted above. Within the run section I am sampling the sampler and using the TTL edge counters. Therefore these input lines have to run within the 200ms duration.
STAGE 2: time duration: 20ms, running amplitude and frequency modulation and zotino ramping all in parallel using DMA and input measurements are the same as stage 1 e.g analog input and TTL edges running now within the time duration of 20ms.
STAGE 3: time duration: 10ms, running amplitude and frequency modulation all in parallel using DMA (no zotino ramping) and input measurements are the same as stage 1.
STAGE 4: time duration 1ms, running amplitude and frequency modulation all in parallel using DMA (no zotino ramping) and input measurements are the same as stage 1.
STAGE 5: time duration 300us, no amplitude or frequency modulation, all the output signals are single tone recorded using DMA but the input measurements are the same as stage 1.
STAGE 6: time duration 2ms, running amplitude and frequency modulation all in parallel using DMA (no zotino ramping) and input measurements are the same as stage 1.
STAGE 7: time duration 10ms, running amplitude and frequency modulation all in parallel using DMA (no zotino ramping) and input measurements are the same as stage 1.

I should point out that during each stage I am measuring the time it takes to playback the DMA recording and in the case of any of the ramps e.g AM, FM or Zotino, this time will reach the stage end period and therefore the delay will be a fraction of the stage time duration. Initialisation of the hardware is not an issue as this has been done in preceding scripts. I have measured the slack before and after each sampler function and the issue arises in stage 4 when the stage time duration is 300us and there is not enough delay for that stage to run the next iteration. I would like to change the code to fit the limitations of the sequence rather than the other way around. I can show you my code and walk you through it within 15 minutes through a Zoom/MS meeting. I believe this would save you alot of time as I can run ARTIQ with any modifications you can see the results on the screen share. I have thought about the code modifications but I have not been able to devise a different code architecture which will implement the output and input signals within the specified time frames. To reiterate this runs correctly without the sampler. Removing the DMA playback and running only sampler works as well therefore this issue is with the structure of the code rather than the individual hardware components. As you have probably read I have been stuck on this issue for more than two weeks now, please let me know if you are interested in an online video meeting.

han94ros

lriesebos I had posted my original code which first scheduled the DDS operations and then scheduled the sampling as you had suggest on Friday 30th July, please see the post in this thread. I was still getting an underflow error. Could you please describe the timeline reversal in conceptual terms.

han94ros

@sb10q Do you think timeline reversal will help solve this issue, do you have any example codes?

han94ros

@sb10q Would controlling the buffering of the sampler to sample on a timestamp but recover the results at a later time avoid losing all the slack. To do this I assume I would need to build the gateware again from source, as documented in this post https://github.com/m-labs/artiq/issues/1693. I would make the appropriate change in sampler.py to add a delay between the write update and the read function, however I am not sure what to change in the target gateware file for the Kasli and enact this change?

« Previous Page Next Page »