Real time peak identification
-
I have a stream of data coming from a respiration belt and I need a good, robust algorithm to help me identify the peaks and troughs as the data is coming in (i.e. live data, not later analysis). The data is roughly sinusoidal with a period of a maybe 2 - 4 seconds. The data is polled about once every 100 ms. The data is noisy, but it's not too bad. We sometimes see sudden spikes if the person wearing the belt fidgets around, but if they are sitting still, it's fairly clean. Obviously you can't identify a peak or a trough until after it's happened, but I'm trying to find the best and fastest way to identify them so we can use them as a trigger for something else. I understand that there are trade-offs involved. I tried calculating an instantaneous derivative by comparing the change in signal between the current point and the second to last point and then looking for when that value falls within some threshold. That kinda works, but is a bit touchy. I tried to improve it by checking that when a point falls within the threshold it also has the opposite sign to the last identified peak/trough to try and eliminate having multiple points right around the peak identified as the peak. That helps a little. I also tried introducing, basically a filter, such that when a peak (or trough) is identified, I stop looking for another one for some period of time (say 1 second). Does anybody have any better ideas?
Removing spikes could be easily done by an FFT (converting to frequency spectrum), a low pass (removing high frequency components) and a reverse FFT 8going back to the time signal again). However, this pretty much kills your response time. There are low-pass filter implementaitons in the time domain (wikipedia 1[^], wikipedia 2[^]) (I don't know wnough to recommend a specific algorithm, I am just parroting what "the guys" do.)
FILETIME to time_t
| FoldWithUs! | sighist | WhoIncludes - Analyzing C++ include file hierarchy -
I have a stream of data coming from a respiration belt and I need a good, robust algorithm to help me identify the peaks and troughs as the data is coming in (i.e. live data, not later analysis). The data is roughly sinusoidal with a period of a maybe 2 - 4 seconds. The data is polled about once every 100 ms. The data is noisy, but it's not too bad. We sometimes see sudden spikes if the person wearing the belt fidgets around, but if they are sitting still, it's fairly clean. Obviously you can't identify a peak or a trough until after it's happened, but I'm trying to find the best and fastest way to identify them so we can use them as a trigger for something else. I understand that there are trade-offs involved. I tried calculating an instantaneous derivative by comparing the change in signal between the current point and the second to last point and then looking for when that value falls within some threshold. That kinda works, but is a bit touchy. I tried to improve it by checking that when a point falls within the threshold it also has the opposite sign to the last identified peak/trough to try and eliminate having multiple points right around the peak identified as the peak. That helps a little. I also tried introducing, basically a filter, such that when a peak (or trough) is identified, I stop looking for another one for some period of time (say 1 second). Does anybody have any better ideas?
Make several simple calculations and see which one works for you on sample/real data. *slope / gradient (as mentioned in other replies) *keep 6/10/12 etc.. Most recent highs - to get a rolling average of them. *Exponential moving average (more responsive that a moving avergage) Collect a few of these and you will find that one of them in comparison to the others gives you the indication than the samples are 'out of band'. Which is what I understand you to be seeking. Most of the calculations above can be maintained on a rolling basis without having to keep too much data.
-
I have a stream of data coming from a respiration belt and I need a good, robust algorithm to help me identify the peaks and troughs as the data is coming in (i.e. live data, not later analysis). The data is roughly sinusoidal with a period of a maybe 2 - 4 seconds. The data is polled about once every 100 ms. The data is noisy, but it's not too bad. We sometimes see sudden spikes if the person wearing the belt fidgets around, but if they are sitting still, it's fairly clean. Obviously you can't identify a peak or a trough until after it's happened, but I'm trying to find the best and fastest way to identify them so we can use them as a trigger for something else. I understand that there are trade-offs involved. I tried calculating an instantaneous derivative by comparing the change in signal between the current point and the second to last point and then looking for when that value falls within some threshold. That kinda works, but is a bit touchy. I tried to improve it by checking that when a point falls within the threshold it also has the opposite sign to the last identified peak/trough to try and eliminate having multiple points right around the peak identified as the peak. That helps a little. I also tried introducing, basically a filter, such that when a peak (or trough) is identified, I stop looking for another one for some period of time (say 1 second). Does anybody have any better ideas?
-
I have a stream of data coming from a respiration belt and I need a good, robust algorithm to help me identify the peaks and troughs as the data is coming in (i.e. live data, not later analysis). The data is roughly sinusoidal with a period of a maybe 2 - 4 seconds. The data is polled about once every 100 ms. The data is noisy, but it's not too bad. We sometimes see sudden spikes if the person wearing the belt fidgets around, but if they are sitting still, it's fairly clean. Obviously you can't identify a peak or a trough until after it's happened, but I'm trying to find the best and fastest way to identify them so we can use them as a trigger for something else. I understand that there are trade-offs involved. I tried calculating an instantaneous derivative by comparing the change in signal between the current point and the second to last point and then looking for when that value falls within some threshold. That kinda works, but is a bit touchy. I tried to improve it by checking that when a point falls within the threshold it also has the opposite sign to the last identified peak/trough to try and eliminate having multiple points right around the peak identified as the peak. That helps a little. I also tried introducing, basically a filter, such that when a peak (or trough) is identified, I stop looking for another one for some period of time (say 1 second). Does anybody have any better ideas?
I've done this with heart rate variability (HRV) and it's rather simple. Determine the frequency range of your useful data and setup a bandpass IIR or FIR filter to filter out all unwanted noise. Use matlab or similar tool to determine the coefficients of the filter. IIR filters are very easy to implement in code as they are just some floating point multiply ops. After your data is filtered, peaks can be easily identified with a simple rule e.g.
if(sample[t] < sample[t-1]). All is in the design of the filter (compromise between delay vs and bandwidth).modified on Thursday, June 2, 2011 3:38 AM
-
You were on the right track with instantaneous derivative. At the actual peak the derivative goes to zero, one side of it is positive, the other side is negative. Remember though that your sample may not be at the peak, so the peak may be between 2 samples rather than the middle sample of a group of 3. So all you need do to is find where the derivatives changes sign, pos to neg is peak, neg to pos is trough. It doesn't tell you the exact point of the peak/trough, just that one happened between the sign changes. Hope this is some use Steven
Thanks for the input Steven. That was pretty much what I started out doing, but it provided to be less robust than I wanted with the noise in the data. Some filtering (based on several other suggestion in this thread) seemed to help and then this method works a lot better.
-
Wjousts, your signal indeed looks nice. If it was perfect, all you would have to do is indeed look for changes in the sign of the derivative, i.e. detect monotonous sequences (increasing then decreasing, in alternation). Added noise causes local perturbations of this dream situation, resulting in small breaches of the monotonocity. The closer you get to the peak, the more likely they get (as the derivative gets smaller and smaller). My way to deal with that is to consider "quasi-increasing sequences" (resp. decreasing), i.e. values that go increasing but allow a backtracking limited by a threshold value. For example, assuming a threshold of 10, the following sequence is quasi-increasing: 0, 22, 31, 27, 45, 63..., while this one is not: 0, 22, 31, 20, 45, 63... (said differently, it stops being quasi-increasing at the value 20). This approach is more robust than mere derivative computation and you can also filter on the total increase (decrease) of the sequence. You can also limit the length of the allowed backtracking to a given number of samples. When will it detect a peak ? When the signal value has decreased to the peak value minus the threshold. I guess that in any case you cannot avoid having to wait some times after a peak, to be sure it is a true one.
-
Removing spikes could be easily done by an FFT (converting to frequency spectrum), a low pass (removing high frequency components) and a reverse FFT 8going back to the time signal again). However, this pretty much kills your response time. There are low-pass filter implementaitons in the time domain (wikipedia 1[^], wikipedia 2[^]) (I don't know wnough to recommend a specific algorithm, I am just parroting what "the guys" do.)
FILETIME to time_t
| FoldWithUs! | sighist | WhoIncludes - Analyzing C++ include file hierarchyThanks Peter. I've used the low pass filter implementation in the second Wikipedia article before with fairly good results, but I wasn't peak picking, just smoothing. I actually have a simple RC filter to electronically filter the data a little, but my electronics knowledge isn't quite up to understanding it properly. I'm not 100% sure that it's really helping at all, but at least it doesn't use up processor cycles (although, maybe it does lag the signal some, like I said, I don't understand it well enough).
-
Thanks YDaoust. I like that idea a lot. It looks like it should be able to deal with shoulders or split peaks fairly nicely so long as a sensible threshold is used. I'll experiment with that idea a little.
-
I have a stream of data coming from a respiration belt and I need a good, robust algorithm to help me identify the peaks and troughs as the data is coming in (i.e. live data, not later analysis). The data is roughly sinusoidal with a period of a maybe 2 - 4 seconds. The data is polled about once every 100 ms. The data is noisy, but it's not too bad. We sometimes see sudden spikes if the person wearing the belt fidgets around, but if they are sitting still, it's fairly clean. Obviously you can't identify a peak or a trough until after it's happened, but I'm trying to find the best and fastest way to identify them so we can use them as a trigger for something else. I understand that there are trade-offs involved. I tried calculating an instantaneous derivative by comparing the change in signal between the current point and the second to last point and then looking for when that value falls within some threshold. That kinda works, but is a bit touchy. I tried to improve it by checking that when a point falls within the threshold it also has the opposite sign to the last identified peak/trough to try and eliminate having multiple points right around the peak identified as the peak. That helps a little. I also tried introducing, basically a filter, such that when a peak (or trough) is identified, I stop looking for another one for some period of time (say 1 second). Does anybody have any better ideas?
I found the following article useful when I was trying to find peaks & troughs in some data:- http://billauer.co.il/peakdet.html[^] Looking at your graph the only slight hiccup would be at 31s, but you can set the delta value to an appropriate value to overcome this I think.
-
I found the following article useful when I was trying to find peaks & troughs in some data:- http://billauer.co.il/peakdet.html[^] Looking at your graph the only slight hiccup would be at 31s, but you can set the delta value to an appropriate value to overcome this I think.
Hi, One good way to remove spikes is to use a median filter. Keep a window of the data, sort the values in the window, and use the one in the center of the range. Don't know if that is appropriate for a real time application, but we use it for seismic data post-processing all the time. Is that sleep-study data? I had to have one a few weeks ago, I have sleep apnea! Regards, Tom