spike forecast

Code: forecast_spike.

To forecast a spike we start first with a good prediction

prediction forecast difference between prediction and forecast

We see that the training process runs smoothly prediction_history traning history

traning recipe

To train the model we proceed in the following way:

denoised

Than we roll different spike advance times until the spike is forecasted

forecast_advance forecast advance effectivity

We can see that around -0.5 seconds the spike is forecasted correctly

forecast_prior forecast prior to the spike

To speed up a bit the exploration we analyze the sec pace series. We see that the forecast for the denoised series is pretty accurate

forecast_anticipation rolling forecast on the next 6 seconds

In some cases forecast results are different for neighboring starting points

forecast_anticipation rolling forecast on the next 6 seconds

Some models are really accurate on the denoised series

forecast_anticipation rolling forecast on the next 6 seconds

We see that in some cases we have false positive

false_positive false positive, a spike is forecasted where there is none

If we analyze the maximum of the peak forecast we see that some models can forecast 3 seconds in advance

forecast_max maximum forecast on a rolling windows of 12 seconds

Using modem features we can have an earlier estimation

modem_foreAv latency maximum forecast on a 6 seconds, modem features

single series

We iterate over every single series and forecast from every single from_peak value and compute the maximum of the forecasted latency

forecast_series maximum latency forecast per series, deci seconds

We than calculate the forecast curves per series and per starting point and calculate the maximum for each curve. For some models the forecast returns many false positives

forecast_flat maximum forecast on a rolling windows of 6 seconds

Using modem features we can forecast earlier on average

modem_foreMax latency maximum forecast on modem features, single series

We than calculate the maximum latency per series per time_to_peak, we identify a peak setting a threshold

modem_truePos true positive rate, modem features

We show the tradeoff using different type of thresholds

modem_threshold true positive rate, dependence on threshold

derivative

Now we take the derivative of the signal

diff_series derivative of camera latency

And we train a model on the with all features derived

diff_truePos latency forecast on single series and starting point

We than calculate the accuracy depending on the threshold

diff_truePos accuracy of forecast depending on the threshold

We see that substantiall changing the size of the window doesn’t change much the results

diff_ahead forecast on different rolling windows

double history

We now use a double history points to forecast the next point

diff_double forecast on double history

Forecast on single series and rolling windos

double_foreSeries double forecast series

Accuracy on single series forecast

double_accuracy double forecast accuracy

feature knockout

Feature importance by substituing the feature by random values

diff_knock_out difference knock out

forecast validation

For the validation we take calendar week 39 and we perform a pre trained model. We check that the model deliver reliable forecasts on a time window of 6 seconds

rolling_forecast forecast on a rolling window of 6 seconds on a denoised series

were we used the simple non derivative single step model for sake of speed. We take the maximum of each forecast and build the forecast line

validation_performance maximum forecast on the rolling window

We than perform a forecast for each second of the week and compare its maximum with the signal maximum on a rolling window of 1, 6, 12 and 30 seconds

validation_series time series of camera latency considering the maximum of the next n steps

We than set a threshold and we count how many data points are above this threshod (spike) in a time bin of 4 seconds

confusion_3sec confusion matrix about the presence of spikes in the next 4 seconds

and compute the same for the next 30 seconds

confusion_30sec confusion matrix about the presence of spikes in the next 30 seconds

We see that the false negative rate is around 0.5% and that the model is pretty conservative since there are no false positive. That means that around 10% of the spikes are still undiscovered.

forecast alternatives

We started performing some naive forecasting on the time series and we compare the results of facebook prophet, a multi layer perceptron regressor and a arima. The first two models take into consideration the sensor features too.

We see that the peaks are really abrupt and the naive models can’t forecast it.

first_peak first peak