Anomaly detection using Prophet and Altair

View/Fork on Kaggle

Packages-used: fbprophet and altair

Data Used:

Data on sunspots since 1749: Sunspots.txt

Using Prophet:

• Prophet expects data to have two columns

  • Timestamp/Date - ds

  • Variable to check for Anomaly/Forecasting - y

    

Fit and Predict:

def fit_predict_model(dataframe, interval_width = 0.99, changepoint_range = 0.8):
    m = Prophet(daily_seasonality = False, yearly_seasonality = False, 
                weekly_seasonality = False, 
                seasonality_mode = 'multiplicative', 
                interval_width = interval_width,
                changepoint_range = changepoint_range)
    m = m.fit(dataframe)
    forecast = m.predict(dataframe)
    forecast['fact'] = dataframe['y'].reset_index(drop = True)
    print('Displaying Prophet plot')
    fig1 = m.plot(forecast)
    return forecast
pred = fit_predict_model(clean_df)

Detecting Anomalies:

• The light blue boundaries in the above graph are yhat_upper and yhat_lower.
• If y value is greater than yhat_upper and less than yhat lower then it is an anomaly.

• Also getting the importance of that anomaly based on its distance from yhat_upper and yhat_lower.
  ```
  def detect_anomalies(forecast):
  forecasted = forecast[[‘ds’,’trend’, ‘yhat’, ‘yhat_lower’, ‘yhat_upper’, ‘fact’]].copy()

  forecast[‘fact’] = df[‘y’]

  forecasted[‘anomaly’] = 0
  forecasted.loc[forecasted[‘fact’] > forecasted[‘yhat_upper’], ‘anomaly’] = 1
  forecasted.loc[forecasted[‘fact’] < forecasted[‘yhat_lower’], ‘anomaly’] = -1

  anomaly importances

  forecasted[‘importance’] = 0
  forecasted.loc[forecasted[‘anomaly’] ==1, ‘importance’] = \

    (forecasted['fact'] - forecasted['yhat_upper'])/forecast['fact']

  forecasted.loc[forecasted[‘anomaly’] ==-1, ‘importance’] = \

    (forecasted['yhat_lower'] - forecasted['fact'])/forecast['fact']

  return forecasted

pred = detect_anomalies(pred)
```

Visualize the results: