time-series-analysis

Time Series Analysis

Overview

Time series analysis covers the end-to-end workflow for sequential temporal data: data preparation, stationarity testing, decomposition, autocorrelation analysis, model selection, temporally-correct cross-validation, and evaluation. Apply this skill for any dataset where observations are ordered in time and the temporal structure is scientifically meaningful.

When to Use This Skill

Use this skill when:

• The dataset has a datetime index and the ordering of observations matters
• You need to forecast future values of a variable
• You need to detect trends, seasonality, or change points
• You are performing anomaly detection on temporal data
• You need to evaluate a forecasting model with proper temporal splits
• You are fitting ARIMA, Prophet, LSTM, or any other time series model

---

Data Preparation

Datetime Index Setup

import pandas as pd

# Load and parse dates
df = pd.read_csv("data.csv", parse_dates=["timestamp"])
df = df.sort_values("timestamp").reset_index(drop=True)
df = df.set_index("timestamp")

# Verify index is datetime
print(df.index.dtype)         # should be datetime64[ns]
print(df.index.is_monotonic_increasing)  # must be True

Frequency Detection and Setting

# Infer frequency from data
inferred_freq = pd.infer_freq(df.index)
print(f"Inferred frequency: {inferred_freq}")
# Common codes: T (minute), H (hourly), D (daily), W (weekly), M (month-end), Q (quarter), A (annual)

# Explicitly set frequency (required by many statsmodels functions)
df = df.asfreq(inferred_freq)  # may introduce NaT rows for missing timestamps
print(df.index.freq)

Handling Missing Timestamps

# Check for gaps
full_index = pd.date_range(start=df.index.min(), end=df.index.max(), freq=inferred_freq)
missing_timestamps = full_index.difference(df.index)
print(f"Missing timestamps: {len(missing_timestamps)}")

# Reindex to fill in gaps, then interpolate
df = df.reindex(full_index)
n_missing = df["value"].isna().sum()
print(f"Missing values after reindex: {n_missing}")

# Interpolation options (choose based on expected pattern)
df["value"] = df["value"].interpolate(method="time")       # linear in time
# df["value"] = df["value"].interpolate(method="cubic")    # cubic spline
# df["value"] = df["value"].ffill()                        # forward fill (flat)
# df["value"] = df["value"].fillna(df["value"].rolling(7, min_periods=1).mean())  # rolling mean