Work with a Time Series in pandas

A time series is a sequence of measurements taken over time: daily sales, hourly temperature, number of connections per minute. What sets it apart from an ordinary table is that the order of the rows carries information. You never shuffle the rows, and you always ask yourself: what is the frequency? Are there gaps?

Build a DatetimeIndex

The first step is to convert the date column and put it in the index:

import numpy as np
import pandas as pd

rng = np.random.default_rng(0)
df = pd.DataFrame({
    "date": pd.date_range("2024-01-01", periods=90, freq="D").astype(str),
    "ventes": rng.integers(50, 150, size=90),
})
df["date"] = pd.to_datetime(df["date"])   # text -> datetime64
df = df.set_index("date").sort_index()     # sorted time index

print(df.loc["2024-02"].head())            # the whole month of February
print(df.loc["2024-01-10":"2024-01-15"])   # a date range (bounds included)

With a DatetimeIndex, you also get useful attributes: df.index.day_of_week (0 = Monday; the old name dayofweek is deprecated since pandas 3.1), df.index.month, df.index.year.

Change frequency with resample

resample groups observations by period, a bit like a groupby on time. You must always specify the aggregation:

mensuel = df["ventes"].resample("ME").sum()      # total per month end
hebdo_moy = df["ventes"].resample("W").mean()    # mean per week

Choose the aggregation according to the nature of the measure: you sum volumes (sales, visits) and average levels (temperature, average stock). Since pandas 2.2, the month-end alias is "ME" ("MS" for month start); the old "M" is no longer accepted in pandas 3.

In the other direction, moving to a finer frequency (upsampling) creates gaps that must be filled, for example with .ffill() or .interpolate().

Rolling windows and shifts

A moving average smooths out noise to reveal the trend:

df["moy_7j"] = df["ventes"].rolling(window=7).mean()   # NaN on the first 6 rows
df["lag_1"] = df["ventes"].shift(1)                    # previous day's value
df["var_j"] = df["ventes"].diff()                      # ventes(t) - ventes(t-1)
df["var_pct"] = df["ventes"].pct_change()              # relative change

• rolling(7) looks at the last 7 values, the current value included. The first rows are NaN until the window is complete (the min_periods parameter relaxes this).
• shift(1) shifts downward: the January 2 row receives the January 1 value. This is the basic building block of the lags we will use for forecasting.
• diff() is equivalent to series - series.shift(1).

Spot the gaps

A “clean” series has a regular frequency. To check:

complet = pd.date_range(df.index.min(), df.index.max(), freq="D")
manquants = complet.difference(df.index)
df = df.reindex(complet)      # missing days appear as NaN

Making gaps explicit is better than ignoring them: a day with no sales (0) is not the same as a day with no data (NaN).

Key takeaways

A sorted time index, a known frequency, a suitable aggregation, and caution with windows: these four reflexes avoid most errors in the rest of the course.