pandas·Capstone·20 min·0/5 exercises

Capstone: a time series report

Is trade growing, and what does a normal week look like?

DatetimeIndexresamplerollingpct_changeseasonalitypartial periods

Watch it happen

Play it through, or step back and forth yourself.

1
Index by dateset_index().sort_index()
lesson 21
2
Find the rhythmgroupby(index.day_name())
lessons 14 and 21
3
Smooth.rolling(7).mean()
lesson 22
4
Change the grain.resample("W").sum()
lesson 22
5
Compare periods.pct_change(7)
lesson 23
6
Reportagg · nlargest · assert
lessons 13 and 12

Thirty-five days of takings. The question is the one every operations report asks: is trade growing, and what does a normal week look like? Both answers are hiding under the day-to-day noise.

The idea

sales is thirty-five days of cups sold, indexed by date. Two questions, both obscured by noise: is trade growing, and what does a normal week look like?

1. Make sure the index is right

sales.index                    # a DatetimeIndex already
sales.index.is_monotonic_increasing
sales.index.freq

Everything below needs a sorted DatetimeIndex. If yours came from a CSV, that's pd.to_datetime then set_index then sort_index, in that order.

Also check for missing days. A gap in a daily series isn't a NaN — the row simply isn't there, so isna() finds nothing:

full = pd.date_range(sales.index.min(), sales.index.max(), freq="D")
full.difference(sales.index)      # the days that never arrived

2. Find the weekly rhythm

sales.groupby(sales.index.day_name()).mean()

This is the "normal week" answer. Weekends are busier here — and knowing that changes how you read everything else, because a Monday dip isn't a decline, it's a Monday.

3. Smooth to see the trend

sales.rolling(7).mean()

A 7-day window is the natural choice for daily data with a weekly cycle: each point averages exactly one of every weekday, so the rhythm cancels out and only the trend remains.

Remember the first six values are NaN. If you plot this next to the raw series, the smoothed line starts a week late — that's correct, not a bug.

4. Change the grain for reporting

sales.resample("W").sum()

And the trap from lesson 22: the buckets follow the calendar. The first and last are almost always partial, so they look like a collapse in trade at both ends of every chart you make. Either drop them or say so:

weekly = sales.resample("W").sum()
weekly.iloc[1:-1]        # complete weeks only

5. Compare like with like

Here's the trap that produces a confidently wrong answer. sales.pct_change() compares each day with the one before — so in data with a weekly rhythm it mostly measures the rhythm. Saturday is always up on Friday, and that tells you nothing about growth.

sales.pct_change()      # measures the weekly cycle
sales.pct_change(7)     # compares each day with the same day last week

pct_change(7) is the everyday version of seasonal adjustment, and it's one argument. Use it whenever the data has a cycle shorter than the trend you're looking for.

6. Report, and assert

best = sales.nlargest(3)
week_on_week = sales.pct_change(7).mean()

assert sales.index.is_monotonic_increasing
assert sales.index.is_unique

The through-line

All three traps in this capstone share a shape: they produce plausible numbers rather than errors. A partial bucket looks like a bad week. A leading NaN looks like missing data. A day-on-day change looks like volatility. Nothing raises, and the chart looks fine.

Which is the argument for checking the ends, the length, and the period — every time.

See it run

The lesson's code, ready to run and to fiddle with.

Putting the kettle on…

Starting up…

Worked example

not graded

Already written and ready to go — press Run to see what it does, then change a number, a column name, anything, and run it again.

tryweekly.iloc[1:-1] to keep only the complete weeks.

Press Run — the output appears here.

Your turn

5 exercises. Write the code yourself, then press Check — a nudge and the answer are there if you want them.

Return the average cups per weekday name — the "normal week" answer.

your answer

Return the 7-day rolling mean of sales, which cancels the weekly rhythm and leaves the trend.

your answer

Return the weekly totals with the partial first and last buckets dropped — complete weeks only.

your answer

Return the average week-on-week change — each day compared with the same day last week. The starter compares with yesterday, which measures the weekly rhythm instead.

your answer

Return the three busiest days in sales, largest first.

your answer