Stacking tables with concat

merge matches rows by a key. concat doesn't match anything — it stacks tables that are already the same shape. Two months of the same export, a folder of files read in a loop, results from several runs.

pd.concat([march, april])              # one under the other
pd.concat([march, april], ignore_index=True)

Trap one: the labels come along

Both inputs bring their own index, so a plain concat gives you repeated labels — 0, 1, 2, 0, 1, 2. Nothing warns you, and then:

combined.loc[0]            # returns TWO rows
combined.index.is_unique   # False

ignore_index=True renumbers from zero. Use it whenever the original labels were just row numbers, which is nearly always. If they meant something — dates, ids — keep them and check uniqueness deliberately.

Trap two: columns align too

Concat two frames with different columns and you get the union, with NaN wherever a frame didn't have that column. No error. This is index alignment from lesson 2, applied to the column axis.

It's how a typo in one file's header quietly doubles your column count — revenue and Revenue become two columns, each half full.

pd.concat([a, b])                  # union of columns, gaps become NaN
pd.concat([a, b], join="inner")    # only columns present in ALL of them

Compare the column lists before you concat, and the row count after. Both are one line and both catch this.

Sideways

pd.concat([a, b], axis=1)

This puts frames side by side, matching rows on the index. Useful when the index means something — two metrics for the same dates, say. Dangerous when it doesn't: with plain row numbers you're pairing unrelated rows by position, and you'll get a table that looks fine and says nothing true.

If you want to match on a column, that's merge, not concat.

Labelling the sources

pd.concat([march, april], keys=["march", "april"])

keys= adds an outer index level recording which frame each row came from — so you can still tell them apart afterwards, and .loc["march"] gets one back. That creates the MultiIndex you'll meet in lesson 20.

Building a table from many files

frames = [pd.read_csv(f) for f in files]
combined = pd.concat(frames, ignore_index=True)

Collect into a list and concat once. Concatenating inside the loop — combined = pd.concat([combined, new]) — copies the whole accumulated frame every iteration, which turns a linear job into a quadratic one.