Text with .str

Count the drinks in our table and you get this:

orders["item"].value_counts()
# masala     6
# ginger     4
# elaichi    2
# Ginger     1

Ginger sold 5, not 4. One capital letter split it in two, and nothing errored. This is what untidied text costs — not a crash, just a wrong number in a report that looks perfectly reasonable.

The .str accessor

String methods don't sit on the Series directly — they live behind .str:

orders["item"].lower()        # AttributeError
orders["item"].str.lower()    # correct

The accessor applies the method to every value at once and passes missing values through untouched rather than raising.

Normalising

orders["item"].str.strip().str.lower()

Those two together fix the overwhelming majority of text problems: invisible leading or trailing whitespace from a CSV, and inconsistent capitalisation. Do it once, on the way in, before anything groups or joins on that column.

Whitespace is the nastier of the two because it's invisible. "Delhi " and "Delhi" look identical in every output you'll ever print, and group separately forever.

Searching

s.str.contains("gin", na=False)      # substring
s.str.contains("gin", case=False)    # ignoring case
s.str.startswith("M") / .endswith("a")
s.str.match(r"^gin")                 # regex, anchored at the start
s.str.len()                          # length of each value

Each returns a mask, so they drop straight into the filtering you already know. The na=False matters: without it, a missing value produces NaN in the mask, and a mask containing NaN can't be used to index.

Splitting and extracting

orders["date"].str.split("-")                  # a Series of lists
orders["date"].str.split("-", expand=True)     # a DataFrame of columns
orders["date"].str.extract(r"(\d{4})-(\d{2})")  # one column per capture group

expand=True is the useful form — it gives you real columns you can assign straight back. extract is for when the structure is buried inside a longer string: an order id, a unit, a code embedded in a description.

Replacing

s.str.replace("-", "/", regex=False)      # a plain swap
s.str.replace(r"\s+", " ", regex=True)    # collapse runs of whitespace
s.str.replace(r"[^0-9.]", "", regex=True) # strip currency symbols

Always say regex= explicitly. The default changed between pandas versions, and being explicit means your code does the same thing everywhere.

A word on speed

.str methods loop in Python underneath, so they're far slower than numeric work — on a few million rows, noticeably so. The fix is the same shape as always: normalise once, then convert to category. After that, grouping and comparing are integer operations and the text is only stored once.

orders["item"] = orders["item"].str.strip().str.lower().astype("category")