Memory, strides and contiguity

This is the deepest lesson in the module and the one that pays off latest. It answers a question you'll otherwise keep bumping into: how can transposing a one-gigabyte array be instant?

Strides

An array is a flat run of bytes plus some bookkeeping. The important piece of bookkeeping is strides: one number per axis saying how many bytes to jump to move one step along that axis.

For a (3, 4) array of int64, a.strides is (32, 8). Moving one column along is 8 bytes — the next element. Moving one row down is 32 bytes — past the four elements of the current row.

Finding an element is then pure arithmetic:

a[2, 1]  ->  base + 2*32 + 1*8  ->  byte 72

One multiply-add per axis. No searching, no pointer chasing. That's the speed from lesson 1, made concrete.

Why transpose is free

Here's the payoff. a.T doesn't move any data — it hands back a new view with the strides swapped, (8, 32). Walking "down a row" of the transpose now jumps 8 bytes instead of 32, which lands you on what used to be the next column. Same bytes, different reading rule, and the cost is independent of array size.

The same trick explains everything else that's free: basic slicing adjusts the starting offset and the strides; a[::2] just doubles a stride; np.broadcast_to sets a stride to zero so the same value is read over and over. That zero stride is exactly the hollow-cell trick from broadcasting.

C order and F order

When the last axis has the smallest stride, rows are stored together and the array is C-contiguous — NumPy's default, named after C. When the first axis has the smallest stride, columns are together and it's F-contiguous, the Fortran and MATLAB convention.

a.flags['C_CONTIGUOUS']     # True
a.T.flags['C_CONTIGUOUS']   # False
a.T.flags['F_CONTIGUOUS']   # True
np.asfortranarray(a)        # convert if a library demands it

When NumPy has to copy

Some operations need a genuine flat C-ordered run of bytes. If your array isn't laid out that way, NumPy has to make one:

a.ravel()      # a view when it can, a copy when it can't
a.flatten()    # always a copy
a.reshape(-1)  # view if possible, otherwise raises or copies

So a.ravel() on a C-contiguous array is free, and a.T.ravel() quietly copies. For twelve elements nobody cares. For a hundred million in a loop, it's the difference between a second and a minute — and np.ascontiguousarray once, outside the loop, is usually the fix.

Checking whether two arrays share memory

When you're not sure whether you're holding a view or a copy — which, after lesson 6, you will sometimes not be:

b.base is a                  # b was derived from a
np.shares_memory(a, b)       # exact, can be slow
np.may_share_memory(a, b)    # fast, conservative

You don't need to think about any of this most days. But when an operation is mysteriously slow, or a change appears somewhere you didn't expect, strides are almost always the explanation.