The stack
NumPy provides the array; pandas adds labelled tables; matplotlib and seaborn plot; scikit-learn models; Jupyter explores. Polars and DuckDB have become the fast alternatives for larger-than-memory or performance-critical work.
Vectorise
Loops over rows or .apply() are slow because each call re-enters the Python interpreter. Express operations over whole columns, use groupby aggregations, set correct dtypes (category for low-cardinality strings), and avoid chained assignment which silently writes to a copy.
import pandas as pd
df = pd.read_parquet("orders.parquet")
df["day"] = df["created_at"].dt.floor("D")
df["country"] = df["country"].astype("category")
daily = (df.loc[df["status"].eq("paid")]
.groupby(["day", "country"], observed=True)
.agg(revenue=("total_cents", "sum"),
buyers=("customer_id", "nunique"))
.reset_index())
daily["revenue"] = daily["revenue"] / 100Reproducibility
Pin dependencies in a lockfile, seed random number generators, keep notebooks for exploration and move anything recurring into tested modules. A notebook that only runs top-to-bottom on one laptop is not an analysis anyone can trust.