Linear and logistic regression
Linear regression fits a weighted sum of features to a continuous target. Logistic regression pushes that sum through a sigmoid to produce a probability. Both are fast, interpretable, and remain strong baselines you should always run first.
Trees and forests
A decision tree recursively splits the feature space to reduce impurity. Single trees overfit, so we ensemble: random forests average many decorrelated trees (bagging), which cuts variance dramatically.
Gradient boosting
Boosting fits trees sequentially, each one correcting the residual errors of the ensemble so far. XGBoost, LightGBM and CatBoost dominate structured/tabular problems and usually beat deep networks there.
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split
X_tr, X_va, y_tr, y_va = train_test_split(X, y, test_size=0.2, random_state=0)
model = GradientBoostingClassifier(n_estimators=300, learning_rate=0.05, max_depth=3)
model.fit(X_tr, y_tr)
print(model.score(X_va, y_va))