Feature Scaling in Python

Machine learning models are trained on datasets where each feature can span a very different numerical range. Pixel brightness might range from 0 to 255, while income ranges from 0 to 1,000,000. When feature magnitudes differ by orders of magnitude, algorithms that rely on distances (k-NN, SVM, k-means) or gradient descent (logistic regression, neural networks) can give disproportionate weight to the largest features. Feature scaling brings all features into a comparable range so the model can learn from all of them equally.

This chapter explains why scaling matters, covers the four scalers available in scikit-learn, shows their outputs on real data, and demonstrates the correct way to integrate scaling into a machine-learning pipeline without leaking test data.

Why Feature Scaling Matters

Consider a simple dataset with two features — age (20–60) and income (20,000–100,000). The income column has values roughly 1,000 times larger than age. For algorithms that measure Euclidean distance, a one-unit change in income dominates a one-unit change in age completely.

Specific situations where scaling is essential:

• Distance-based algorithms — K-Nearest Neighbors and K-Means Clustering compute distances between data points. Unscaled features make the distance metric meaningless.
• Gradient-descent optimisers — Logistic Regression and Linear Regression converge much faster when all features live in similar ranges.
• Regularised models — Ridge, Lasso, and ElasticNet penalise large coefficients equally. A large-scale feature artificially attracts a small coefficient, hiding its true importance.

When you do NOT need scaling: tree-based models (Decision Trees, Random Forests, Gradient Boosting) split on individual feature thresholds. The absolute scale of a feature does not affect where the best split falls, so scaling has no effect on their accuracy.

The Four scikit-learn Scalers

StandardScaler

StandardScaler (also called z-score normalisation) transforms each feature to have a mean of 0 and a standard deviation of 1.

Formula per feature:

z = (x − mean) / std

This is the default choice when you have no strong reason to prefer another. It works well for normally distributed data and is expected by most regularised linear models.

from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
import numpy as np

data = load_iris()
X = data.data          # shape (150, 4)
y = data.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)   # fit + transform on training data
X_test_scaled  = scaler.transform(X_test)        # transform only (no re-fit)

# Inspect the result
print("Training set — mean per feature (should be ~0):")
print(np.round(X_train_scaled.mean(axis=0), 4))

print("\nTraining set — std per feature (should be ~1):")
print(np.round(X_train_scaled.std(axis=0), 4))

Output:

Training set — mean per feature (should be ~0):
[ 0. -0. -0. -0.]

Training set — std per feature (should be ~1):
[1. 1. 1. 1.]

Notice that fit_transform is called only on the training set. Calling it on the test set as well would leak test-set statistics into the scaler parameters — see the Data Leakage section below.

MinMaxScaler

MinMaxScaler rescales every feature to a fixed range, by default [0, 1].

Formula per feature:

x_scaled = (x − min) / (max − min)

Use MinMaxScaler when you need values in a bounded range — for example, feeding data into a neural network with sigmoid output units, or when the downstream algorithm expects non-negative inputs.

from sklearn.preprocessing import MinMaxScaler

mm_scaler = MinMaxScaler()   # default feature_range=(0, 1)
X_train_mm = mm_scaler.fit_transform(X_train)
X_test_mm  = mm_scaler.transform(X_test)

print("Min per feature (should be 0):", np.round(X_train_mm.min(axis=0), 4))
print("Max per feature (should be 1):", np.round(X_train_mm.max(axis=0), 4))

Output:

Min per feature (should be 0): [0. 0. 0. 0.]
Max per feature (should be 1): [1. 1. 1. 1.]

Gotcha: MinMaxScaler is very sensitive to outliers. A single extreme value compresses all other values into a narrow band at one end of the range. If your data contains outliers, consider RobustScaler instead.

RobustScaler

RobustScaler uses the median and the interquartile range (IQR) instead of the mean and standard deviation, making it resistant to outliers.

Formula per feature:

x_scaled = (x − median) / IQR

where IQR = Q3 − Q1 (75th percentile minus 25th percentile).

from sklearn.preprocessing import RobustScaler

rb_scaler = RobustScaler()
X_train_rb = rb_scaler.fit_transform(X_train)
X_test_rb  = rb_scaler.transform(X_test)

print("Median per feature (should be ~0):")
print(np.round(np.median(X_train_rb, axis=0), 4))

Output:

Median per feature (should be ~0):
[0. 0. 0. 0.]

RobustScaler is the right choice when your dataset contains outliers that you cannot or do not want to remove.

MaxAbsScaler

MaxAbsScaler divides every value by the maximum absolute value in the training set, placing each feature in the range [−1, 1] without shifting the data.

Formula per feature:

x_scaled = x / max(|x|)

This is the only scaler that leaves zero-centred data centred. It is specifically designed for sparse matrices (e.g., TF-IDF vectors from text) where shifting the mean would destroy sparsity.

from sklearn.preprocessing import MaxAbsScaler

ma_scaler = MaxAbsScaler()
X_train_ma = ma_scaler.fit_transform(X_train)
X_test_ma  = ma_scaler.transform(X_test)

print("Max absolute value per feature (should be 1):")
print(np.round(np.abs(X_train_ma).max(axis=0), 4))

Output:

Max absolute value per feature (should be 1):
[1. 1. 1. 1.]

Comparing All Four Scalers

The table below summarises when to use each scaler:

Avoiding Data Leakage

Data leakage happens when information from the test set influences the preprocessing step. The typical mistake is fitting the scaler on the entire dataset before splitting:

# WRONG — leaks test-set statistics into the scaler
scaler = StandardScaler()
X_all_scaled = scaler.fit_transform(X)           # uses test rows
X_train_bad, X_test_bad, _, _ = train_test_split(
    X_all_scaled, y, test_size=0.2, random_state=42
)

The correct order is always split first, then fit the scaler on training data only:

# CORRECT
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)   # fit on train only
X_test_scaled  = scaler.transform(X_test)        # apply the same transformation

For more on the train/test split itself, see Train/Test Split.

Using a Pipeline to Keep Scaling Safe

A scikit-learn Pipeline is the most robust way to combine scaling with a model. The pipeline guarantees that fit_transform is called on training data and only transform on any held-out fold — even during cross-validation.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_iris
import numpy as np

X, y = load_iris(return_X_y=True)

pipe = Pipeline(steps=[
    ("scaler", StandardScaler()),
    ("model",  LogisticRegression(max_iter=200)),
])

scores = cross_val_score(pipe, X, y, cv=5, scoring="accuracy")
print("CV accuracy scores:", np.round(scores, 3))
print("Mean accuracy:      ", round(scores.mean(), 3))

Output:

CV accuracy scores: [0.967 1.    0.933 0.9   1.   ]
Mean accuracy:       0.96

The pipeline eliminates the risk of data leakage across all five cross-validation folds automatically.

Hyperparameter Tuning the Scaler

You can tune scaler choice alongside model hyperparameters using Grid Search:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import load_iris
import numpy as np

X, y = load_iris(return_X_y=True)

pipe = Pipeline(steps=[
    ("scaler", StandardScaler()),
    ("model",  LogisticRegression(max_iter=300)),
])

param_grid = {
    "scaler": [StandardScaler(), MinMaxScaler(), RobustScaler()],
    "model__C": [0.1, 1.0, 10.0],
}

grid = GridSearchCV(pipe, param_grid, cv=5, scoring="accuracy")
grid.fit(X, y)

print("Best scaler:", grid.best_params_["scaler"].__class__.__name__)
print("Best C:     ", grid.best_params_["model__C"])
print("Best score: ", round(grid.best_score_, 3))

Output:

Best scaler: StandardScaler
Best C:      10.0
Best score:  0.973

This lets the data tell you which scaler works best for a given model and dataset.

Related Chapters

• Train/Test Split — split your data before fitting any scaler
• Categorical Data — encode non-numeric features before scaling
• K-Nearest Neighbors — distance-based model that requires scaled features
• K-Means Clustering — clustering algorithm sensitive to feature scale
• Linear Regression — gradient-descent model that converges faster with scaling
• Logistic Regression — regularised classifier that benefits from StandardScaler
• Cross Validation — use a Pipeline to ensure scaling is leak-free during CV
• Grid Search — tune scaler choice alongside model hyperparameters