SMOTE: Inventing Minority Examples Without Copying Them

What problem does this solve?

A visual inspection team on a packaging line has 40,000 good-part records and 310 defect records. Class weighting lifted recall from zero but the model still misses whole defect categories, because 310 examples spread across six defect types leaves some types with barely thirty rows.

The instinct is to duplicate the defect rows until the classes balance. That is random oversampling, and it teaches the model to memorise those exact 310 points rather than learn what a defect looks like. The decision boundary tightens around individual samples and collapses on anything slightly different.

SMOTE is the response to that specific failure: generate points that are new but plausible, sitting between real defects rather than on top of them.

How the solution works

For each minority sample, SMOTE finds its k nearest minority neighbours (k=5 by default), picks one at random, and creates a new point at a random position along the straight line joining them. Repeat until the class ratio reaches the target.

Because the synthetic point lies between two real examples, it inherits their general character without being identical to either. The minority region becomes denser and the classifier can draw a boundary around a region rather than around individual dots.

Variants exist for the cases plain SMOTE handles badly: SMOTENC for mixed categorical and continuous features, BorderlineSMOTE to concentrate on samples near the decision boundary, and ADASYN to weight generation toward the minority points that are hardest to classify.

1. 1
   Select a minority sample Take one real example from the under-represented class.
2. 2
   Find its neighbours Compute the k nearest minority-class neighbours in feature space, using Euclidean distance by default.
3. 3
   Pick one neighbour Choose one of those k neighbours at random.
4. 4
   Interpolate Generate a new point at x_new = x + rand(0,1) * (x_neighbour - x), landing somewhere on the segment between the two.
5. 5
   Repeat to target Continue until the minority class reaches the requested sampling ratio, then train on the enlarged training set.

Reference architecture

SMOTE belongs to exactly one layer. Most incorrect implementations come from placing it somewhere else.

Deployment options: The resampler is a training-time artefact and ships no inference-time dependency. What must ship is the pipeline definition, so a retrain six months later reproduces the same construction.

Key capabilities

Integrations

SMOTE is a step in a training pipeline, so its integration surface is the pipeline definition and the artefact store, not a runtime API.

Industry use cases

Implementation approach

1. 1
   Check the feature types Any categorical, ordinal or one-hot column disqualifies plain SMOTE. Use SMOTENC or encode deliberately.
2. 2
   Count the minority k_neighbors must be smaller than the minority count in the smallest fold, or fitting fails outright.
3. 3
   Build the pipeline Use imblearn's Pipeline so the sampler is confined to the training fold on every split.
4. 4
   Compare honestly Benchmark against class weighting alone. SMOTE frequently does not beat it, and that is a valid result.
5. 5
   Record the governance position Document what the synthetic records are derived from and where they may be stored.

Security & deployment

Synthetic minority records generated from personal data are derived personal data. Store them inside the same boundary as the source, exclude them from any dataset shared for benchmarking, and delete them with the training artefacts. Where the minority class is very small, interpolation between two records can approximate a real individual closely enough to matter.

A worked example

Two defect records described by two measured features — surface deviation in millimetres and cycle temperature in degrees.

1. Point A. A real defect at (0.40 mm, 62 degrees).
2. Point B. Its nearest minority neighbour, a real defect at (0.60 mm, 70 degrees).
3. Draw lambda. Suppose the random draw gives lambda = 0.25.
4. Interpolate each feature. Surface: 0.40 + 0.25 * (0.60 - 0.40) = 0.45 mm. Temperature: 62 + 0.25 * (70 - 62) = 64 degrees.
5. Result. A synthetic defect at (0.45 mm, 64 degrees) — a plausible part that was never actually produced.

Now change the second feature to 'shift code' with values 1, 2 and 3. Interpolating between shift 1 and shift 2 yields shift 1.25, which does not exist. That single substitution is why SMOTENC exists and why plain SMOTE on categorical columns is a silent error rather than a loud one.

In code

The pipeline below is the correct construction. Using imblearn's Pipeline rather than sklearn's is deliberate: it applies the resampler to the training fold only, so cross-validation scores stay honest.

from sklearn.datasets import make_classification
from sklearn.model_selection import StratifiedKFold, cross_val_score, train_test_split
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline

X, y = make_classification(
    n_samples=5000, n_features=12, n_informative=5,
    weights=[0.985, 0.015], random_state=7,
)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=7,
)

# imblearn's Pipeline resamples the TRAIN fold only on each CV split.
# sklearn's Pipeline would leak synthetic points into the validation fold.
pipe = Pipeline([
    ("smote", SMOTE(k_neighbors=5, random_state=7)),
    ("clf", RandomForestClassifier(n_estimators=300, random_state=7)),
])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=7)
scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="average_precision")
print("PR-AUC per fold:", scores.round(3))
print("mean PR-AUC:", scores.mean().round(3))

# k_neighbors must be < the minority count in the SMALLEST training fold,
# otherwise fitting raises ValueError.
print("minority in train:", int((y_train == 1).sum()))

Five per-fold PR-AUC values and their mean. If the folds disagree widely, the minority class is too small for the result to be stable and the honest conclusion is that more labelled defects are needed, not a different sampler.

Diagnostic checks

• Confirm the sampler sits inside the cross-validation fold. If validation PR-AUC is far above test PR-AUC, it does not.
• Inspect a sample of synthetic rows by hand. Impossible values — fractional shift codes, negative durations, out-of-range categories — indicate the wrong variant.
• Compare against a class-weighted baseline. If SMOTE does not beat it, do not ship the extra complexity.
• Vary the random seed across several runs. Large swings mean the minority class is too small for a stable conclusion.
• Check whether synthetic points bridge across a genuine gap between two distinct minority sub-populations, creating a region that contains no real examples.

Limitations & prerequisites

• SMOTE increases minority density but adds no new information about the class.
• It ignores the majority class entirely when generating points, so synthetic examples can land inside majority territory and blur the boundary.
• Performance is sensitive to k_neighbors, the sampling ratio and the random seed.
• It cannot repair labelling error, feature weakness or measurement bias — and will faithfully amplify all three.

SMOTE against the alternatives

The honest comparison is against class weighting, not against doing nothing.

Try class weighting first; it is one parameter, cannot leak, and frequently closes most of the gap on its own.

Key takeaways

• SMOTE interpolates between real minority neighbours rather than duplicating rows.
• It must run inside the training fold; before the split it produces an invalid score.
• Interpolation requires continuous, meaningfully ordered features — categorical columns need SMOTENC.
• Benchmark it against class weighting honestly; often weighting alone is enough.
• Synthetic minority records inherit the data classification of the records they came from.

FAQ

Related topics & solutions

• Class Imbalance: When 95% Accuracy Means the Model Found Nothing
• ADASYN: Generating Where the Model Is Actually Struggling
• Random Oversampling: Duplication as a Deliberate Choice
• Data Leakage: The Model That Only Works Before Deployment
• AI Products Portfolio
• Label Noise: The Accuracy Ceiling Nobody Put There Deliberately
• Outliers: Telling a Broken Sensor From a Broken Chiller
• Missing Values: Why the Gap Itself Carries Information