Data and Data Quality
Most model failures are data failures wearing a modelling costume. This section covers class imbalance and the resampling methods around it, leakage, label noise, missing values and outliers, the four biases that quietly shape a dataset, and the several distinct ways data shifts once a model is live.
ADASYN: Generating Where the Model Is Actually Struggling
ADASYN generates synthetic minority examples like SMOTE, but distributes them unevenly: it creates more points around minority samples that are surrounded by majority neighbours, and fewer around minority…
Class Imbalance: When 95% Accuracy Means the Model Found Nothing
Class imbalance is when one class holds far more training examples than another — 950 legitimate transactions against 50 fraudulent ones, for instance. A model trained on that…
Data Leakage: The Model That Only Works Before Deployment
Data leakage is when information that will not be available at prediction time is present during training. The model learns from it, the validation set contains the same…
Duplicate Data: When One Patient Becomes Four Records
Duplicate data is the same real-world entity represented more than once. In a hospital that is rarely an exact copy: it is one patient registered four times across…
Label Noise: The Accuracy Ceiling Nobody Put There Deliberately
Label noise is incorrect or inconsistent ground truth in the training data. It matters because a supervised model can only be as consistent as the labels it was…
Missing Values: Why the Gap Itself Carries Information
Missing values are absent entries in an otherwise structured dataset. What matters is not how many there are but why they are absent. If a temperature sensor drops…
Outliers: Telling a Broken Sensor From a Broken Chiller
An outlier is an observation far from the rest of the distribution. In machine learning the useful question is never whether a point is unusual, but why. A…
Random Oversampling: Duplication as a Deliberate Choice
Random oversampling copies existing minority-class rows at random until the class ratio reaches a chosen target. It creates no new information — every duplicated row is byte-identical to…
Random Undersampling: Throwing Away Data on Purpose
Random undersampling discards majority-class rows at random until the class ratio reaches a chosen target. Unlike oversampling it makes the training set smaller and faster, and it introduces…
Selection Bias: When Your Data Only Contains the People Who Showed Up
Selection bias is the gap between the population you want to describe and the population your data actually contains. It is not noise and it does not shrink…
SMOTE: Inventing Minority Examples Without Copying Them
SMOTE creates new minority-class examples by interpolating between an existing minority point and one of its nearest minority neighbours, placing a synthetic point somewhere along the line between…
Need this for a live project?
Tell us the environment, data and constraints — we scope a technical assessment or POC with our engineers in Dubai.
Request Technical Assessment