A few years ago, I looked at a fraud model that showed 98.5% accuracy. The team was excited, but when we checked the confusion matrix, we saw the model barely caught any fraud. Since fraud made up only 1.5% of the data, just predicting “not fraud” every time gave that high accuracy. This is the classic imbalanced data problem. Accuracy can look impressive, but the model misses the cases you care about, like fraud, disease, churn, or equipment failure. In this article, I’ll show you how to build a synthetic data pipeline to address this using SMOTE, ADASYN, CTGAN, and a few other methods, and how to evaluate the results honestly.
What Synthetic Oversampling Actually Does
Most models focus on the majority class because that’s where most examples are. Oversampling helps the minority class get more attention, and there are three main ways to do this:
- SMOTE picks a minority sample, finds its nearest minority neighbours, and creates new points along the lines between them.
- ADASYN works like SMOTE but generates more samples near hard-to-classify regions, where majority points surround minority points.
- CTGAN is a type of neural network that learns the distribution of the minority class and creates new samples from it. It can find more complex patterns, but it needs more data and takes longer to train.
I also use Borderline-SMOTE and SMOTE-Tomek. SMOTE-Tomek first oversamples, then removes unclear majority points near the class boundary.
Synthetic Data Pipeline for Imbalanced Datasets
Step 1: Setup, Data and Class Distribution
pip install scikit-learn imbalanced-learn ctgan pandas numpy
import numpy as np, pandas as pd
from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import precision_score, recall_score, f1_score, average_precision_score
from imblearn.over_sampling import SMOTE, ADASYN, BorderlineSMOTE
from imblearn.combine import SMOTETomek
from imblearn.pipeline import Pipeline
from ctgan import CTGAN
X, y = make_classification(
n_samples=10000, n_features=12, n_informative=6,
weights=[0.97, 0.03], flip_y=0.01, class_sep=0.8, random_state=42
)
print("Full data:", Counter(y))Full data: Counter({np.int64(0): 9656, np.int64(1): 344})
Always look at the class distribution before you start. For example, if the minority class is only 3% of your data, you know what challenge you’re facing.
Step 2: Split First, Then Preprocess
This is where most data leakage happens. If you resample or scale before splitting, information from your test set leaks into your training set, and your metrics become unreliable:
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42
)
scaler = StandardScaler().fit(X_train) # fit on train ONLY
X_train_s = scaler.transform(X_train)
X_test_s = scaler.transform(X_test)
print("Train:", Counter(y_train), "| Test:", Counter(y_test))Train: Counter({np.int64(0): 7725, np.int64(1): 275}) | Test: Counter({np.int64(0): 1931, np.int64(1): 69})
Keep the test set untouched so it keeps its real, imbalanced distribution. It should match your production data, not your training set.
Step 3: Generate Synthetic Data
samplers = {
"Baseline (no resampling)": None,
"SMOTE": SMOTE(random_state=42),
"ADASYN": ADASYN(random_state=42),
"Borderline-SMOTE": BorderlineSMOTE(random_state=42),
"SMOTE-Tomek": SMOTETomek(random_state=42),
}
datasets = {}
for name, sampler in samplers.items():
if sampler is None:
datasets[name] = (X_train_s, y_train)
else:
datasets[name] = sampler.fit_resample(X_train_s, y_train)
# CTGAN: learn the minority distribution, then sample from it
minority = pd.DataFrame(X_train_s[y_train == 1]).add_prefix("f")
ctgan = CTGAN(epochs=300, verbose=False)
ctgan.fit(minority)
needed = int((y_train == 0).sum() - (y_train == 1).sum())
synthetic = ctgan.sample(needed).values
X_ct = np.vstack([X_train_s, synthetic])
y_ct = np.concatenate([y_train, np.ones(needed, dtype=int)])
datasets["CTGAN"] = (X_ct, y_ct)Remember, each sampler should only use the training data. If your dataset has categorical columns, use SMOTENC instead of SMOTE, since regular SMOTE mixes numbers and can create categories that don’t make sense.
Preparing for your first AI/ML interview?
It’s important to understand these concepts, but being able to explain them in interviews is what really counts. I covered the key ideas and practical interview tips in my book, Cracking Your First AI/ML Interview.
Step 4: Train and Evaluate the Right Way
Forget about accuracy for imbalanced problems. Instead, I focus on four metrics:
- Precision: of the cases flagged positive, how many really were?
- Recall: of the real positives, how many did we catch?
- F1-score: the balance between the two.
- PR-AUC: the area under the precision-recall curve, which is far more honest than ROC-AUC when positives are rare.
results = []
for name, (Xr, yr) in datasets.items():
model = RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1)
model.fit(Xr, yr)
proba = model.predict_proba(X_test_s)[:, 1]
pred = (proba >= 0.5).astype(int)
results.append({
"Method": name,
"Precision": precision_score(y_test, pred, zero_division=0),
"Recall": recall_score(y_test, pred),
"F1": f1_score(y_test, pred),
"PR-AUC": average_precision_score(y_test, proba),
})
print(pd.DataFrame(results).round(3).sort_values("PR-AUC", ascending=False))Method Precision Recall F1 PR-AUC
1 SMOTE 0.764 0.609 0.677 0.632
4 SMOTE-Tomek 0.764 0.609 0.677 0.632
2 ADASYN 0.672 0.594 0.631 0.597
3 Borderline-SMOTE 0.680 0.493 0.571 0.551
0 Baseline (no resampling) 0.792 0.275 0.409 0.506
5 CTGAN 0.302 0.232 0.262 0.250
You’ll notice a pattern: oversampling often increases recall but lowers precision, so your model finds more positives but also more false alarms. If PR-AUC doesn’t improve much, resampling might not be helping, and your features could be the issue. Sometimes, simple class weights or threshold tuning work better than advanced generators, so always try them as a baseline.
Step 5: Leakage-Proof Cross-Validation
When tuning or comparing models, place the resampler inside an imblearn pipeline so it only runs on each training fold:
pipe = Pipeline([
("scale", StandardScaler()),
("smote", SMOTE(random_state=42)),
("clf", RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1)),
])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="average_precision")
print(f"CV PR-AUC: {scores.mean():.3f} ± {scores.std():.3f}")CV PR-AUC: 0.694 ± 0.049
If you oversample before cross-validation, synthetic samples can end up in both the training and validation sets. This gives you great scores during testing, but leads to disappointment when you deploy the model.
When to Use What:
- SMOTE or Borderline-SMOTE are good default choices for numeric tabular data.
- ADASYN is helpful when the decision boundary is unclear, but it can also increase noise.
- CTGAN is worth using if you have enough minority samples—think thousands, not just dozens—and your data has complex, mixed-type features.
- Try class weights or threshold tuning first. They are simple and often work just as well.
The Takeaway
Synthetic data can’t add information that isn’t already in your minority class. What it can do is help your model notice rare events, but only if you evaluate it honestly.
The most important habit is straightforward: split your data first, resample only the training set, and evaluate your model on real-distribution data using PR-AUC. If you do this well, your models will perform in production, and that’s how good machine learning engineers build their reputation.
I hope you found this article on building a synthetic data pipeline for imbalanced datasets helpful. For more tips on AI and machine learning, you’re welcome to follow me on Instagram.
