Feature engineering is one of the steps that machine learning engineers spend most of their time on during model training. During interviews, many practical problems will be based on feature engineering to test your skills in preparing and selecting the right features for your model. So, in this article, I’ll take you through five popular feature engineering practical concepts for interviews with the most popular questions.
Feature Engineering Practical Interview Concepts
Below are five popular feature engineering practical concepts for interviews with the most popular questions.
Feature Encoding for Categorical Variables
Categorical features must be converted into a numeric form for machine learning models. The choice of encoding method (e.g., one-hot encoding, label encoding, or target encoding) depends on the type of categorical data (nominal or ordinal) and the machine learning algorithm being used.
Example Problem: You are working on a dataset with a Region column containing categories like North, South, East, and West. How will you encode this column for use in both tree-based models and linear models?
For tree-based models, use label encoding (numeric mapping of categories). For linear models, use one-hot encoding to avoid imposing an ordinal relationship. Here’s how to solve it using Python:
import pandas as pd
from sklearn.preprocessing import OneHotEncoder, LabelEncoder
# sample data
data = {'Region': ['North', 'South', 'East', 'West', 'North']}
df = pd.DataFrame(data)
# label encoding
label_encoder = LabelEncoder()
df['Region_Label'] = label_encoder.fit_transform(df['Region'])
# one-hot encoding
one_hot_encoder = OneHotEncoder(sparse_output=False)
encoded = one_hot_encoder.fit_transform(df[['Region']])
encoded_df = pd.DataFrame(encoded, columns=one_hot_encoder.get_feature_names_out(['Region']))
# combine original data with one-hot encoded columns
df = pd.concat([df, encoded_df], axis=1)
print(df)Region Region_Label Region_East Region_North Region_South Region_West
0 North 1 0.0 1.0 0.0 0.0
1 South 2 0.0 0.0 1.0 0.0
2 East 0 1.0 0.0 0.0 0.0
3 West 3 0.0 0.0 0.0 1.0
4 North 1 0.0 1.0 0.0 0.0
Handling Missing Values
Missing values can distort the model’s performance. Common approaches include mean/median imputation for numerical features and mode or a special category for categorical features. Advanced techniques include predictive imputation using machine learning models.
Example Problem: You have a Salary column with missing values. Simply filling missing values with the mean is causing model bias. How will you handle these missing values effectively?
Use predictive imputation by training a regression model to predict missing values based on other features. Here’s how to solve it using Python:
from sklearn.linear_model import LinearRegression
import numpy as np
# sample data
data = {'Age': [25, 30, 35, 40, 45], 'Salary': [50000, 60000, None, 80000, None]}
df = pd.DataFrame(data)
# separate rows with and without missing values
df_with_missing = df[df['Salary'].isnull()]
df_without_missing = df[df['Salary'].notnull()]
# train a regression model to predict missing values
model = LinearRegression()
model.fit(df_without_missing[['Age']], df_without_missing['Salary'])
predicted_salaries = model.predict(df_with_missing[['Age']])
# fill missing values
df.loc[df['Salary'].isnull(), 'Salary'] = predicted_salaries
print(df)Age Salary
0 25 50000.0
1 30 60000.0
2 35 70000.0
3 40 80000.0
4 45 90000.0
Feature Scaling for Numeric Variables
Scaling ensures that numeric features have comparable ranges, improving model convergence and performance. Methods include Standard Scaling (z-score), Min-Max Scaling, and Robust Scaling.
Example Problem: You are working on a dataset with numeric features like Age and Income. Income ranges from 10,000 to 1,000,000, while Age ranges from 18 to 80. How will you scale these features for models like Logistic Regression and KNN?
Use Standard Scaling (z-score normalization) for models like Logistic Regression and KNN to ensure all features are on the same scale. Here’s how to solve it using Python:
from sklearn.preprocessing import StandardScaler
# sample data
data = {'Age': [18, 25, 30, 45, 60],
'Income': [20000, 50000, 100000, 200000, 500000]}
df = pd.DataFrame(data)
# standardize features
scaler = StandardScaler()
scaled_features = scaler.fit_transform(df)
scaled_df = pd.DataFrame(scaled_features, columns=df.columns)
print(scaled_df)Age Income
0 -1.167023 -0.884648
1 -0.702866 -0.712314
2 -0.371325 -0.425091
3 0.623296 0.149356
4 1.617918 1.872697
Feature Interaction
Feature interactions capture non-linear relationships between features. Polynomial features (e.g., x1 * x2, x1^2) can significantly improve model performance if non-linear relationships.
Example Problem: You are working on a housing dataset where Size and Rooms are two numeric features. How will you create interaction features to capture their combined effect on Price?
Use PolynomialFeatures from sklearn to generate interaction terms and higher-order features. Here’s how to solve it using Python:
from sklearn.preprocessing import PolynomialFeatures
# sample data
data = {'Size': [1000, 1500, 2000], 'Rooms': [3, 4, 5]}
df = pd.DataFrame(data)
# generate polynomial features
poly = PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)
interaction_features = poly.fit_transform(df)
interaction_df = pd.DataFrame(interaction_features,
columns=poly.get_feature_names_out(['Size', 'Rooms']))
print(interaction_df)Size Rooms Size Rooms
0 1000.0 3.0 3000.0
1 1500.0 4.0 6000.0
2 2000.0 5.0 10000.0
Feature Selection
Feature selection reduces dimensionality by retaining only the most important features. Methods include Univariate Selection (ANOVA, Chi-square), Recursive Feature Elimination (RFE), and Embedded Methods (Lasso, Tree-based models).
Example Problem: You are working on a dataset with 50 features, but only a few are relevant. How will you identify and retain only the important features?
Use Recursive Feature Elimination (RFE) with a machine learning model like Logistic Regression or Random Forest to rank and select features. Here’s how to solve it using Python:
from sklearn.feature_selection import RFE
from sklearn.ensemble import RandomForestClassifier
# simulated data
np.random.seed(42)
X = np.random.rand(100, 50) # 50 features
y = np.random.choice([0, 1], size=100) # binary target
# feature selection with RFE
model = RandomForestClassifier()
rfe = RFE(estimator=model, n_features_to_select=10)
X_selected = rfe.fit_transform(X, y)
print("Selected Features (Indices):", rfe.get_support(indices=True))Selected Features (Indices): [ 1 10 14 16 19 29 34 37 38 42]
Summary
So, below are five popular feature engineering practical concepts for interviews:
- Encoding categorical variables for different models.
- Handling missing values without introducing bias.
- Scaling numeric features for model stability.
- Capturing feature interactions for non-linear relationships.
- Selecting the most important features for model performance.
I hope you liked this article on feature engineering practical concepts for interviews. Feel free to ask valuable questions in the comments section below. You can follow me on Instagram for many more resources.
