Learning types & models
Supervised vs unsupervised vs reinforcement learning?
Supervised learning trains on labelled data to predict outputs (spam detection). Unsupervised learning finds structure in unlabelled data (clustering customers). Reinforcement learning learns from reward feedback through trial and error (game-playing agents).
Regression vs classification?
Regression predicts a continuous value (house price); classification predicts a discrete category (spam or not). The output type decides which algorithms and metrics you use.
Decision tree vs random forest?
A decision tree splits data on features to reduce impurity but overfits easily. A random forest trains many trees on random subsets and averages them, cutting variance and generalising better.
Fitting & generalisation
Overfitting vs underfitting - how do you spot each?
Overfitting memorises training noise: high training accuracy but poor validation/test scores (high variance). Underfitting is too simple to capture the pattern: poor scores everywhere (high bias). The train-versus-validation gap tells you which you have.
Explain the bias-variance trade-off.
Simple models have high bias (they miss patterns); complex models have high variance (they chase noise). Total error is minimised at the balance point, which is what model selection and regularisation aim for.
L1 vs L2 regularisation?
Both penalise large weights to curb overfitting. L1 (Lasso) can drive weights to exactly zero, doing feature selection; L2 (Ridge) shrinks weights smoothly without eliminating them.
Why split into train/validation/test, and what is cross-validation?
Separate sets prevent you from evaluating on data the model saw, which would leak and inflate scores. K-fold cross-validation rotates the validation fold to give a more reliable estimate when data is limited.
Evaluation
Accuracy vs precision vs recall vs F1?
Accuracy is the fraction correct, but it misleads on imbalanced data. Precision is how many predicted positives are truly positive; recall is how many actual positives you caught; F1 is their harmonic mean, balancing the two.
What is a confusion matrix and ROC-AUC?
A confusion matrix breaks predictions into true/false positives and negatives, from which precision and recall are computed. ROC-AUC measures how well the model ranks positives above negatives across all thresholds.