ML Fundamentals
Training, inference, features, labels, loss functions, optimization
Machine Learning is a subfield of AI where systems learn patterns from data rather than being explicitly programmed. The fundamental workflow is: collect data → prepare features → choose a model → train → evaluate → deploy → monitor.
The key insight: instead of writing rules by hand, you show examples to an algorithm and it infers the rules itself.
Key Points
- Training: the process of fitting a model to a labelled dataset by minimising a loss function
- Inference: using a trained model to make predictions on new, unseen data
- Features (X): the input variables used to make predictions
- Labels (y): the target output the model tries to predict
- Loss Function: measures how wrong the model is (MSE for regression, cross-entropy for classification)
- Optimiser: algorithm that adjusts model parameters to minimise loss (SGD, Adam)
- Overfitting: model memorises training data, fails on new data → fix with regularisation, more data
- Underfitting: model too simple, cannot capture patterns → fix with more complex model or features
- Hyperparameters: settings set before training (learning rate, number of layers) — not learned from data
- Train/Val/Test split: typically 70/15/15 or 80/10/10 — test set used only once
| Concept | Analogy | Practical Example |
|---|---|---|
| Training data | Practice exam questions | 10,000 labelled cat/dog images |
| Features | Question attributes | Pixel values, edges, textures |
| Loss function | How wrong your answer is | Cross-entropy error on predictions |
| Optimiser | Study strategy to improve | Adam adjusting model weights |
| Overfitting | Memorising past exams | Model scores 99% train, 60% test |
Real-World Example
Spotify's song recommendation runs continuous ML training — every stream, skip, and playlist addition is a training signal. The model continually learns your taste and generalises to suggest songs you haven't heard yet.