ML Algorithm Cheatsheet

How to choose the right algorithm — decision guide by problem type, data size, and accuracy needs

Choosing the right ML algorithm is one of the most common challenges in practice. The answer depends on four key factors: what kind of problem you have (classification, regression, clustering, anomaly detection), how much labelled data you have, speed vs accuracy trade-off, and linearity of your data.

This cheatsheet condenses the decision logic from the Microsoft Azure ML Algorithm Cheat Sheet — a widely used reference for algorithm selection.

Key Points

  • Start with the simplest algorithm that could work — complexity is not always better
  • No single "best" algorithm — try 2-3 candidates and compare on a validation set
  • Data size matters: small data → simpler models; large data → deep learning or boosting
  • Linearly separable data → logistic regression / linear SVM; non-linear → tree-based or neural net
  • Accuracy vs speed: Random Forest > Logistic Regression in accuracy but slower to train
  • Interpretability needed → Decision Tree, Logistic Regression, Linear Regression
  • Tabular data → Gradient Boosting (XGBoost/LightGBM) almost always wins competitions
  • Image / audio / text → Deep Learning (CNN, Transformer)
  • No labels available → Unsupervised (K-Means, DBSCAN, PCA, Autoencoders)
  • Rare events or fraud → Anomaly Detection (Isolation Forest, One-Class SVM)
ML Algorithm Selection Guide START Are you looking for rare events or outliers? (fraud · defects · network intrusions) YES Anomaly Detection Isolation Forest One-Class SVM · Autoencoder Local Outlier Factor (LOF) NO Do you have labelled training data? (supervised vs unsupervised learning) YES NO Unsupervised K-Means · DBSCAN PCA · t-SNE Hierarchical Clustering Predict a category or a numeric value? (classification vs regression) Category Value Classification 2-class → Logistic Regression Multi-class → Random Forest Text / NLP → BERT / Naive Bayes Images → CNN High accuracy → XGBoost / SVM Regression Linear → Linear Regression Non-linear → Decision Forest Complex → Neural Network Tabular → XGBoost / LightGBM Time series → LSTM / Prophet Data Size Guidance < 1K rows Simple models Logistic / Linear Reg 1K – 100K rows Tree-based models XGBoost · RF · SVM 100K – 1M rows Gradient Boosting LightGBM · CatBoost > 1M rows Deep Learning Neural Networks Based on Microsoft Azure ML Algorithm Cheat Sheet · learn.microsoft.com

Algorithm selection flowchart — based on the Microsoft Azure ML Cheat Sheet

Problem TypeBest AlgorithmsWhen to UseAvoid When
Binary ClassificationLogistic Regression, SVM, XGBoostSpam/not-spam, churn yes/no, fraud yes/noOutput needs to be a continuous value
Multi-class ClassificationRandom Forest, XGBoost, Neural Network3+ categories: sentiment, topic, digit recognitionOnly 2 classes — use binary instead
RegressionLinear Regression, XGBoost, Neural NetworkPrice, temperature, demand forecastingTarget is a category not a number
ClusteringK-Means, DBSCAN, HierarchicalCustomer segmentation, topic discovery, no labelsYou have labels — use classification instead
Anomaly DetectionIsolation Forest, One-Class SVM, AutoencoderFraud, defects, network intrusions — rare eventsNormal events are as rare as anomalies
RecommendationCollaborative Filtering, Matrix FactorisationProduct, movie, content recommendationsNo user interaction history exists
Time SeriesARIMA, LSTM, Prophet, LightGBMDemand forecast, stock prices, sensor dataData has no temporal dependency
NLP / TextTF-IDF + LR, BERT, TransformerSentiment, NER, classification, generationData is not text/sequence based
Image / VisionCNN, ResNet, ViT (Vision Transformer)Object detection, image classification, OCRDataset is too small (< few thousand images)

Real-World Example

Kaggle competition winners almost always use XGBoost or LightGBM for structured/tabular data, and fine-tuned Transformers for text/image tasks. The Microsoft Azure ML Cheat Sheet is the go-to reference for enterprise teams deciding which algorithm to try first — it has been downloaded millions of times.

←PreviousML Pipeline & MLOps