Optimisation
Machine Learning
Optimisation in machine learning is the process of adjusting model parameters to minimise (or maximise) an objective function. Gradient-based optimisation methods are the backbone of neural network training.
Understanding Optimisation
Optimisation in machine learning is the process of adjusting a model's parameters to minimise a loss function, effectively finding the best configuration that maps inputs to desired outputs. Gradient descent and its variants, including stochastic gradient descent, Adam, and AdaGrad, are the workhorses of neural network optimisation, iteratively updating parameters in the direction that reduces error. The optimisation landscape is filled with challenges like local minima, saddle points, vanishing gradients, and the need to balance learning rate schedules carefully. Beyond model training, optimisation encompasses hyperparameter tuning through techniques like grid search, Bayesian optimisation, and neural architecture search. In reinforcement learning, optimisation involves maximising cumulative rewards through policy gradient methods. The efficiency of optimisation directly determines how quickly models converge and how well they generalize, making it one of the most studied areas in deep learning research.
Related in Machine Learning
Accuracy
Accuracy is a metric that measures the proportion of correct predictions out of total predictions made by a model. While intuitive, accuracy can be misleading on imbalanced datasets where one class dominates.
Active Learning
Active learning is a machine learning approach where the model selectively queries an oracle (often a human) for labels on the most informative data points. This reduces the total amount of labelled data needed to train an accurate model.
Anomaly Detection
Anomaly detection is the identification of data points, events, or patterns that deviate significantly from expected behaviour. AI-based anomaly detection is used in fraud prevention, cybersecurity, and industrial monitoring.
AutoML
Automated Machine Learning (AutoML) is the process of automating the end-to-end pipeline of applying machine learning, including feature engineering, model selection, and hyperparameter tuning. AutoML democratizes AI by reducing the expertise required.
Bagging
Bagging (Bootstrap Aggregating) is an ensemble technique that trains multiple models on random subsets of training data and combines their predictions. Random Forest is the most well-known bagging-based algorithm.
Bayesian Network
A Bayesian network is a probabilistic graphical model that represents variables and their conditional dependencies using a directed acyclic graph. It enables reasoning under uncertainty and causal inference.
Bias-Variance Tradeoff
The bias-variance tradeoff is the fundamental tension in machine learning between model simplicity (high bias) and model flexibility (high variance). Optimal models balance underfitting and overfitting to generalize well to new data.
Binary Classification
Binary classification is a supervised learning task where the model assigns inputs to one of exactly two categories. Spam detection (spam vs. not spam) and medical diagnosis (positive vs. negative) are common examples.