Regression algorithms编辑历史

# Regression Algorithms

## 1 Introduction

### 1.1 Definition and scope

Regression algorithms are a class of supervised learning methods used to model the relationship between one or more independent variables (features) and a continuous dependent variable (target). They are fundamental to predictive analytics, time series forecasting, and scientific modeling across many domains. Common examples include linear regression, decision tree regression, and support vector regression, each with distinct assumptions and use cases. The goal of regression is to learn a function that minimizes the error between predicted and actual values, often quantified by loss functions such as mean squared error.

### 1.2 Historical context

The origins of regression trace back to the early 19th century with the work of Carl Friedrich Gauss and Adrien-Marie Legendre on the method of least squares. The term “regression” was coined by Francis Galton in the 1880s while studying the inheritance of traits, observing that extreme values tended to “regress” toward the mean. Throughout the 20th century, regression evolved from simple linear models into a broad family of algorithms incorporating regularization, non-linear transformations, and tree-based methods, driven by advances in statistics and computational power.

## 2 Core Algorithms

### 2.1 Linear Regression

Linear regression assumes a linear relationship between the independent variables and the target variable. It is the simplest and most interpretable regression model.

#### 2.1.1 Simple Linear Regression

Simple linear regression models the relationship between a single feature \( x \) and the target \( y \) as \( y = \beta_0 + \beta_1 x + \epsilon \), where \( \beta_0 \) is the intercept, \( \beta_1 \) is the slope, and \( \epsilon \) is the error term. The parameters are estimated by minimizing the sum of squared residuals.

#### 2.1.2 Multiple Linear Regression

Multiple linear regression extends the simple case to multi