Linear Prediction Models

Linear models

Linear prediction models assume that there is a linear relationship between the independent variables and the dependent variable. Therefore, these models exhibit high bias and low variance.

The high bias of these models is due to the assumption of nonlinearity. If this assumption does not sufficiently represent the data, then linear models will be inaccurate.

On the other hand, linear models also have a low variance. This means that if several linear models would be trained using different data, they would perform similarly on the same test data set. This is because linear models are inflexible because there are few parameters to be tuned.

Thus, linear models are interpretable and robust. However, if their assumptions are not met, they willl perform poorly.

When do use linear models?

Linear models excel under the following circumstances:

  • There are few data available, which would lead to overfitting with more complex models.
  • There are indications for a linear association between features and outcome.
  • Interpretation rather than predictive performance alone is important.

Posts about linear models

The following posts deal with linear models for prediction.

Interpreting Generalized Linear Models

Interpreting Generalized Linear Models

0

Generalized linear models (GLMs) are related to conventional linear models but there are some important differences. For example, GLMs are based on the deviance rather than the conventional residuals and they enable the use of different distributions and linker functions. This post investigates how these aspects influence the interpretation of GLMs.

Finding a Suitable Linear Model for Ozone Prediction

Finding a Suitable Linear Model for Ozone Prediction

0

Although ordinary least-squares regression is often used, it is not appropriate for all types of data. Using the airquality data set, I try to find a generalized linear model that fits the data better. For this purpose, I use the following methods: weighted regression, Poisson regression, and imputation.