13 Section 13. Notes on ‘Ch 14. Introduction to regression models’

13.1.1 14.1 Conditional modeling

• question: how does one quantity \(y\) vary as a function of another quantity or vector of quantities \(x\)?
  • conditional distribution of \(y\) given \(x\) parameterized as \(p(y|\theta,x)\)
• key statistical modeling issues:
  1. defining \(y\) and \(x\) so that \(y\) is reasonably linear as a function of the columns of \(X\)
  • may need to transform \(x\)
  2. set priors on the model parameters

13.1.2 14.2 Bayesian analysis of classical regression

• simplest case: ordinary linear regression
  • observation errors are independent and have equal variance

\[ y | \beta, \sigma, X \sim \text{N}(X \beta, \sigma^2 I) \]

13.3.1 Identifiability and collinearity

• “the parameters in a classical regression cannot be uniquely estimated if there are more parameters than data points or, more generally, if the columns of the matrix \(X\) of explanatory variables are not linearly independent” (pg 365)

13.3.2 Nonlinear relations

• may need to transform variables
• can include more than one transformation in the model as separate covariates
• GLMs and non-linear models are discussed in later chapters

13.3.3 Indicator variables

• include a categorical variable in a regression using a indicator variable
  • separate effect for each category
  • or model as related with a hierarchical model

13.3.4 Interactions

• “If the response to a unit change in \(x_i\) depends in what value another predictor \(x_j\) has been fixed at, then it is necessary to include interaction terms in the model” (pg 367)
  • \((x_i - \bar{x_i})(x_j - \bar{x_j})\)