Multi-collineartity, Variance Inflation
and Orthogonalization in Regression


Chong Ho (Alex) Yu, Ph.D., D. Phil. (2022)

Analogy in variable space

Now let's use a vehicle as an analogy. A few years ago Chrysler introduced a model called Intrepid. In this model the four wheels are spread out so that the car is better-supported and thus stable. We can look at a regression model as an automobile model--it requires support and stability. Now let us examine the following regression model.

The following figure is a 3D plot of three hypothetical variables--X, Y, and Z, with a regression plane. The regression plane is a function defined by the data points. Geometrically, we can think of the plane as an object that is supported by the points, just like a car is supported by four wheels. There exists the threat of collinearity between Y and Z if the dots do not spread out in the X space enough to provide stable support for the plane. If the points are too close to each other, the model would collapse.

If the regression plane is too abstract to you, let's look at some physical objects. In the following pictures, the paper is the regression plane and the fingers are the data points. In the left picture my fingers spread out and thus the plane is well-supported. On the contrary, in the right picture my fingers are too close to each other. As a result, the plane tends to fall down.


Back


Menu


Next


Navigation

Index

Simplified Navigation

Table of Contents

Search Engine

Contact