Also refer to the following table:
|
Iterative methods (SGD, LBFGS) |
Closed form Normal Equation |
|
Choosing learning Rate |
No parameter |
|
Iterations can be large |
Does not iterate |
|
Good performance on large feature sets |
Slow and impractical on large feature sets |
|
Error prone: getting stuck due to poor parameter selection |
(xTx)-1 is computationally expensive - in the order of n3 |
Here is a quick reference on configuration of the LinearRegression object, but be sure to see Chapter 5, Practical Machine Learning with Regression and Classification in Spark 2.0 - Part I and Chapter 6, Practical Machine Learning with Regression and Classification in Spark 2.0 - Part II for more details.
- L1: Lasso regression
- L2: Ridge regression
- L1 - L2: Elastic net in which you can adjust the dial
The following link is a write-up from Columbia University that explains normal equations as they relate to solving Linear Regression problems:
- http://www.stat.columbia.edu/~fwood/Teaching/w4315/Fall2009/lecture_11
- Octave ( https://www.gnu.org/software/octave/) from GNU is a popular matrix manipulation software and you should have it in your toolkit.
- The following link contains a quick tutorial to get you started: http://www.lauradhamilton.com/tutorial-linear-regression-with-octave