org.apache.spark.mllib.regression

Class LinearRegressionWithSGD

Object

org.apache.spark.mllib.regression.GeneralizedLinearAlgorithm<LinearRegressionModel>

org.apache.spark.mllib.regression.LinearRegressionWithSGD

All Implemented Interfaces:

java.io.Serializable, Logging

public class LinearRegressionWithSGD
extends GeneralizedLinearAlgorithm<LinearRegressionModel>
implements scala.Serializable

Train a linear regression model with no regularization using Stochastic Gradient Descent. This solves the least squares regression formulation f(weights) = 1/n ||A weights-y||^2^ (which is the mean squared error). Here the data matrix has n rows, and the input RDD holds the set of rows of A, each with its corresponding right hand side label y. See also the documentation for the precise formulation.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
LinearRegressionWithSGD() Deprecated. Use ml.regression.LinearRegression or LBFGS. Since 2.0.0.

Method Summary

Modifier and Type	Method and Description
static int	getNumFeatures()
static boolean	isAddIntercept()
GradientDescent	optimizer() The optimizer to solve the problem.
static M	run(RDD<LabeledPoint> input)
static M	run(RDD<LabeledPoint> input, Vector initialWeights)
static GeneralizedLinearAlgorithm<M>	setIntercept(boolean addIntercept)
static GeneralizedLinearAlgorithm<M>	setValidateData(boolean validateData)
static LinearRegressionModel	train(RDD<LabeledPoint> input, int numIterations) Train a LinearRegression model given an RDD of (label, features) pairs.
static LinearRegressionModel	train(RDD<LabeledPoint> input, int numIterations, double stepSize) Train a LinearRegression model given an RDD of (label, features) pairs.
static LinearRegressionModel	train(RDD<LabeledPoint> input, int numIterations, double stepSize, double miniBatchFraction) Train a LinearRegression model given an RDD of (label, features) pairs.
static LinearRegressionModel	train(RDD<LabeledPoint> input, int numIterations, double stepSize, double miniBatchFraction, Vector initialWeights) Train a Linear Regression model given an RDD of (label, features) pairs.

Methods inherited from class org.apache.spark.mllib.regression.GeneralizedLinearAlgorithm

getNumFeatures, isAddIntercept, run, run, setIntercept, setValidateData

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

LinearRegressionWithSGD

public LinearRegressionWithSGD()

Deprecated. Use ml.regression.LinearRegression or LBFGS. Since 2.0.0.

Construct a LinearRegression object with default parameters: {stepSize: 1.0, numIterations: 100, miniBatchFraction: 1.0}.

Method Detail

train

public static LinearRegressionModel train(RDD<LabeledPoint> input,
                                          int numIterations,
                                          double stepSize,
                                          double miniBatchFraction,
                                          Vector initialWeights)

Train a Linear Regression model given an RDD of (label, features) pairs. We run a fixed number of iterations of gradient descent using the specified step size. Each iteration uses miniBatchFraction fraction of the data to calculate a stochastic gradient. The weights used in gradient descent are initialized using the initial weights provided.

Parameters:

input - RDD of (label, array of features) pairs. Each pair describes a row of the data matrix A as well as the corresponding right hand side label y

numIterations - Number of iterations of gradient descent to run.

stepSize - Step size to be used for each iteration of gradient descent.

miniBatchFraction - Fraction of data to be used per iteration.

initialWeights - Initial set of weights to be used. Array should be equal in size to the number of features in the data.

Returns:

(undocumented)

train

public static LinearRegressionModel train(RDD<LabeledPoint> input,
                                          int numIterations,
                                          double stepSize,
                                          double miniBatchFraction)

Train a LinearRegression model given an RDD of (label, features) pairs. We run a fixed number of iterations of gradient descent using the specified step size. Each iteration uses miniBatchFraction fraction of the data to calculate a stochastic gradient.

Parameters:

input - RDD of (label, array of features) pairs. Each pair describes a row of the data matrix A as well as the corresponding right hand side label y

numIterations - Number of iterations of gradient descent to run.

stepSize - Step size to be used for each iteration of gradient descent.

miniBatchFraction - Fraction of data to be used per iteration.

Returns:

(undocumented)

train

public static LinearRegressionModel train(RDD<LabeledPoint> input,
                                          int numIterations,
                                          double stepSize)

Train a LinearRegression model given an RDD of (label, features) pairs. We run a fixed number of iterations of gradient descent using the specified step size. We use the entire data set to compute the true gradient in each iteration.

Parameters:

input - RDD of (label, array of features) pairs. Each pair describes a row of the data matrix A as well as the corresponding right hand side label y

stepSize - Step size to be used for each iteration of Gradient Descent.

numIterations - Number of iterations of gradient descent to run.

Returns:

a LinearRegressionModel which has the weights and offset from training.

train

public static LinearRegressionModel train(RDD<LabeledPoint> input,
                                          int numIterations)

Train a LinearRegression model given an RDD of (label, features) pairs. We run a fixed number of iterations of gradient descent using a step size of 1.0. We use the entire data set to compute the true gradient in each iteration.

Parameters:

input - RDD of (label, array of features) pairs. Each pair describes a row of the data matrix A as well as the corresponding right hand side label y

numIterations - Number of iterations of gradient descent to run.

Returns:

a LinearRegressionModel which has the weights and offset from training.

getNumFeatures

public static int getNumFeatures()

isAddIntercept

public static boolean isAddIntercept()

setIntercept

public static GeneralizedLinearAlgorithm<M> setIntercept(boolean addIntercept)

setValidateData

public static GeneralizedLinearAlgorithm<M> setValidateData(boolean validateData)

run

public static M run(RDD<LabeledPoint> input)

run

public static M run(RDD<LabeledPoint> input,
                    Vector initialWeights)

optimizer

public GradientDescent optimizer()

Description copied from class: GeneralizedLinearAlgorithm

The optimizer to solve the problem.

Specified by:

optimizer in class GeneralizedLinearAlgorithm<LinearRegressionModel>

Returns:

(undocumented)