Linear Regression Problem with single variable using Octave

From Deep_Learning_Machine_Learning_and_Artificial_Intelligence
Jump to: navigation, search

This is an exercise problem from Stanford machine learning class. The question can be found.

https://github.com/kaileepm/mlclass-ex/blob/master/mlclass-ex1/ex1.pdf

And framework for the exercise is also provided.

https://github.com/kaileepm/mlclass-ex/tree/master/mlclass-ex1

As a first step, let's try to load data, and plot the data

data = load('ex1data1.txt');

X = data(:, 1);
y = data(:, 2);
m = length(y);

plot(X, y, "k.");
xlabel('Population of City in 10,000s');
ylabel('Profit in $10,000s');

load() loads CSV formatted training examples to a matrix. First column is the feature (x1) and the second column is the target (y).

m uses length() to get the number of rows (same as number of training example pairs). plot() shows two dimensional figure of the samples.

function J = MyComputeCost(X, y, theta)
        % J(theta) = (1/2m) * sum((h(theta)(x(i)) - y(i)) ^ 2)
        % sum of (prediction - target) ^ 2
        % h(theta)(x) = theta * X
        m = length(y);
        prediction = X * theta;
        errors = (prediction - y);
        sqrErrors = errors .^ 2; % element-wise square
        sumSqrErrors = sum(sqrErrors);

        J = 1 / (2 * m) * sumSqrErrors;
endfunction
function theta_new = MyGradientDescentLMSOnce(X, y, theta, alpha)
        % adjust theta
        % theta(j) = theta(j) - alpha * (1/m) * sum((errors * xj)

        m = length(y);
        prediction = X * theta;
        errors = (prediction - y);

        % x0 = 1
        theta_0 = theta(1) - alpha * (1 / m) * sum(errors);
        % x1
        x1 = X(:, 2);
        theta_1 = theta(2) - alpha * (1 / m) * sum(errors .* x1); % element wise multiplication

        theta_new = [ theta_0; theta_1 ];
endfunction
function [theta_new, J_history] = MyGradientDescent(X, y, theta, alpha, iterations)
        for i = 1:iterations
                theta = MyGradientDescentLMSOnce(X, y, theta, alpha);
                J_history(i) = MyComputeCost(X, y, theta);
                % fprintf('iteration %d: theta = (%f, %f), cost = %f\n', i, theta(1), theta(2), J_history(i));
        end

        theta_new = theta;
endfunction

Full codes

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox