programming assignment week 3 practice lab logistic regression

• 🌐 All Sites
• _APDaga DumpBox
• _APDaga Tech
• _APDaga Invest
• _APDaga Videos
• 🗃️ Categories
• _Free Tutorials
• __Python (A to Z)
• __Internet of Things
• __Coursera (ML/DL)
• __HackerRank (SQL)
• __Interview Q&A
• _Artificial Intelligence
• __Machine Learning
• __Deep Learning
• _Internet of Things
• __Raspberry Pi
• __Coursera MCQs
• __Linkedin MCQs
• __Celonis MCQs
• _Handwriting Analysis
• __Graphology
• _Investment Ideas
• _Open Diary
• _Troubleshoots
• _Freescale/NXP
• 📣 Mega Menu
• _Logo Maker
• _Youtube Tumbnail Downloader
• 🕸️ Sitemap

Coursera: Machine Learning (Week 3) [Assignment Solution] - Andrew NG

Recommended Machine Learning Courses: Coursera: Machine Learning    Coursera: Deep Learning Specialization Coursera: Machine Learning with Python Coursera: Advanced Machine Learning Specialization Udemy: Machine Learning LinkedIn: Machine Learning Eduonix: Machine Learning edX: Machine Learning Fast.ai: Introduction to Machine Learning for Coders

• ex2.m - Octave/MATLAB script that steps you through the exercise
• ex2 reg.m - Octave/MATLAB script for the later parts of the exercise
• ex2data1.txt - Training set for the first half of the exercise
• ex2data2.txt - Training set for the second half of the exercise
• submit.m - Submission script that sends your solutions to our servers
• mapFeature.m - Function to generate polynomial features
• plotDecisionBoundary.m - Function to plot classifier's decision boundary
• [*] plotData.m - Function to plot 2D classification data
• [*] sigmoid.m - Sigmoid Function
• [*] costFunction.m - Logistic Regression Cost Function
• [*] predict.m - Logistic Regression Prediction Function
• [*] costFunctionReg.m - Regularized Logistic Regression Cost
• Video - YouTube videos featuring Free IOT/ML tutorials

plotData.m :

Sigmoid.m :, costfunction.m :, check-out our free tutorials on iot (internet of things):.

predict.m :

Costfunctionreg.m :, 61 comments.

how could you do this please explain me...

What explanation you want? Please be more specific.

How can i download these files?

You can copy the the code from above code sections.

Hi Akshay, Please may I have theses files as well: ex2.m ex2 reg.m ex2data1.txt ex2data2.txt submit.m mapFeature.m plotDecisionBoundary.m

You can get those files from Coursera assignments. I don't have those with me now.

can you please tell me what you did by this grad = (1/m)* (X'*(h_x-y));

its the simplified version of derivative term d/d0*j0 which we call gradient. check the formula once and you will understand it

this means:- take the transpose of feature matrix X(i.e X') and multiply it with the difference of matrices h_x and y i.e the matrix with sigmoid outputs and the result matrix(y). Finally multiply the end product with 1/m , where m is the number of training examples. This is the vectorized implementation of the code that's actually way more lengthier to implement using loops.

Hi, can you please explain the predict function?

In this gradient decent the number of iteration are not specified so how is the gradient decent working? can someone please explain?

I used the exact code at the end but I'm still getting 65/100 not able to figure out the reason

Did you figure out the reason yet?

Hi !! why didn't you use sum() function for grad even why formula contains that ?

sum() is used for the summation in the formula. But here while coding for grad computation: grad = (1/m)* (X'*(h_x-y)); Here We are doing matrix multiplication which itself consist of "sum of product". So, no need of external sum function. Please try to do it on paper by yourself, you will get clear idea. Thanks

we have learned that Z= theta transpose X then why are using Z=X multiplied by theta in the above codes ?

When we are calculating z(small z) for a single sample, then it is z=theta' * x. (here small x) But When you do the same computation for all the samples at the same time then we call it as Z (Capital Z). Z = X * theta. (Here Capital X) Try to do it using pen-paper, you will get clear understanding.

Hii, thanks for your help mr. Akshay. I had this one doubt about predict.m function: I tried coding for predict function in the following way: h_x = sigmoid(X*theta); if (0<=h_x<0.5) p=0; elseif (0.5<=h_x<=1) p=1; endif I know I did it in a long way but the accuracy that I am getting 60.00. Your code gave me the accuracy 89.00. Can you please help me understand what's wrong with this and what's the exact difference between your code and mines'?

P is a matrix with dimensions m x 1. Solution: You can put your code in a "for" loop and check the value of each element in h_x and accordingly set the value of each element in p. It will work.

hey bro it says z not defined why???

Hi, I think you are doing this assignment in Octave and that's why you are facing this issue. Chethan Bhandarkar has provided solution for it. Please check it out: https://www.apdaga.com/2018/06/coursera-machine-learning-week-2.html?showComment=1563986935868#c4682866656714070064 Thanks

I have copy the exact code for plotData.m , and all the others program worked very well but I am still getting 70/100. Can you tel what's the problem ?

Can you tell me , how can I run "ex2" script in console ?

hi I want to clarify few things from you, I have read in regression, these are few important points which have not been covered in andrew ng regression topic, how to find how significant your variable is, significance of p value and R^2 (R-square) values. I would like to know more about them. kindly share some sources.

HI, The line code reg_term = (lambda/(2*m)) * sum(theta(2:end).^2); in costFunctionReg function, can you explain more about this part theta(2:end) , what does it mean and how did you deduce it,

sir,please explain me predict.m function I used for i=1:size(X,1) if sigmoid(X*theta)>=0.5 p=sigmoid(X*theta); end as well as, h_x = sigmoid(X*theta); for i=1:size(X,1) if (0<=h_x<0.5) p=0; elseif (0.5<=h_x<=1) p=1; end but i am getting 40 accuracy it is working only with your code.why sir?

Hi there, I am trying the the same code as yours of sigmoid() function but each time it is getting an error saying that 'z' undefined near line 6 column 18 error: called from sigmoid at line 6 column 5 what to do please help me out..

Hello Akshay, It'd be great if you kindly share the code for "fminunc" in this week's files(wherever needed), coz i don't understand that particular function well, neither did i get its solution anywhere else on internet.

Hi Ankit, Sorry but I don't have the code for "fminunc".

grad(2:end) = (1/m)* (X(:,2:end)'*(h_x-y))+(lambda/m)*theta(2:end); can u please explain this..

Hey it says my plot is empty can someone help?

I am facing this type of problem in matlab , what can i do ? how to fix that n where ?? 'fminunc' requires Optimization Toolbox. Error in ex2 (line 99) fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

In sigmoid error in line 6 (the preallocated value assigned to variable 'g' might be unused) what should i do

How's value of 'g' is unused. 'g' is nothing but output of sigmoid function. If you are getting some msg, it must be warning not error. So, don't worry about it, keep it as it is. (But I don't think you should get any kind of warning like this). line 6, is called initialization of variable.

Hi Akshay can you please explain why we use this X(:,2:end) and theta(2:end) instead of plain X and theta??

It's because as per the theory in videos, We don't apply regularization on theta_0. Regularization is applied from theta_1 onwards. and that's why 2 gradients. 1st corresponding to theta_0 and other for theta_1 onwards.

And also why use two gradents?

Good day sir, im new in this course...i could not fully understand the assignment in week 3...as i enter my code...i think still in error..

please explain the predict function

Predict function is fairly simple. You have implemented your gradient and now you just have to predict whether the answer will be 1 or 0... So, what will you do is check for the result > 0.5. If it is above the 0.5, then prediction will be true (1), otherwise false (0)

@Hassan Ashas Thank you very much for your explanation.

costfuntion is not returning the scalar value, it is returning the 1*100 matrix.

Opening and closing brackets are not matching you code. NOTE: check the brackets are "2*m" YOUR CODE: reg_term = (lambda/2*m)) * sum(theta(2:end).^2); WORKING CODE: reg_term = (lambda/(2*m)) * sum(theta(2:end).^2);

Hello Akshay, While computing cost function I am getting so many outputs

You should only get [J, grad] as a output of costFunction & costFunctionReg.

Error - theta may not be defined , predict function

hi i have a doubt i took theta as [zeros(n+1),1] it is giving me 0 and i cant submit the assignment can you specify initial value of theta and theta and values of X. i am totally confused

nothing is working here every time it is showing >> plotData error: 'y' undefined near line 14 column 12 error: called from plotData at line 14 column 5 >>

J = (1 / m) * sum ((- y. * Log (h_x)) - ((1-y). * Log (1-h_x))) the log representation in this equation means ln isn't it? So, shouldn't we write it as log (1-h_x) / log (10).

I made it this way: function [J, grad] = costFunctionReg(theta, X, y, lambda) %COSTFUNCTIONREG Compute cost and gradient for logistic regression with regularization % J = COSTFUNCTIONREG(theta, X, y, lambda) computes the cost of using % theta as the parameter for regularized logistic regression and the % gradient of the cost w.r.t. to the parameters. % Initialize some useful values m = length(y); % number of training examples % You need to return the following variables correctly J = 0; grad = zeros(size(theta)); % ====================== YOUR CODE HERE ====================== % Instructions: Compute the cost of a particular choice of theta. % You should set J to the cost. % Compute the partial derivatives and set grad to the partial % derivatives of the cost w.r.t. each parameter in theta [J, grad] = costFunction(theta, X, y); feats = theta(2:end); J = J + lambda / (2 * m) * (feats' * feats); grad(2:end) = grad(2:end) + lambda / m * feats; % ============================================================= end

My question is about the solved subroutine 'plotDecisionBoundary.m' Line 20 : plot_y I didn't understand the definition of this Infact how this particular code helped to plot the decision boundary! Please explain..

so in cost function grad is basically you doing gradient descent right? but what is the use of 1/m? i'm really confused sorry

While calculating cost function, we are doing sum (summation) operation over 'm' samples. And then dividing it by 'm' in order to scale the output (as a scaling factor).

Muje 55 marks hi aa rahe he mane code bhi sahi likha he phir bhi...logistic regression cost and regularised logistic regression gradient dono me 0 marks he..

i really confused in assignment, i enjoyed all the stuff that prof.Ng doing buat why it turns out to become nightmare when im face the programming assignment? In the cosfunctionreg.m why you put grad(1) = (1/m)* (X(:,1)'*(h_x-y)); whats this mean? grad(2:end) = (1/m)* (X(:,2:end)'*(h_x-y))+(lambda/m)*theta(2:end); what grad(2:end) mean?

These 2 lines are for calcuating gradient with regularization. since we don't add regularization term to 1st entry. (we have to write 2 seperate lines of code for it)

Hi dear Akshay. I'm trying to submit week 3 assignment but I keep seeing the error: !! Submission failed: unexpected error: Error: File: costFunctionReg.m Line: 22 Column: 3 Invalid expression. Check for missing or extra characters. Can you help me out?

I am getting a syntax error in exercise "CostfunctionReg.m" at grad(1) = (1/m)* (X(:,1)'*(h_x-y)); please tell me why am i getting this error. yes i am running it in octave but please don't tell me to go through the another link . please just tell me the issue.

!! Submission failed: Index exceeds array bounds. Function: getResponse LineNumber: 132of submitWithConfiguration

Here in the cost function though y and log(h_x) both have the same dimensions (mx1), how the dot product is possible between them?

We are not doing the dot product of y and log(h_x) while calculating cost function. Multiplication represented by dot astrix (.*) means element wise multiplication in matlab. Eg. -y.*log(h_x) Please check the code once again.

Our website uses cookies to improve your experience. Learn more

Contact form

Week 3 - Logistic Regression

Download template here

The following chunk will set up your document. Run it, then ignore it.

If the system prompts you to install a package, or gives you a “package not found” error, simply run install.packages("packagename") once to install it.

The data set

We will be using the flights data set from the nycflights13 package. nycflights13 is an R data package containing all out-bound flights from NYC.

We will build a classification model that sees if any given flight is delayed or not. Furthermore, let us trim down the number of variables we are working with. Lastly, let us select to only work with flights taken place during the first month.

now that we have performed some cleaning, will we proceed to perform a train-test split.

we will use the training data set for visual exploratory data analysis to reinforce the idea that we don’t touch the testing data set.

We can look at many things with this data set. What we want to look at is how any of the variables relate to delay .

We see a varied amount of flights throughout the day. This makes sense, no one wants to leave early or late from the airport.

We see some strong artifacts in the time of scheduled departure. Most flights leave on a multiple of 5 which we confirm below.

By combining hour and minute we can look at how much the different flights have departure delays. There are some really long delays in here!

If we color the points by delay we see that it appears that most of the delayed arrivals happen because of a delayed departure.

Let’s begin with a logistic model. We will look at how dep_delay and distance affects delay .

Our first step is to establish which model(s) we want to try on the data.

For now, this is just a logistic model.

To establish the model, we need to determine which R package it comes from (the “engine”) and whether we are doing regression or classification .

(These functions come from the tidymodels package that we loaded in the setup chunk.)

Next, we will fit the model to our data:

Let’s check out the output of this model fit:

the coefficients are shown as log-odds terms. We could also get this information using tidy()

setting exponentiate = TRUE , gives us the odds instead of log-odds.

We can also take a look at how well the model is doing. By using augment() we can generate predictions, and conf_mat() and autoplot() let us create a confusion matrix and visualize it.

Experiment with using some of the other predictors in your model. Are the answers surprising? Evaluate your models with conf_mat() and accuracy() .

Once you have a model you like, predict on the test data set and calculate the performance metric. Compare to the performance metrics you got for the training data set.

Instantly share code, notes, and snippets.

mGalarnyk / machineLearningWeek3Quiz1.md

• Download ZIP
• Star ( 2 ) 2 You must be signed in to star a gist
• Fork ( 2 ) 2 You must be signed in to fork a gist
• Embed Embed this gist in your website.
• Share Copy sharable link for this gist.
• Clone via HTTPS Clone using the web URL.
• Learn more about clone URLs
• Save mGalarnyk/a890c3d6bb42923cb0543c7d7809cb77 to your computer and use it in GitHub Desktop.

Machine Learning Week 3 Quiz 1 (Logistic Regression) Stanford Coursera

Github repo for the Course: Stanford Machine Learning (Coursera) Quiz Needs to be viewed here at the repo (because the image solutions cant be viewed as part of a gist)

chukwurah498 commented Sep 3, 2020

There ought to be an explanation why you felt these are answers. While I agree on some , there must be explanation as the instructor does.

Sorry, something went wrong.

Coursera Machine Learning

Coursera machine learning by prof. andrew ng, machine learning by prof. andrew ng.

Table of Contents

Breif intro, video lectures index, programming exercise tutorials, programming exercise test cases, useful resources, extra information.

• Online E-Books

Aditional Information

The most of the course talking about hypothesis function and minimising cost funtions

A hypothesis is a certain function that we believe (or hope) is similar to the true function, the target function that we want to model. In context of email spam classification, it would be the rule we came up with that allows us to separate spam from non-spam emails.

Cost Function

The cost function or Sum of Squeared Errors(SSE) is a measure of how far away our hypothesis is from the optimal hypothesis. The closer our hypothesis matches the training examples, the smaller the value of the cost function. Theoretically, we would like J(θ)=0

Gradient Descent

Gradient descent is an iterative minimization method. The gradient of the error function always shows in the direction of the steepest ascent of the error function. Thus, we can start with a random weight vector and subsequently follow the negative gradient (using a learning rate alpha)

Differnce between cost function and gradient descent functions

Bias and Variance

When we discuss prediction models, prediction errors can be decomposed into two main subcomponents we care about: error due to “bias” and error due to “variance”. There is a tradeoff between a model’s ability to minimize bias and variance. Understanding these two types of error can help us diagnose model results and avoid the mistake of over- or under-fitting.

Source: http://scott.fortmann-roe.com/docs/BiasVariance.html

Hypotheis and Cost Function Table

Regression with Pictures

• Linear Regression
• Logistic Regression

https://class.coursera.org/ml/lecture/preview

https://www.coursera.org/learn/machine-learning/discussions/all/threads/m0ZdvjSrEeWddiIAC9pDDA

https://www.coursera.org/learn/machine-learning/discussions/all/threads/0SxufTSrEeWPACIACw4G5w

https://www.coursera.org/learn/machine-learning/resources/NrY2G

Week 1 - Due 07/16/17:

• Welcome - pdf - ppt
• Linear regression with one variable - pdf - ppt
• Linear Algebra review (Optional) - pdf - ppt
• Lecture Notes

Week 2 - Due 07/23/17:

• Linear regression with multiple variables - pdf - ppt
• Octave tutorial pdf
• Programming Exercise 1: Linear Regression - pdf - Problem - Solution
• Program Exercise Notes

Week 3 - Due 07/30/17:

• Logistic regression - pdf - ppt
• Regularization - pdf - ppt
• Programming Exercise 2: Logistic Regression - pdf - Problem - Solution

Week 4 - Due 08/06/17:

• Neural Networks: Representation - pdf - ppt
• Programming Exercise 3: Multi-class Classification and Neural Networks - pdf - Problem - Solution

Week 5 - Due 08/13/17:

• Neural Networks: Learning - pdf - ppt
• Programming Exercise 4: Neural Networks Learning - pdf - Problem - Solution

Week 6 - Due 08/20/17:

• Advice for applying machine learning - pdf - ppt
• Machine learning system design - pdf - ppt
• Programming Exercise 5: Regularized Linear Regression and Bias v.s. Variance - pdf - Problem - Solution

Week 7 - Due 08/27/17:

• Support vector machines - pdf - ppt
• Programming Exercise 6: Support Vector Machines - pdf - Problem - Solution

Week 8 - Due 09/03/17:

• Clustering - pdf - ppt
• Dimensionality reduction - pdf - ppt
• Programming Exercise 7: K-means Clustering and Principal Component Analysis - pdf - Problems - Solution

Week 9 - Due 09/10/17:

• Anomaly Detection - pdf - ppt
• Recommender Systems - pdf - ppt
• Programming Exercise 8: Anomaly Detection and Recommender Systems - pdf - Problems - Solution

Week 10 - Due 09/17/17:

• Large scale machine learning - pdf - ppt

Week 11 - Due 09/24/17:

• Application example: Photo OCR - pdf - ppt
• Linear Algebra Review and Reference Zico Kolter
• CS229 Lecture notes
• CS229 Problems
• Financial time series forecasting with machine learning techniques
• Octave Examples

Online E Books

• Introduction to Machine Learning by Nils J. Nilsson
• Introduction to Machine Learning by Alex Smola and S.V.N. Vishwanathan
• Introduction to Data Science by Jeffrey Stanton
• Bayesian Reasoning and Machine Learning by David Barber
• Understanding Machine Learning, © 2014 by Shai Shalev-Shwartz and Shai Ben-David
• Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman
• Pattern Recognition and Machine Learning, by Christopher M. Bishop

Course Status

• What are the top 10 problems in deep learning for 2017?
• When will the deep learning bubble burst?

Statistics Models

• HMM - Hidden Markov Model
• CRFs - Conditional Random Fields
• LSI - Latent Semantic Indexing
• MRF - Markov Random Fields
• SIGIR - Special Interest Group on Information Retrieval
• ACL - Association for Computational Linguistics
• NAACL - The North American Chapter of the Association for Computational Linguistics
• EMNLP - Empirical Methods in Natural Language Processing
• NIPS - Neural Information Processing Systems