Gradient Descent part 1

July 3, 2018 in R, machine learning, intuition

Intro to Gradient Descent What’s going on here This is my first Data Science post on my blog. In this post i’ll be exploring my understanding of the Gradient Descent algorithm. My next post will explore how to implement this algorithm in R. Then i’ll have a play around with the function so we can all see the results of some pretty cool maths. The Gradient Descent algorithm is the first algorithm presented by Andrew NG in his Machine Learning course.

Konrad Hurren

Perpetual apprentice; economist; strong opinions, loosely held

Professional economist; hobby data scientist

New Zealand

no post found

Logistic regression - Gradient Descent

Aug 8, 2018

Logistic Regression with gradient descent In this post we’re looking at my second favourite machine learning algorithm - logistic regression. This algorithm is incredibly useful: Imagine you’re a central planner and you have a dataset of, let’s say, 18 - 24 year olds. Prior to gathering this data you decided you would randomly allocate driver’s licences to half of these 18 - 24 year olds without reuiring them to sit their formal tests.

Gradient Descent The Smart Way

Jul 7, 2018

Gradient Descent, the smart way In this blog post I want to document my version of a gradient descent algorithm. First we’ll take a look at the data, and fit a linear model to it to understand what we’re trying to get to. Remember we can solve for the \(\hat{\beta}\) matrix either through assuming the \(\epsilon\) are IID and solve for a closed form solution using a Maximum Likelihood Estimator (this is the Econometrics way).

Gradient Descent part deux (or part duh)

Jul 7, 2018

So in my last post I described the maths and intuition of gradient descent. Now I want to go through how to implement gradient descent for a linear regression in R. During the building phase for this post I ran through the gradient descent algorithm the “dumb” way just to cement in my own mind how it’s working. And I thought that process might actually be quite instructive for a blog post.