Logistic Regression with gradient descent
In this post we’re looking at my second favourite machine learning algorithm - logistic regression. This algorithm is incredibly useful: Imagine you’re a central planner and you have a dataset of, let’s say, 18 - 24 year olds. Prior to gathering this data you decided you would randomly allocate driver’s licences to half of these 18 - 24 year olds without reuiring them to sit their formal tests.
Gradient Descent, the smart way
In this blog post I want to document my version of a gradient descent algorithm. First we’ll take a look at the data, and fit a linear model to it to understand what we’re trying to get to. Remember we can solve for the \(\hat{\beta}\) matrix either through assuming the \(\epsilon\) are IID and solve for a closed form solution using a Maximum Likelihood Estimator (this is the Econometrics way).
So in my last post I described the maths and intuition of gradient descent. Now I want to go through how to implement gradient descent for a linear regression in R.
During the building phase for this post I ran through the gradient descent algorithm the “dumb” way just to cement in my own mind how it’s working. And I thought that process might actually be quite instructive for a blog post.
Intro to Gradient Descent
What’s going on here
This is my first Data Science post on my blog. In this post i’ll be exploring my understanding of the Gradient Descent algorithm. My next post will explore how to implement this algorithm in R. Then i’ll have a play around with the function so we can all see the results of some pretty cool maths.
The Gradient Descent algorithm is the first algorithm presented by Andrew NG in his Machine Learning course.
Welcome
Who’s Konrad, what is this blog and why am I here?
I have an educational background in economics and econometrics; as a naturally inquisitive person, after university I started reading things unrelated to my job. This lead me to finding this half new half old thing called Machine Learning . After reading through some of the basics I thought “hey some of this looks incredibly similar to econometrics, with some nomenclature differences.
