Skip to main content

Gradient Descent

If you are familiar with maths, you might know that minimum or maximum value of a function is got by equating its derivative to zero. In other words, the value obtained when the slope(m) is zero considered as the min or max of that line(function).

Gradient descent is one approach to find the minimum value of cost function. The algorithm goes through the θ values step by step, in direction of negative slopes and stops at slope zero.
The above image from hackernoon explains this concept well. (Please note, the parameter representation here is w not θ).

Gradient descent algorithm is given by,

θj :=θj αθj J(θ0 ,θ1 )

α - Learning rate (Length of each step)
J(θ₀,θ₁) - Cost function
j - Iterator

The value of α plays a key role in determining the gradient descent. A smaller value of α will reduce the speed of computation. On the other hand, a larger value of α may skip the converging point(the situation is called as overshooting).

The screenshot from Andrew NG's course explains both issues perfectly.

In the equation,

the derivative part or slope will reduce and reaches zero at the minimum value. i.e,

θj:=θjα * 0 =>
𝜃 = 𝜃𝚥 => Minimum value.


Popular posts from this blog

Setting up Python Flask server on internet via Port forwarding

In this post we will cover topics;

Starting a server using Python Flask.Accessing server in Local Network.Adding a Port Forwarding rule in NAT settings.Accessing server in Internet via public IP.Difference between static IP and dynamic IP.

Step 1 :-

As the first step, we need to setup a server on our system.A server exposes your code to a particular port.Some examples are Apache Tomcat, Spring boot (inbuilt), Python Django (inbuilt), Nginx.
Here we have a simple server in Python Flask.

For a GET request in root path the server returns a "Hello World!".That's it.
Going to start the server on my local IP address( at default flask port(5000) using below commands,
$flask run  --host=

Step 2 :-

For security purpose all ports for external communication is closed by router's firewall by default. So no communication can be done to outside world. For same reason our server won't be available to other devices.
For our…

Automate Blog Post creation using Blogger APIs and Python

This article will discuss the following topics,

Blogger's REST APIs.Google's OAuth 2.0 playground.Accessing spreadsheet in Python.Creating blogger posts via Python.An experiment to automate blog post creation when you have structured data. I was searching for IFSC codes for some bank transactions and came across Reserve Bank's website, which has a list of excel files containing Bank Codes, Contact Info and related details of each bank's branch. This time I will be looking towards Bharat co-operative bank of Mumbai. Here is how the corresponding data looks like.
In the above picture, each row represents lots of details about each branch of the Bank. Let's think of creating a list of blog posts where each post gives IFSC codes of a particular branch along with address and contact details.

As the first step, I have created a blog in blogger. In the Blogger dashboard, Please also note the parameter called blog ID which is a kind of unique identifier to the blog. It is …

Creation of Orphan Process in Linux using Python

Hi all,

In this article we will learn about,
Orphan process in Linux/Unix.How orphan process is created ?Creation of orphan process using python.Usage of python modules - multiprocessing , subprocessOrphan Process :-

An orphan process is a child process whose parent got finished or terminated. After the termination of parent process, init process will adopt it.If you look into details, the parent pid(ppid) of an orphan will be 1 which is the pid of init. Sometimes orphan process is intentionally created for certain needs, like if you run an application and you don't want the process created by your application to be killed, even after closing the application.

Creation of Orphan process using Python :-

Here, we are going to create an orphan process through python code.
The logic behind the code is,

Create a process using python multiprocessing module and target function will be parent.
In parent function another process is spawned , making it's target function as child. Using su…